Intelligent operation system and method for urban rail transit equipment

Abstract

The present application discloses an intelligent operation system and method for urban rail transit equipment, which uses machine vision technology and image intelligent analysis technology to conduct in-depth learning and analysis of track status images collected along the track, thereby characterizing and capturing the multi-scale state characteristics of the track, and based on the multi-scale state characteristics, realizes automated visual disease detection, so as to alleviate the limitations of traditional manual track detection and further realize efficient, safe and economical operation of rail transit.

Description

Technical Field

[0001] The present application relates to the field of smart operation, and more specifically, to a smart operation system and method for urban rail transit equipment. Background Art

[0002] With the acceleration of urbanization, urban rail transit, as an important part of urban public transportation, has become increasingly important for safe and efficient operation. However, the maintenance and inspection of rail transit facilities face many challenges, including facility complexity, low inspection efficiency, and limitations of manual inspection.

[0003] More specifically, in track inspection, the method of relying on manual inspection is not only inefficient, but also easily affected by human factors such as fatigue and lack of concentration, resulting in instability in the inspection results. In addition, with the continuous expansion of rail transit lines, the cost of relying on manual inspection for track inspection is getting higher and higher, and it is difficult to achieve scale and automation. Therefore, an optimized solution is expected. Summary of the invention

[0004] In order to solve the above technical problems, this application is proposed. The embodiments of this application provide an intelligent operation system and method for urban rail transit equipment, which uses machine vision technology and image intelligent analysis technology to deeply study and analyze the track state images collected along the track, thereby characterizing and capturing the multi-scale state characteristics of the track, and based on the multi-scale state characteristics, realizes automated visual disease detection, so as to alleviate the limitations of traditional manual track detection and further realize efficient, safe and economical operation of rail transit.

[0005] According to one aspect of the present application, there is provided an urban rail transit equipment intelligent operation system, which includes:

[0006] A track status image real-time acquisition module is used to obtain the track status image acquired by the camera;

[0007] A multi-scale foreground feature processing module, used for performing multi-scale foreground feature extraction and feature enhancement on the track state image to obtain an enhanced track state shallow feature map, an enhanced track state middle feature map and an enhanced track state deep feature map;

[0008] A cross-scale interaction module, used for cross-scale interaction of the enhanced track state shallow feature map, the enhanced track state middle feature map and the enhanced track state deep feature map to obtain a track state cross-space multi-scale representation feature vector;

[0009] An operation result generating module, used for determining an operation result based on the track state cross-spatial multi-scale representation feature vector, wherein the operation result is used for indicating whether there is a visible disease on the track;

[0010] The maintenance response module is used to generate a track maintenance prompt in response to the operation result that there is a visible disease on the track.

[0011] According to another aspect of the present application, a method for intelligent operation of urban rail transit equipment is provided, which includes:

[0012] Acquire the track status image collected by the camera;

[0013] Performing multi-scale foreground feature extraction and feature enhancement on the track state image to obtain an enhanced track state shallow feature map, an enhanced track state middle feature map, and an enhanced track state deep feature map;

[0014] Performing cross-scale interaction on the enhanced track state shallow feature map, the enhanced track state middle feature map and the enhanced track state deep feature map to obtain a track state cross-space multi-scale representation feature vector;

[0015] Determine an operation result based on the track state cross-spatial multi-scale representation feature vector, wherein the operation result is used to indicate whether there is a visible disease on the track;

[0016] In response to the operation result indicating that there is visible damage on the track, a track maintenance reminder is generated.

[0017] Compared with the prior art, the present application provides an intelligent operation system and method for urban rail transit equipment, which uses machine vision technology and image intelligent analysis technology to conduct in-depth study and analysis of track status images collected along the track, thereby characterizing and capturing the multi-scale state characteristics of the track, and based on the multi-scale state characteristics, realizes automated visual defect detection, so as to alleviate the limitations of traditional manual track detection and further realize efficient, safe and economical operation of rail transit. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0019] Figure 1 A block diagram of an intelligent operation system for urban rail transit equipment according to an embodiment of the present application;

[0020] Figure 2 A system architecture diagram of an urban rail transit equipment intelligent operation system according to an embodiment of the present application;

[0021] Figure 3 A block diagram of a multi-scale foreground feature processing module in an urban rail transit equipment intelligent operation system according to an embodiment of the present application;

[0022] Figure 4 This is a flow chart of the intelligent operation method of urban rail transit equipment according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0024] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0025] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0026] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0027] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0028] In track inspection, the method of relying on manual inspection is not only inefficient, but also easily affected by human factors such as fatigue and lack of concentration, resulting in unstable inspection results. In addition, with the continuous expansion of rail transit lines, the cost of relying on manual inspection for track inspection is getting higher and higher, and it is difficult to achieve scale and automation. Therefore, an optimized solution is expected.

[0029] In the technical solution of the present application, an intelligent operation system for urban rail transit equipment is proposed. Figure 1 It is a block diagram of the intelligent operation system of urban rail transit equipment according to an embodiment of the present application. Figure 2 : is a system architecture diagram of the urban rail transit equipment intelligent operation system according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to the embodiment of the present application, the urban rail transit equipment intelligent operation system 300 includes: a track state image real-time acquisition module 310, which is used to obtain a track state image collected by a camera; a multi-scale foreground feature processing module 320, which is used to extract and enhance the multi-scale foreground features of the track state image to obtain an enhanced track state shallow feature map, an enhanced track state middle feature map and an enhanced track state deep feature map; a cross-scale interaction module 330, which is used to perform cross-scale interaction on the enhanced track state shallow feature map, the enhanced track state middle feature map and the enhanced track state deep feature map to obtain a track state cross-space multi-scale representation feature vector; an operation result generation module 340, which is used to determine the operation result based on the track state cross-space multi-scale representation feature vector, and the operation result is used to indicate whether there is a visible disease on the track; a maintenance response module 350, which is used to generate a track maintenance prompt in response to the operation result that the track has a visual disease.

[0030] In particular, the track state image real-time acquisition module 310 is used to obtain the track state image acquired by the camera. The track state image refers to an image used to describe and record the state of the railway track. These images are usually used to monitor the condition of the track, detect problems and anomalies on the track, and guide track maintenance and repair work.

[0031] In particular, the multi-scale foreground feature processing module 320 is used to extract and enhance the multi-scale foreground features of the track state image to obtain an enhanced track state shallow feature map, an enhanced track state middle feature map, and an enhanced track state deep feature map. In particular, in a specific example of the present application, Figure 3As shown, the multi-scale foreground feature processing module 320 includes: a track foreground area detection unit 321, which is used to input the track state image into the track area detection module to obtain a track state foreground focus image; a track multi-scale state feature extraction unit 322, which is used to input the track state foreground focus image into the track state feature extractor to obtain a track state shallow feature map, a track state middle feature map and a track state deep feature map; a feature enhancement unit 323, which is used to input the track state shallow feature map, the track state middle feature map and the track state deep feature map into a feature enhancement module based on a compression-suppression architecture to obtain the enhanced track state shallow feature map, the enhanced track state middle feature map and the enhanced track state deep feature map.

[0032] Specifically, the track foreground area detection unit 321 is used to input the track state image into the track area detection module to obtain a track state foreground focused image. Considering that the track state image may contain a lot of background information that is irrelevant to the track state, such as buildings, etc. Therefore, in the technical solution of the present application, the track state image is input into the track area detection module based on the Yolo model to identify and filter the background noise in the track state image, focusing on the area directly related to the track detection, thereby obtaining a track state foreground focused image.

[0033] It is worth noting that YOLO (You Only Look Once) is a real-time target detection algorithm. The core idea of ​​its design is to regard the target detection task as a separate regression problem, and to achieve efficient target detection by directly predicting the target category and bounding box coordinates in a single neural network.

[0034] Accordingly, in a possible implementation, the track state image can be input into a track area detection module based on a Yolo model to obtain the track state foreground focused image through the following steps, for example: inputting the track state image; preprocessing the track state image, including resizing, normalizing, increasing batch dimensions, and other operations; inputting the preprocessed track state image into the YOLO model for track area detection; the YOLO model detects the bounding box and related information of the track area in the track state image, and identifies the position and boundary of the track; based on the track area bounding box information detected by the YOLO model, extracting the foreground part of the track area in the original track state image; synthesizing the extracted track state foreground part into the track state foreground focused image to highlight the track area.

[0035] Specifically, the track multi-scale state feature extraction unit 322 is used to input the track state foreground focused image into the track state feature extractor to obtain a track state shallow feature map, a track state middle layer feature map and a track state deep layer feature map. Because the track state may present different feature patterns at different spatial scales. For example, the shallow information of the track surface may include subtle surface changes such as rust, wear, and cracks; the middle layer information of the track may represent the local structural information of the track, such as the local shape and arrangement of track joints, bolts, track plates, etc.; and the deep layer information may contain the global morphological characteristics of the track, such as the overall bending, deformation or damage of the track. Macroscopic manifestations. In order to capture the above-mentioned multi-scale track state feature patterns, in the technical solution of the present application, the track state foreground focused image is input into the track state feature extractor based on the pyramid network to obtain a track state shallow feature map, a track state middle layer feature map and a track state deep layer feature map. In this way, the detailed information and global structural information of the track state can be obtained at the same time, which helps to understand the health of the track more comprehensively. It is worth mentioning that the pyramid network introduces multiple parallel branches or levels in the network, and each branch / level processes feature information of different scales. This enables the network to better capture the features of objects at different scales and improves its robustness to scale changes.

[0036] Specifically, the feature enhancement unit 323 is used to input the shallow feature map of the track state, the middle feature map of the track state, and the deep feature map of the track state into the feature enhancement module based on the compression-suppression architecture to obtain the enhanced shallow feature map of the track state, the enhanced middle feature map of the track state, and the enhanced deep feature map of the track state. In order to further reduce the impact of noise and irrelevant interference information on the expression of the track state, in the technical solution of the present application, the shallow feature map of the track state, the middle feature map of the track state, and the deep feature map of the track state are respectively input into the feature enhancement module based on the compression-suppression architecture to obtain the enhanced shallow feature map of the track state, the enhanced middle feature map of the track state, and the enhanced deep feature map of the track state.

[0037] In an embodiment of the present application, the track state shallow feature map, the track state middle feature map and the track state deep feature map are respectively input into a feature enhancement module based on a compression-suppression architecture to obtain the enhanced track state shallow feature map, the enhanced track state middle feature map and the enhanced track state deep feature map, including: first calculating the global mean of each feature matrix of the track state shallow feature map along the channel dimension to obtain a track state shallow feature compression information representation vector; here, the representative information and statistical characteristics of the track state shallow feature map are characterized by calculating the global mean, thereby achieving preliminary compression of features. Next, the track state shallow feature compression representation vector is subjected to one-dimensional convolution coding to obtain the track state shallow feature compression information inter-correlation representation feature vector; and the track state shallow feature compression information representation vector and the track state shallow feature compression information inter-correlation representation feature vector are cascaded to obtain the track state shallow feature compression information multi-scale representation vector; that is, the implicit correlation feature pattern in the compressed information is extracted by one-dimensional convolution coding, and the implicit correlation feature pattern in the compressed information is fused with the original compressed information representation to form a multi-scale information representation. Then, the track state shallow feature compression information multi-scale representation vector is input into a compression information feature extraction module including a multi-layer perceptron and a SiLU activation function to obtain the track state shallow feature compression information multi-scale correlation feature vector; here, the compression information feature extraction module is designed to enhance the nonlinearity of the multi-scale information representation. In particular, the SiLU activation function assigns a higher feature weight to the tiny detail information in the feature distribution, so that the effective tiny information is reused multiple times and the irrelevant background feature information is suppressed. That is, the SiLU activation function can reduce the loss of tiny effective detail information as the network deepens and alleviate the problem of low contrast between tiny detail information and background, thereby improving the learning and expression ability of the deep network for tiny features. In this way, the Sigmoid function is used to normalize the multi-scale associated feature vector of the shallow feature compression information of the track state to obtain the shallow weight feature vector of the track state; finally, based on the shallow weight feature vector of the track state, the shallow feature map of the track state is feature amplified and suppressed to obtain the enhanced shallow feature map of the track state. That is, in the technical solution of the present application, the Sigmoid function is used to normalize the multi-scale associated feature vector of the shallow feature compression information of the track state to obtain the weight values ​​of different channels, and finally the Kronecker product of each feature matrix of the shallow weight feature vector of the track state and the shallow feature map of the track state along the channel dimension is calculated to obtain the enhanced shallow feature map of the track state. Similarly, the process of obtaining the enhanced middle-level feature map of the track state and the enhanced deep feature map of the track state is similar to the process of obtaining the enhanced shallow feature map of the track state, which will not be repeated here.

[0038] In summary, in the above embodiment, the shallow feature map of the track state, the middle feature map of the track state, and the deep feature map of the track state are respectively input into the feature enhancement module based on the compression-suppression architecture to obtain the enhanced shallow feature map of the track state, the enhanced middle feature map of the track state, and the enhanced deep feature map of the track state, including: using the feature enhancement module based on the compression-suppression architecture to enhance the shallow feature map of the track state, the middle feature map of the track state, and the deep feature map of the track state with the following enhancement formula to obtain the enhanced shallow feature map of the track state, the enhanced middle feature map of the track state, and the enhanced deep feature map of the track state; wherein the formula is: ;in, is the shallow characteristic map of the track state The coordinates in the channel are The characteristic value of and are the height and width of the shallow feature map of the track state, is the shallow feature compression information representation vector of the track state eigenvalues, is the shallow feature compression information representation vector of the track state, represents one-dimensional convolutional coding, Indicates cascade, is the multi-scale representation vector of the shallow feature compression information of the track state, represents a multi-layer perceptron, represents the sigmoid function, is the shallow characteristic map of the track state, Represents the Kronecker product of the calculation vector and each feature matrix along the channel dimension of the feature map. It is the shallow characteristic map of the enhanced track state.

[0039] It is worth mentioning that in other specific examples of the present application, multi-scale foreground feature extraction and feature enhancement can be performed on the track state image in other ways to obtain an enhanced track state shallow feature map, an enhanced track state middle feature map and an enhanced track state deep feature map, for example: input the track state image; use a model such as a convolutional neural network (CNN) to use convolution kernels of different sizes or pooling operations to obtain features of different scales; extract multi-scale foreground features from the input track state image; perform feature enhancement operations on the extracted multi-scale features to enhance the expressiveness and discrimination of the features; the following methods can be used for feature enhancement: residual concatenation; Connection: Introduce residual connection to help the network learn deeper features; Attention mechanism: Introduce attention mechanism to increase the weight of important features and reduce noise interference; For shallow feature maps, use shallower network layers or larger receptive fields to capture the overall information of the track state; Middle-level feature maps use deeper network structures or more complex feature extraction operations to capture richer track state information; Deep feature maps usually contain more abstract and advanced feature representations, which are obtained through deeper network structures and more complex feature processing operations; In order to obtain the enhanced track state shallow feature map, the enhanced track state middle-level feature map and the enhanced track state deep feature map.

[0040] In particular, the cross-scale interaction module 330 is used to perform cross-scale interaction on the enhanced track state shallow feature map, the enhanced track state middle feature map and the enhanced track state deep feature map to obtain a track state cross-space multi-scale representation feature vector. In particular, in a specific example of the present application, that is, a feature map cross-space and cross-scale interaction encoder is constructed using a Transformer structure to fully interact and fuse the enhanced track state shallow feature map, the enhanced track state middle feature map and the enhanced track state deep feature map, so that the feature information at each level can complement each other. Then, the track state cross-space multi-scale representation feature vector is input into the classifier-based operation result generator to obtain the operation result, which is used to indicate whether there is a visual disease on the track. Specifically, in the embodiment of the present application, the visible defects may include but are not limited to foreign matter on the track bed, bounce loss, loose bolts, and rail spalling. In this way, in response to the operation result that the track has visible defects, a track maintenance prompt is generated.

[0041] In particular, the operation result generation module 340 is used to determine the operation result based on the track state cross-space multi-scale representation feature vector, and the operation result is used to indicate whether there is a visible defect in the track. In particular, in a specific example of the present application, the track state cross-space multi-scale representation feature vector is input into an operation result generator based on a classifier to obtain the operation result. Specifically, the process of inputting the track state cross-space multi-scale representation feature vector into an operation result generator based on a classifier to obtain the operation result includes: using multiple fully connected layers of the classifier to fully connect encode the track state cross-space multi-scale representation feature vector to obtain an encoded classification feature vector; passing the encoded classification feature vector through the Softmax classification function of the classifier to obtain the operation result.

[0042] A classifier is a machine learning model or algorithm that is used to classify input data into different categories or labels. Classifiers are part of supervised learning and perform classification tasks by learning the mapping relationship from input data to output categories.

[0043] A fully connected layer is a common type of layer in neural networks. In a fully connected layer, each neuron is connected to all neurons in the previous layer, and each connection has a weight. This means that each neuron in the fully connected layer receives inputs from all neurons in the previous layer, and performs a weighted sum of these inputs through the weights, and then passes the result to the next layer.

[0044] The Softmax classification function is a commonly used activation function for multi-classification problems. It converts each element of the input vector into a probability value between 0 and 1, and the sum of these probability values ​​is equal to 1. The Softmax function is often used in the output layer of a neural network, especially for multi-classification problems, because it can map the network output to the probability distribution of each category. During the training process, the output of the Softmax function can be used to calculate the loss function and update the network parameters through the back propagation algorithm. It is worth noting that the output of the Softmax function does not change the relative size relationship between the elements, but only normalizes them. Therefore, the Softmax function does not change the characteristics of the input vector, but only converts it into a probability distribution form.

[0045] In a preferred embodiment, inputting the track state cross-spatial multi-scale representation feature vector into a classifier-based operation result generator to obtain an operation result comprises:

[0046] Probabilistically converting each eigenvalue of the track state cross-space multi-scale representation eigenvector based on a probability activation function, such as a sigmoid function, to obtain a probabilistic track state cross-space multi-scale representation eigenvector;

[0047] Obtaining the track state cross-space multi-scale representation feature vector and inputting it into the classifier-based operation result generator to obtain a visual defect probability value indicating the presence of visual defects;

[0048] Determine a track state cross-space multi-scale representation class cognitive symbol value based on a comparison between each eigenvalue of the probabilistic track state cross-space multi-scale representation eigenvector and the visual defect probability value, wherein the track state cross-space multi-scale representation class cognitive symbol value is equal to one, zero and negative one in response to the eigenvalue of the probabilistic track state cross-space multi-scale representation eigenvector being greater than, equal to and less than the abnormality probability value, respectively;

[0049] Calculating the mean of all eigenvalues ​​of the probabilistic orbital state cross-space multi-scale representation eigenvector to obtain an overall phase shift value of the orbital state cross-space multi-scale representation class;

[0050] After multiplying each eigenvalue of the probabilistic orbital state cross-space multi-scale representation eigenvector by the orbital state cross-space multi-scale representation class cognitive symbol value and the orbital state cross-space multi-scale representation class overall phase shift value, weighted difference calculation is performed, and the absolute value is taken to obtain the optimized eigenvalue of the probabilistic orbital state cross-space multi-scale representation eigenvector;

[0051] The optimized track state cross-spatial multi-scale representation feature vector composed of the optimized feature values ​​is input into a classifier-based operation result generator to obtain an operation result.

[0052] Specifically, the optimization steps are expressed as: ;in, and are respectively the i-th eigenvalue of the probabilistic orbital state cross-space multi-scale representation eigenvector and the optimized orbital state cross-space multi-scale representation eigenvector, is the visual disease probability value, is the cognitive symbolic value of the orbital state across space and multi-scale representation, is the cognitive symbolic value of the orbital state across space and multi-scale representation, and are the first weighted hyperparameter and the second weighted hyperparameter, respectively. Indicates taking the absolute value.

[0053] Here, the enhanced track state shallow feature map, the enhanced track state middle feature map and the enhanced track state deep feature map respectively express the cross-scale and cross-depth image semantic features of the track state foreground focused image based on image space distribution enhancement. Therefore, after the enhanced track state shallow feature map, the enhanced track state middle feature map and the enhanced track state deep feature map are input into the feature map cross-space and cross-scale interactive encoder based on the Transformer structure, the obtained track state cross-space multi-scale representation feature vector will also have a priori-posteriori class probability causal association loss relative to the semantic representation of the source image due to the difference in autocorrelation attention weights under different feature scale-depth expressions, thereby affecting the accuracy of the classification results.

[0054] Therefore, in the above-mentioned preferred embodiment, the applicant of the present application obtains the class-cognitive phase transformation response of the feature vector of the track state cross-space multi-scale representation by comparing the probabilistic amplitude of the eigenvalue of the feature vector of the track state cross-space multi-scale representation with the class probability, and performs a feature distribution sequence invariance transformation based on the class differential distribution expansion on the phase shift response of the eigenvalue of the feature vector of the track state cross-space multi-scale representation relative to the class probability representation of the feature set as a whole, so as to realize the causal constraint of the posterior class probability of the feature vector of the track state cross-space multi-scale representation on its prior feature distribution representation, so as to improve the accuracy of the operation results obtained by the operation result generator based on the classifier when the feature vector of the track state cross-space multi-scale representation is input.

[0055] In particular, the maintenance response module 350 is used to generate a track maintenance reminder in response to the operation result that there is a visible disease on the track. In this way, track maintenance reminders can be effectively generated to help maintenance personnel and management teams to understand the track conditions in a timely manner, formulate corresponding maintenance plans, and ensure the safety and smoothness of track operations.

[0056] As described above, the urban rail transit equipment intelligent operation system 300 according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with an urban rail transit equipment intelligent operation algorithm. In one possible implementation, the urban rail transit equipment intelligent operation system 300 according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the urban rail transit equipment intelligent operation system 300 can be a software module in the operating system of the wireless terminal, or it can be an application developed for the wireless terminal; of course, the urban rail transit equipment intelligent operation system 300 can also be one of the many hardware modules of the wireless terminal.

[0057] Alternatively, in another example, the urban rail transit equipment intelligent operation system 300 and the wireless terminal may also be separate devices, and the urban rail transit equipment intelligent operation system 300 may be connected to the wireless terminal via a wired and / or wireless network, and transmit interactive information in accordance with an agreed data format.

[0058] Furthermore, a method for intelligent operation of urban rail transit equipment is also provided.

[0059] Figure 4 Flow chart of the intelligent operation method of urban rail transit equipment according to an embodiment of the present application. Figure 4 As shown, according to the embodiment of the present application, the intelligent operation method of urban rail transit equipment includes the following steps: S1, obtaining a track state image captured by a camera; S2, performing multi-scale foreground feature extraction and feature enhancement on the track state image to obtain an enhanced track state shallow feature map, an enhanced track state middle feature map and an enhanced track state deep feature map; S3, performing cross-scale interaction on the enhanced track state shallow feature map, the enhanced track state middle feature map and the enhanced track state deep feature map to obtain a track state cross-space multi-scale representation feature vector; S4, determining an operation result based on the track state cross-space multi-scale representation feature vector, the operation result is used to indicate whether there is a visible disease on the track; S5, generating a track maintenance prompt in response to the operation result that the track has a visible disease.

[0060] In summary, according to the embodiment of the present application, the intelligent operation method of urban rail transit equipment is explained, which uses machine vision technology and image intelligent analysis technology to conduct in-depth study and analysis of track status images collected along the track, thereby characterizing and capturing the multi-scale state characteristics of the track, and based on the multi-scale state characteristics, realizes automated visual defect detection, so as to alleviate the limitations of traditional manual track detection and further realize efficient, safe and economical operation of rail transit.

[0061] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. An intelligent operation system for urban rail transit equipment, characterized in that: include: A track status image real-time acquisition module is used to obtain the track status image acquired by the camera; A multi-scale foreground feature processing module, used for performing multi-scale foreground feature extraction and feature enhancement on the track state image to obtain an enhanced track state shallow feature map, an enhanced track state middle feature map and an enhanced track state deep feature map; A cross-scale interaction module, used for cross-scale interaction of the enhanced track state shallow feature map, the enhanced track state middle feature map and the enhanced track state deep feature map to obtain a track state cross-space multi-scale representation feature vector; An operation result generation module is used to determine the operation result based on the track state cross-space multi-scale representation feature vector, and the operation result is used to indicate whether there are visible defects on the track, wherein the visible defects include foreign matter on the track bed, bounce loss, loose bolts, and rail spalling and falling blocks; The maintenance response module is used to generate a track maintenance prompt in response to the operation result that there is a visible disease on the track.

2. The urban rail transit equipment intelligent operation system according to claim 1 is characterized in that: The multi-scale foreground feature processing module comprises: A track foreground area detection unit, used for inputting the track state image into a track area detection module to obtain a track state foreground focused image; A track multi-scale state feature extraction unit, used for inputting the track state foreground focused image into a track state feature extractor to obtain a track state shallow feature map, a track state middle feature map and a track state deep feature map; The feature enhancement unit is used to input the shallow feature map of the track state, the middle feature map of the track state and the deep feature map of the track state into the feature enhancement module based on the compression-suppression architecture to obtain the enhanced shallow feature map of the track state, the enhanced middle feature map of the track state and the enhanced deep feature map of the track state.

3. The urban rail transit equipment intelligent operation system according to claim 2 is characterized in that: The track foreground area detection unit is used to: The track state image is input into a track area detection module based on a Yolo model to obtain the track state foreground focus image.

4. The urban rail transit equipment intelligent operation system according to claim 3 is characterized in that: The track multi-scale state feature extraction unit is used to: The track state foreground focused image is input into a track state feature extractor based on a pyramid network to obtain the track state shallow feature map, the track state middle feature map and the track state deep feature map.

5. The urban rail transit equipment intelligent operation system according to claim 4 is characterized in that: The feature enhancement unit comprises: A compression information representation subunit, used to calculate the global mean of each feature matrix of the track state shallow feature map along the channel dimension to obtain a track state shallow feature compression information representation vector; A compression information association coding subunit, used for performing one-dimensional convolution coding on the track state shallow feature compression representation vector to obtain a track state shallow feature compression information association representation feature vector; A compressed information multi-scale representation subunit, used for cascading the track state shallow feature compressed information representation vector and the track state shallow feature compressed information association representation feature vector to obtain a track state shallow feature compressed information multi-scale representation vector; A compressed information multi-scale feature extraction subunit, used for inputting the track state shallow feature compressed information multi-scale representation vector into a compressed information feature extraction module including a multi-layer perceptron and a SiLU activation function to obtain a track state shallow feature compressed information multi-scale correlation feature vector; A compressed information multi-scale feature normalization subunit is used to use a Sigmoid function to perform a normalization operation on the track state shallow feature compressed information multi-scale associated feature vector to obtain a track state shallow weight feature vector; The feature amplification and suppression subunit is used to perform feature amplification and suppression operations on the track state shallow feature map based on the track state shallow weight feature vector to obtain the enhanced track state shallow feature map.

6. The urban rail transit equipment intelligent operation system according to claim 5, characterized in that: The feature amplification and suppression subunit is used to: The Kronecker product of the track state shallow weight feature vector and each feature matrix of the track state shallow feature map along the channel dimension is calculated to obtain the enhanced track state shallow feature map.

7. The urban rail transit equipment intelligent operation system according to claim 6, characterized in that: The cross-scale interaction module is used to: The enhanced track state shallow feature map, the enhanced track state middle feature map and the enhanced track state deep feature map are input into the feature map cross-space and cross-scale interactive encoder based on the Transformer structure to obtain the track state cross-space multi-scale representation feature vector.

8. The urban rail transit equipment intelligent operation system according to claim 7 is characterized in that: The operation result generating module is used to: The track state cross-spatial multi-scale representation feature vector is input into a classifier-based operation result generator to obtain the operation result.

9. A smart operation method for urban rail transit equipment, characterized in that: include: Acquire the track status image collected by the camera; Performing multi-scale foreground feature extraction and feature enhancement on the track state image to obtain an enhanced track state shallow feature map, an enhanced track state middle feature map, and an enhanced track state deep feature map; Performing cross-scale interaction on the enhanced track state shallow feature map, the enhanced track state middle feature map and the enhanced track state deep feature map to obtain a track state cross-space multi-scale representation feature vector; Determine the operation result based on the track state cross-space multi-scale representation feature vector, the operation result is used to indicate whether there are visible defects on the track, wherein the visible defects include foreign matter on the track bed, bounce loss, loose bolts, and rail spalling and falling blocks; In response to the operation result indicating that there is visible damage on the track, a track maintenance reminder is generated.

Images