Multi-dimensional monitoring and identification method for inspection robot of rail transit

Abstract

The invention discloses a multi-dimensional monitoring and identification method for an inspection robot of rail transit, and the method comprises the steps: obtaining rail data in real time through arranging a laser radar and a visual inertia fused synchronous positioning module on the inspection robot, and constructing an inspection backbone network; segmenting a rail transit trunk line in real time by adopting a self-adaptive module in the inspection trunk network, and obtaining coordinates and data features between rail data sampling points; building a track monitoring framework based on the three-dimensional point cloud, monitoring the track state in real time according to the data characteristics by adopting an edge detection method, and obtaining track state monitoring data; traversal detection is performed on the track state monitoring data by adopting a neural network, a panoramic image data set of a traffic track is acquired in real time, and the traffic track is automatically monitored and identified, so that the inspection efficiency is remarkably improved, the labor cost is reduced, safe and stable operation of track traffic is ensured, the robustness and adaptability of the system are enhanced, and the system reliability is improved. And effective monitoring and management of the track state are also realized.

Description

Technical Field

[0001] The present invention relates to the technical field of rail transit monitoring, and in particular to a multi-dimensional monitoring and identification method of a rail transit inspection robot. Background Art

[0002] In recent years, with the rapid development of urban rail transit systems, the requirements for the safety and reliability of rail facilities have been increasing. Traditional rail inspections mainly rely on manual inspections or simple automated equipment. This method is not only inefficient but also difficult to ensure the comprehensiveness and accuracy of detection. Therefore, more and more research and development work is focused on the use of intelligent inspection robots for automatic monitoring and identification of rail transit.

[0003] Currently, inspection robots are usually equipped with a variety of sensors, such as LiDAR, cameras, and IMUs, to achieve all-round monitoring of the track and its surrounding environment. Computer vision algorithms, deep learning models, and simultaneous localization and mapping (SLAM) technology are used to process and analyze the collected data to improve detection accuracy. Through wireless communication technology, inspection robots can upload data to the central control system in real time, allowing managers to understand and respond to track conditions in a timely manner. Using the built-in navigation system, inspection robots can move autonomously in complex track environments and complete scheduled inspection tasks. However, the current multi-dimensional monitoring and identification methods for rail transit inspection robots still have the following problems:

[0004] (1) Insufficient adaptability to complex environments: In severe weather conditions, sensor performance may be affected, resulting in inaccurate or ineffective data collection. Although there are many types of sensors, how to effectively integrate data from different sources and extract valuable information from them is a difficult problem;

[0005] (2) In order to ensure the efficiency of inspection work, powerful computing resources are needed to support rapid data processing and decision-making. The hardware requirements are high and the real-time processing capabilities are limited. In addition, during the long-term operation of the inspection robot, the stability and reliability of the inspection robot may decrease due to mechanical wear, software failures, etc. Summary of the Invention

[0006] The purpose of the present invention is to provide a multi-dimensional monitoring and identification method for a rail transit inspection robot to solve the technical problems of insufficient adaptability to complex environments, limited real-time processing capabilities, and poor long-term stability in the existing technology.

[0007] In order to solve the above technical problems, the present invention specifically provides the following technical solutions:

[0008] The present invention provides a multi-dimensional monitoring and identification method for a rail transit inspection robot, comprising the following steps:

[0009] By setting up a laser radar and visual inertial fusion synchronous positioning module on the inspection robot, a rail transit map is constructed, track data is obtained in real time, and the said track data is used to construct an inspection backbone network based on the rail transit map;

[0010] An adaptive module is used in the inspection backbone network to segment the rail transit trunk line in real time, and a self-attention mechanism is used to obtain the similarity between the local neighborhood points of the track data, and the coordinates and data features between the track data sampling points are obtained;

[0011] A track monitoring framework based on a three-dimensional point cloud is constructed according to the coordinates and data features between the track data sampling points, and an edge detection method is used to monitor the track status in real time based on the data features to obtain track status monitoring data;

[0012] A neural network is used to perform traversal detection on the track status monitoring data, a dynamic adjustment and optimization recognition model for historical inspection data is established, a panoramic image data set of the traffic track is obtained in real time, and the traffic track is automatically monitored and identified.

[0013] As a preferred solution of the present invention, a laser radar and visual-inertial fusion synchronous positioning module is set on the inspection robot to build a rail transit map and obtain track data in real time, including:

[0014] LiDAR is used to collect 3D point cloud data of the track and surrounding environment. Multiple cameras installed on the inspection robot capture image information of the surrounding environment. Combined with the inertial measurement unit (IMU), the robot's acceleration and angular velocity changes are recorded to obtain multi-source rail transit data.

[0015] Denoising the multi-source rail transit data, extracting visual feature data, optimizing the multi-source rail transit data using a synchronized time reference, and obtaining time-synchronized multi-source fusion data;

[0016] Extracting the pose estimation data of the inspection robot in the current frame of the multi-source fusion data, constructing a rail transit map, continuously detecting the robot's access area, performing loop closure operations, and optimizing the global rail transit map in real time;

[0017] The track status is continuously monitored according to the global rail transit map to obtain real-time track data.

[0018] As a preferred solution of the present invention, the rail data is used to construct an inspection backbone network based on a rail transit map, including:

[0019] Generate the original point cloud set (x1, x2, ..., x n), the maximum pooling layer symmetric strategy of the PointNet network is used to process the original point cloud set (x1, x2, ..., x n ) is disordered, and the ordered original point cloud set (x1, x2, ..., x n ) as network input data;

[0020] Multi-layer perceptron MLP is used to extract features and transform the original point cloud set (x1, x2, ..., x n ) is mapped to a 1024-dimensional feature space, and then the maximum pooling layer is used to process the high-dimensional features to obtain a point cloud segmentation network. The expression of the point cloud segmentation network is:

[0021] f({x1,x2,…,x n})=ρ(s(h(x1),h(x2),…,h(x3)))

[0022] Among them, f represents the point cloud countermeasure segmentation function, h represents the feature extraction layer, s represents the symmetric function, and ρ represents the 1024-dimensional feature extraction layer;

[0023] Different scale ranges are established for the points used in the point cloud segmentation network, and several regions with different radius sizes are divided with the sampling points as the center. Each different scale region contains a different number of point clouds. The scale features of each sampling point are trained using a multi-layer perceptron (MLP) to obtain local area features.

[0024] The backbone network is segmented according to the local area characteristics to obtain an inspection backbone network.

[0025] As a preferred solution of the present invention, an adaptive module is used in the inspection backbone network to segment the rail transit trunk line in real time, and a self-attention mechanism is used to obtain the similarity between the local neighborhood points of the track data, including:

[0026] In the inspection backbone network, N downsampled input points are extracted. s points, extract the feature set F of the downsampled point set s , for the N s The farthest sampling FPS is used for each down-sampling point to obtain relatively uniform original sampling points;

[0027] The adaptive sampling module is used to process the original sampling points, and the K-NN algorithm is used to group and search the neighborhood points of each sampling point. The aggregation function is used to aggregate the features of each neighborhood to all other neighborhood points to obtain the final neighborhood aggregation feature g i,k , whose expression is:

[0028]

[0029] Among them, x i,k Represents the sampling point x i The kth neighboring point of i,j Represents the sampling point x i The jth neighboring point of i,k ,x i,j ) represents the relationship function, calculate the sampling point x i and its high-dimensional relationship between the neighborhood, γ(x i,j ) represents the function of the number of conversion channels, N(x i ) represents the sampling point x i The feature set of the kth neighborhood point, σ represents the aggregation function, g i,k Represents the aggregated features of the kth neighborhood point of the i-th sampling point after updating;

[0030] A multi-layer MLP network with a Softmax activation function is used on k neighborhood points according to the aggregated features to obtain a mapping relationship between adjacent neighborhood points, a coordinate axis is established for the sampling points according to the mapping relationship, and a normalized weight of each coordinate axis and feature channel is obtained;

[0031] The inspection backbone network is segmented in real time according to the normalized weights to obtain real-time rail transit backbone lines.

[0032] As a preferred solution of the present invention, obtaining coordinates and data features between track data sampling points according to the rail transit trunk line includes:

[0033] Locating the target rail transit trunk line through the sampling points on the rail transit trunk line, using the XY coordinates of the two-dimensional image to index the corresponding point cloud data, establishing the same coordinate system for all the point cloud data, and obtaining the coordinates of the track data;

[0034] The local features of the point cloud data are obtained according to the normalized weights, and the area of each feature point is expanded by segmenting the inspection backbone network to obtain the global data features of the track data.

[0035] As a preferred solution of the present invention, a track monitoring framework based on a three-dimensional point cloud is constructed according to the coordinates and data features between the track data sampling points, including:

[0036] Using the coordinates and data features between the track data sampling points as high-quality point cloud data of the track in an initial state, and using the high-quality point cloud data to construct a track reference model;

[0037] Acquire real-time daily inspection data of the inspection robot, and update the daily inspection data into the track reference model after each inspection to obtain real-time track status;

[0038] Setting a status indicator for the track state based on the track benchmark model and historical data, and setting a threshold for the status indicator. If the threshold is exceeded, the track will be considered to be potentially dangerous;

[0039] Performing data monitoring on the real-time track status, using the monitoring data as training samples to automatically detect using a machine learning algorithm, monitoring the track status in real time, and automatically marking abnormal areas;

[0040] A visualization platform is built as the visualization interface of the track monitoring framework to display the overall status of the track in real time.

[0041] As a preferred solution of the present invention, edge detection method is used to monitor the track status in real time based on the data features to obtain track status monitoring data, including:

[0042] Select any track in the inspection backbone network on the track monitoring framework, project each point on the track onto a two-dimensional coordinate plane, and use a KD-Tree to perform a neighbor search with any point P as the center in the two-dimensional coordinate plane to obtain M neighboring points. The neighborhood set of any point P is defined as M p , whose expression is:

[0043] M p ={p0∈P|p0∈M p}

[0044] Among them, p0 is one of the neighbors of P, and p0 is the neighborhood set M p one of the;

[0045] The M neighborhood sets corresponding to any point P are connected with the rest of the neighborhood points with point P as the center, and a series of angles θ={θ1,θ2,…,θ m}, sort the angles θ to get the maximum value θ max ;

[0046] The maximum angle between two consecutive points is used to calculate the boundary probability. If the maximum angle is greater than a set threshold, point P is defined as a boundary point.

[0047] By traversing each point to determine whether it is a boundary point, all boundary points are obtained, and two straight lines of edge points are obtained by straight line fitting of the boundary points. Track boundary data is simulated in real time on the visualization interface to obtain monitoring data of the track status.

[0048] As a preferred solution of the present invention, the track state monitoring data is subjected to traversal detection using a neural network, including:

[0049] Preprocessing the track status monitoring data, annotating the monitoring data, and integrating the track status monitoring data into a unified framework using a multimodal fusion network;

[0050] A sliding window mode is used to perform traversal detection block by block, and the data in each window is fed into a neural network for training to obtain track status anomaly data;

[0051] Performing abnormality location and abnormality classification on the track state abnormality data, calculating confidence scores for the abnormality location and abnormality classification, and setting thresholds to filter out abnormal data with low confidence;

[0052] Adjacent abnormal areas are merged into a whole, the detection results are visualized, and superimposed on the track visualization interface to intuitively display the location and type of the abnormal area.

[0053] As a preferred solution of the present invention, a dynamic adjustment and optimization recognition model of historical inspection data is established to obtain a panoramic image dataset of the traffic track in real time, and automatically monitor and identify the traffic track, including:

[0054] The monitoring data of the abnormal area is subjected to multimodal fusion using an adaptive mechanism, and the inspection robot is revisited in the corresponding abnormal area to obtain potential recognition errors by comparing the current detection results with historical records;

[0055] The corrected track data is added to the training set, the track model of the track monitoring framework is optimized, and the panoramic image information obtained by the inspection robot is processed using a panoramic image target recognition algorithm;

[0056] The improved TinyYOLOv3 neural network model uses a deep learning algorithm to perform target recognition on the panoramic image information, obtain a full set of rail transit data, and automatically monitor and identify the status of the traffic track in real time.

[0057] As a preferred solution of the present invention, the panoramic image target recognition algorithm includes:

[0058] Generating a panoramic image of the track area using the multi-source rail transit data collected by the inspection robot, and annotating historical and newly collected panoramic images;

[0059] The annotated panoramic image is subjected to data preprocessing, and an improved TinyYOLOv3 neural network model is used to extract features from the multi-source rail transit data to obtain track-related features;

[0060] The abnormal area of the track is calculated using weighted cross entropy loss, and the track status is regularly evaluated by repeatedly revisiting the abnormal area by the inspection robot;

[0061] The track status is synchronized to the visualization platform in real time, and the working status of the inspection robot, the health status of the track and abnormal areas are displayed in real time.

[0062] Compared with the prior art, the present invention has the following beneficial effects:

[0063] The present invention adopts a point cloud segmentation network based on deep learning to segment the track line, uses a large amount of track point cloud data to train the neural network, and obtains the point cloud segmentation model more efficiently. The segmentation algorithm using deep learning can accurately identify track components, no longer relying on prior information about the track geometry, thereby improving the accuracy of track monitoring.

[0064] Multi-source rail transit data is collected in the form of multi-source data fusion to generate high-quality panoramic images. The improved TinyYOLOv3 neural network model is used for feature extraction to achieve high-precision detection and real-time monitoring of track defects. Adaptive sampling and dynamic adjustment mechanisms are adopted, combined with sliding window mode and edge detection algorithm to optimize the identification and classification of abnormal areas, and intuitively display the health status and abnormal locations of the track through a visualization platform. It supports 24 / 7 uninterrupted monitoring and preventive maintenance, significantly improves inspection efficiency, reduces labor costs, and ensures the safe and stable operation of rail transit. It not only enhances the robustness and adaptability of the system, but also realizes effective monitoring and management of track status. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other implementation drawings based on the provided drawings without inventive effort.

[0066] Figure 1 This is a flow chart of the multi-dimensional monitoring and identification method of the inspection robot provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0068] like Figure 1 As shown, the present invention provides a multi-dimensional monitoring and identification method for a patrol robot of rail transit, comprising the following steps:

[0069] By setting up a laser radar and visual inertial fusion synchronous positioning module on the inspection robot, a rail transit map is constructed, track data is obtained in real time, and the said track data is used to construct an inspection backbone network based on the rail transit map;

[0070] In this embodiment, a multi-sensor fusion approach is used to improve positioning accuracy, reduce cumulative errors, dynamically adapt to complex and changing environments, and maintain good environmental perception capabilities;

[0071] In this embodiment, by constructing an accurate rail transit map to reflect the specific conditions of the rail facilities along the track, an automated decision-making basis is provided for the path planning of the inspection robot, which helps the inspection robot to allocate tasks and execute them more intelligently.

[0072] In this embodiment, based on the constructed map information, the most optimized inspection route can be designed to ensure that each key area can be effectively inspected while minimizing unnecessary movement distance. When new obstacles or other abnormal conditions are discovered, the system can respond quickly and adjust the inspection plan to ensure the safety and continuity of the operation, thereby building an efficient inspection backbone network.

[0073] An adaptive module is used in the inspection backbone network to segment the rail transit trunk line in real time, and a self-attention mechanism is used to obtain the similarity between the local neighborhood points of the track data, and the coordinates and data features between the track data sampling points are obtained;

[0074] In this embodiment, the self-attention mechanism can effectively capture subtle differences and similarities between local neighborhood points in the track data, and can more accurately identify small defects or changes in the track surface, such as cracks and wear, thereby improving the accuracy of detection and accurately extracting local features. Even when the data is not completely consistent or there is noise, the self-attention mechanism can maintain a high recognition accuracy by focusing on the most relevant local features, thereby enhancing the system's adaptability to different environmental conditions.

[0075] In this embodiment, an adaptive module is used to dynamically adjust the segmentation strategy according to the actual track conditions, ensuring that each sub-area is fully inspected, which helps to optimize resource allocation and improve overall inspection efficiency. The self-attention mechanism allows the model to focus on the most important information, reducing unnecessary computation, enabling the system to complete complex computing tasks in a shorter time and achieve real-time monitoring and analysis.

[0076] A track monitoring framework based on a three-dimensional point cloud is constructed according to the coordinates and data features between the track data sampling points, and an edge detection method is used to monitor the track status in real time based on the data features to obtain track status monitoring data;

[0077] In this embodiment, by utilizing three-dimensional point cloud data, highly accurate recognition of subtle changes in the track surface is achieved. The edge detection method can effectively highlight the discontinuities of the track surface, allowing the system to more accurately locate and quantify these defects. Multi-level data analysis can be performed to further improve the comprehensiveness and accuracy of detection.

[0078] In this embodiment, a track monitoring framework based on three-dimensional point clouds is adopted, and an edge detection method is used to monitor the track status in real time. This not only greatly improves the accuracy and efficiency of track detection, but also enhances the robustness and adaptability of the system, which is of great significance for ensuring the safety and reliability of rail transit.

[0079] A neural network is used to perform traversal detection on the track status monitoring data, a dynamic adjustment and optimization recognition model for historical inspection data is established, a panoramic image data set of the traffic track is obtained in real time, and the traffic track is automatically monitored and identified.

[0080] In this embodiment, a neural network can be used to automatically extract advanced features from complex track condition monitoring data, improve the accuracy and reliability of defect detection, and instantly analyze the collected panoramic image data sets, quickly identify potential problems and take immediate action, thereby reducing risks caused by delays.

[0081] In this embodiment, by establishing a dynamic adjustment and optimization recognition model for historical inspection data, parameters and strategies can be self-adjusted according to different track conditions and inspection needs to complete the task in the best way. The automated monitoring and recognition process reduces the need for manual inspection, reduces labor costs, detects small problems in a timely manner and repairs them in a timely manner, avoids greater damage, extends the service life of the equipment, and reduces maintenance costs.

[0082] By installing a laser radar and visual-inertial fusion synchronous positioning module on the inspection robot, a rail transit map is constructed and track data is obtained in real time, including:

[0083] LiDAR is used to collect 3D point cloud data of the track and surrounding environment. Multiple cameras installed on the inspection robot capture image information of the surrounding environment. Combined with the inertial measurement unit (IMU), the robot's acceleration and angular velocity changes are recorded to obtain multi-source rail transit data.

[0084] In this embodiment, different types of sensors such as lidar, cameras, and IMUs are used to complement information, which can significantly improve the accuracy and robustness of positioning. For example, when visual features are not obvious or lighting conditions are poor, the geometric information provided by the lidar can make up for the lack of visual data; and vice versa. Although the IMU has high accuracy in the short term, it will have drift problems in long-term use. After combining the lidar and visual data, this cumulative error can be corrected through closed-loop detection and other methods to improve positioning accuracy.

[0085] In this embodiment, the three-dimensional point cloud data obtained by the lidar and the image information captured by the visual sensor can fully understand the status of the track and the surrounding environment, including but not limited to the track surface condition, obstacle location, etc. Even in the face of complex and changing environments, it can maintain good environmental perception capabilities, ensure the smooth progress of inspection work, and enhance environmental perception capabilities.

[0086] Denoising the multi-source rail transit data, extracting visual feature data, optimizing the multi-source rail transit data using a synchronized time reference, and obtaining time-synchronized multi-source fusion data;

[0087] In this embodiment, the collected multi-source rail transit data is denoised to improve data quality and reduce interference factors in subsequent analysis. The multi-source data is optimized using a synchronous time reference to ensure that data from different sensors can remain consistent in time, thereby improving the accuracy of position estimation.

[0088] Extracting the pose estimation data of the inspection robot in the current frame of the multi-source fusion data, constructing a rail transit map, continuously detecting the robot's access area, performing loop closure operations, and optimizing the global rail transit map in real time;

[0089] The track status is continuously monitored according to the global rail transit map to obtain real-time track data.

[0090] In this embodiment, based on the extracted multi-source fusion data, the position and posture of the inspection robot can be estimated more accurately and used to build a high-quality rail transit map. This is not limited to initial mapping, but also supports real-time updating of the map to reflect the latest track conditions. When the robot returns to a previously visited area, performing a loop closure operation can further optimize the consistency and accuracy of the global map and reduce cumulative errors.

[0091] In this embodiment, the performance of the lidar and visual sensor fusion algorithm can be significantly improved, including higher perception accuracy, stronger robustness and better real-time performance. It is not only applicable to rail transit inspection robots, but can also be extended to other multimodal perception application scenarios.

[0092] In this embodiment, the point cloud data generated by the lidar is filtered and denoised to improve its quality and usability. The IMU data is calibrated and synchronized to ensure its time alignment with the lidar data. Finally, the image data captured by the industrial camera is corrected and fused with other sensor data to improve the accuracy and robustness of the system.

[0093] The rail data is used to construct an inspection backbone network based on the rail transit map, including:

[0094] Generate the original point cloud set (x1, x2, ..., x n ), the maximum pooling layer symmetric strategy of the PointNet network is used to process the original point cloud set (x1, x2, ..., x n ) is disordered, and the ordered original point cloud set (x1, x2, ..., x n ) as network input data;

[0095] In this embodiment, the PointNet network uses a multi-layer perceptron (MLP) for feature extraction. The MLP not only has input and output layers, but also can have multiple hidden layers in the middle. After the MLP maps the features of each point to a high-dimensional redundant space, it uses the maximum pooling layer to process the high-dimensional features to reduce feature loss and realize a point-by-point MLP point cloud segmentation network.

[0096] Multi-layer perceptron MLP is used to extract features and transform the original point cloud set (x1, x2, ..., x n ) is mapped to a 1024-dimensional feature space, and then the maximum pooling layer is used to process the high-dimensional features to obtain a point cloud segmentation network. The expression of the point cloud segmentation network is:

[0097] f({x1,x2,…,x n})=ρ(s(h(x1),h(x2),…,h(x3)))

[0098] Among them, f represents the point cloud countermeasure segmentation function, h represents the feature extraction layer, s represents the symmetric function, and ρ represents the 1024-dimensional feature extraction layer;

[0099] In this embodiment, the point cloud segmentation network is implemented through convolution with shared weights. First, the spatial posture correction is performed based on a T-Net, and then the feature extraction is performed through MLP to obtain 64-dimensional redundant features. The feature space is aligned again based on T-Net, and finally the aligned features are mapped to a higher 1024-dimensional feature space. For each input point, it is ultimately represented by a 1024-dimensional feature vector, and such a vector is obviously redundant for a 3-dimensional point cloud. The maximum eigenvalue on the 1024-dimensional channel can be retained through the maximum pooling layer, so that the global features of all input point clouds can be obtained, and finally the global features are used for classification or segmentation.

[0100] Different scale ranges are established for the points used in the point cloud segmentation network, and several regions with different radius sizes are divided with the sampling points as the center. Each different scale region contains a different number of point clouds. The scale features of each sampling point are trained using a multi-layer perceptron (MLP) to obtain local area features.

[0101] In this embodiment, a layered approach is adopted, and convolution kernels are continuously used to scan and process the pixels in the image, continuously increasing the receptive field of the feature map. Each pixel contains increasingly rich information, and a multi-level feature learning network is used to divide the entire point set into numerous overlapping local regions based on distance metrics in three-dimensional space. Local features are extracted from small regions to capture fine object geometric structures, and then these local features are grouped into sub-regions with a larger spatial range to obtain higher-dimensional features. The above steps are repeated until a complete point set feature is obtained as the receptive field continues to increase, thereby obtaining higher-precision local region features.

[0102] The backbone network is segmented according to the local area characteristics to obtain an inspection backbone network.

[0103] In this embodiment, the local features learned by MLP represent that each sampling point carries more information, and the receptive field gradually increases. As the receptive field of the network increases, the feature dimension extracted becomes higher and higher until the entire global feature is extracted. The backbone feature data is segmented according to the global feature, and the inspection backbone network is optimized after obtaining the backbone network.

[0104] In the inspection backbone network, an adaptive module is used to segment the rail transit trunk line in real time, and a self-attention mechanism is used to obtain the similarity between the local neighborhood points of the track data, including:

[0105] In the inspection backbone network, N downsampled input points are extracted. s points, extract the feature set F of the downsampled point set s , for the N sThe farthest sampling FPS is used for each down-sampling point to obtain relatively uniform original sampling points;

[0106] In this embodiment, by combining the coordinates and data features between sampling points, a more comprehensive track health assessment model can be constructed, which is not limited to changes in geometric shapes, but also includes considerations of material properties, environmental impacts, and other aspects.

[0107] The adaptive sampling module is used to process the original sampling points, and the K-NN algorithm is used to group and search the neighborhood points of each sampling point. The aggregation function is used to aggregate the features of each neighborhood to all other neighborhood points to obtain the final neighborhood aggregation feature g i,k , whose expression is:

[0108]

[0109] Among them, x i,k Represents the sampling point x i The kth neighboring point of i,j Represents the sampling point x i The jth neighboring point of i,k ,x i,j ) represents the relationship function, calculate the sampling point x i and its high-dimensional relationship between the neighborhood, γ(x i,j ) represents the function of the number of conversion channels, N(x i ) represents the sampling point x i The feature set of the kth neighborhood point, σ represents the aggregation function, g i,k Represents the aggregated features of the kth neighborhood point of the i-th sampling point after updating;

[0110] In this embodiment, the feature set F s As the input matrix X, the self-attention mechanism has three core matrices: Q (Query), K (Key), and V (Value). These three matrices are essentially linear transformations of the input matrix X. By linearly changing X, the model's fitting ability is improved. Attention uses Q to query which of K is more important, and the corresponding weight matrix is multiplied by V. This allows V to focus on more important information and ignore less important information. The difference between the self-attention mechanism and ordinary attention is that the three matrices are the same thing, that is, the three come from the same X.

[0111] In this embodiment, the farthest point sampling FPS is first used to obtain relatively uniform original sampling points, and the point set and feature set of the sampling points are obtained from the previous layer. Then, the adaptive sampling module is used to process the original sampling points. The K-NN algorithm is used to group and search the neighborhood points of each sampling point, and then the self-attention mechanism is used to update the local neighborhood features.

[0112] In this embodiment, the self-attention mechanism allows the model to focus on the most important information, reducing unnecessary computation, enabling the system to complete complex computational tasks in a shorter time and achieve real-time monitoring and analysis. The number and distribution of sampling points are dynamically adjusted through the adaptive sampling module to ensure that each sub-area is fully inspected, while minimizing unnecessary movement distance and computing resource consumption, thereby achieving efficient data processing and analysis.

[0113] A multi-layer MLP network with a Softmax activation function is used on k neighborhood points according to the aggregated features to obtain a mapping relationship between adjacent neighborhood points, a coordinate axis is established for the sampling points according to the mapping relationship, and a normalized weight of each coordinate axis and feature channel is obtained;

[0114] The inspection backbone network is segmented in real time according to the normalized weights to obtain real-time rail transit backbone lines.

[0115] In this embodiment, the neighborhood of each sampling point is grouped and searched using the K-NN algorithm, and the features of the neighborhood points are aggregated using an aggregation function. This can more accurately capture subtle changes and features on the track surface, which helps improve the detection accuracy of track defects. Even when the data is not completely consistent or noisy, the self-attention mechanism can maintain high recognition accuracy by focusing on the most relevant local features, thereby enhancing the system's adaptability to different environmental conditions.

[0116] Obtaining coordinates and data features between track data sampling points based on the rail transit trunk line, including:

[0117] Locating the target rail transit trunk line through the sampling points on the rail transit trunk line, using the XY coordinates of the two-dimensional image to index the corresponding point cloud data, establishing the same coordinate system for all the point cloud data, and obtaining the coordinates of the track data;

[0118] In this embodiment, by converting all point cloud data into the same coordinate system, it is ensured that data from different sources can be spatially aligned, which not only improves the consistency of the data, but also provides a solid foundation for subsequent analysis. By using the XY coordinate index of the two-dimensional image to correspond to the point cloud data, the sampling points on the target rail transit trunk line can be located more accurately, reducing the errors caused by coordinate mismatch.

[0119] The local features of the point cloud data are obtained according to the normalized weights, and the area of each feature point is expanded by segmenting the inspection backbone network to obtain the global data features of the track data.

[0120] In this embodiment, normalized weights are used to extract local features of point cloud data, allowing the system to more carefully capture subtle changes in the track surface, such as cracks and wear, which helps improve the detection capability of small-scale defects. By segmenting the inspection backbone network and expanding the area of each feature point, the overall condition of the track can be understood from a macro perspective, and a complete track health assessment model can be formed by combining local features. The comprehensiveness of the system is enhanced by combining the global and local approaches.

[0121] A track monitoring framework based on a three-dimensional point cloud is constructed based on the coordinates and data features between the track data sampling points, including:

[0122] Using the coordinates and data features between the track data sampling points as high-quality point cloud data of the track in an initial state, and using the high-quality point cloud data to construct a track reference model;

[0123] Acquire real-time daily inspection data of the inspection robot, and update the daily inspection data into the track reference model after each inspection to obtain real-time track status;

[0124] In this embodiment, high-quality point cloud data in the initial state is used to construct a track baseline model, which helps to accurately identify any changes that deviate from the normal state. New data is updated to the baseline model after each inspection to ensure that the model always reflects the latest track conditions, thereby improving the accuracy of long-term monitoring. In addition, the system can obtain and process data collected by the inspection robot in real time, quickly discover potential problems, and take immediate measures to reduce risks caused by delays. Based on the real-time status monitoring data, the system can dynamically adjust the inspection strategy or maintenance plan according to the current track condition to ensure timely response to emergencies.

[0125] In this embodiment, regardless of severe weather or complex geographical conditions, the deep learning-based method has strong adaptability and can enhance the generalization ability of the model by training a diverse data set. Through preprocessing and feature extraction, it can effectively reduce the occurrence of false positive results and enhance the system's resistance to noise.

[0126] Setting a status indicator for the track state based on the track benchmark model and historical data, and setting a threshold for the status indicator. If the threshold is exceeded, the track will be considered to be potentially dangerous;

[0127] Performing data monitoring on the real-time track status, using the monitoring data as training samples to automatically detect using a machine learning algorithm, monitoring the track status in real time, and automatically marking abnormal areas;

[0128] A visualization platform is built as the visualization interface of the track monitoring framework to display the overall status of the track in real time.

[0129] In this embodiment, a visualization platform is constructed as a visualization interface of the track monitoring framework, which can display the overall status of the track and the location and details of any abnormal areas in real time, making it easier for users to understand and manage. The visualization platform can also provide interactive functions, allowing users to zoom in, zoom out, or view details of specific areas, further enhancing the user experience. It can monitor the track status 24 hours a day and immediately issue an alarm if a problem is found to ensure that the problem is handled in a timely manner.

[0130] In this embodiment, through the automated process, the inspection work becomes more efficient, the possibility of human error is reduced, and the overall work efficiency is improved. The automated monitoring and identification process reduces the need for manual inspection and reduces labor costs.

[0131] The track status is monitored in real time using an edge detection method on the data features to obtain track status monitoring data, including:

[0132] Select any track in the inspection backbone network on the track monitoring framework, project each point on the track onto a two-dimensional coordinate plane, and use a KD-Tree to perform a neighbor search with any point P as the center in the two-dimensional coordinate plane to obtain M neighboring points. The neighborhood set of any point P is defined as M p , whose expression is:

[0133] M p ={p0∈P|p0∈M p}

[0134] Among them, p0 is one of the neighbors of P, and p0 is the neighborhood set M p one of the;

[0135] In this embodiment, it is chosen to project all points on the track to the XOY coordinate plane to avoid calculating the normal vector of each point. This can greatly improve the projection efficiency. The segmented track point cloud is projected to the XOY plane. Projecting the point cloud to the coordinate plane only requires setting the coordinate value of a certain dimension to a constant, which greatly improves the calculation efficiency.

[0136] The M neighborhood sets corresponding to any point P are connected with the rest of the neighborhood points with point P as the center, and a series of angles θ={θ1,θ2,…,θ m}, sort the angles θ to get the maximum value θ max ;

[0137] In this embodiment, by performing a neighborhood search using KD-Tree and calculating the angle between each point and its neighboring points, subtle changes in the track surface, such as cracks and wear, can be captured more carefully. This method helps improve the ability to detect small-scale defects. By using the maximum angle to determine the boundary point, the boundary or abnormal area of the track surface can be accurately identified, reducing the false alarm rate and missed detection rate.

[0138] The maximum angle between two consecutive points is used to calculate the boundary probability. If the maximum angle is greater than a set threshold, point P is defined as a boundary point.

[0139] By traversing each point to determine whether it is a boundary point, all boundary points are obtained, and two straight lines of edge points are obtained by straight line fitting of the boundary points. Track boundary data is simulated in real time on the visualization interface to obtain monitoring data of the track status.

[0140] In this embodiment, a point cloud segmentation network based on deep learning is used to segment the track line. A large amount of track point cloud data can be used to train the neural network, and the point cloud segmentation model can be obtained more efficiently. The deep learning segmentation algorithm can accurately identify the track components and no longer rely on the prior information of the track geometry. The segmented point cloud is projected onto the coordinate plane for edge detection, and then the upper and lower edge straight line fitting is completed. The measurement accuracy of the track wear parameters is improved by measuring the Euclidean distance between the two lines.

[0141] The track condition monitoring data is traversed and detected using a neural network, including:

[0142] Preprocessing the track status monitoring data, annotating the monitoring data, and integrating the track status monitoring data into a unified framework using a multimodal fusion network;

[0143] A sliding window mode is used to perform traversal detection block by block, and the data in each window is fed into a neural network for training to obtain track status anomaly data;

[0144] In this embodiment, a sliding window approach is used to perform traversal detection block by block, which can effectively process large-scale data sets while ensuring that each local area can be fully inspected. This block processing method not only improves computational efficiency but also reduces memory usage.

[0145] Performing abnormality location and abnormality classification on the track state abnormality data, calculating confidence scores for the abnormality location and abnormality classification, and setting thresholds to filter out abnormal data with low confidence;

[0146] Adjacent abnormal areas are merged into a whole, the detection results are visualized, and superimposed on the track visualization interface to intuitively display the location and type of the abnormal area.

[0147] In this embodiment, the powerful feature extraction capability of the neural network can be utilized to automatically identify high-level features from complex track status monitoring data. By preprocessing the monitoring data, the impact of noise on the detection results can be effectively reduced, thereby improving the robustness of the system. Whether in adverse weather conditions or complex background interference, the deep learning-based method has strong adaptability and can enhance the generalization ability of the model by training a diverse data set.

[0148] Establish a dynamic adjustment and optimization recognition model for historical inspection data, obtain a panoramic image dataset of traffic tracks in real time, and automatically monitor and identify traffic tracks, including:

[0149] The monitoring data of the abnormal area is subjected to multimodal fusion using an adaptive mechanism, and the inspection robot is revisited in the corresponding abnormal area to obtain potential recognition errors by comparing the current detection results with historical records;

[0150] In this embodiment, new data is updated to the baseline model after each inspection to ensure that the model always reflects the latest track conditions, thereby improving the accuracy of long-term monitoring. By continuously collecting track status monitoring data, a long-term historical database can be established, which helps to analyze the changing trends of track health conditions and provide early warning of possible problems.

[0151] The corrected track data is added to the training set, the track model of the track monitoring framework is optimized, and the panoramic image information obtained by the inspection robot is processed using a panoramic image target recognition algorithm;

[0152] The improved TinyYOLOv3 neural network model uses a deep learning algorithm to perform target recognition on the panoramic image information, obtain a full set of rail transit data, and automatically monitor and identify the status of the traffic track in real time.

[0153] The panoramic image target recognition algorithm includes:

[0154] Generating a panoramic image of the track area using the multi-source rail transit data collected by the inspection robot, and annotating historical and newly collected panoramic images;

[0155] In this embodiment, by integrating data from multiple sensors such as lidar and cameras to generate panoramic images, a more comprehensive and accurate description of the track environment can be provided, thereby enhancing the robustness and adaptability of the system. By using an improved TinyYOLOv3 neural network model, high-level features can be effectively and automatically extracted from complex panoramic images, thereby improving the detection accuracy of track defects.

[0156] The annotated panoramic image is subjected to data preprocessing, and an improved TinyYOLOv3 neural network model is used to extract features from the multi-source rail transit data to obtain track-related features;

[0157] In this embodiment, TinyYOLOv3, as a lightweight target detection model, can achieve fast reasoning while maintaining high accuracy and is suitable for real-time applications. This allows the system to immediately detect potential problems during the inspection process and take quick action. By reasonably segmenting the panoramic image, it can effectively manage large-scale data sets, improve computing efficiency and reduce memory usage.

[0158] The abnormal area of the track is calculated using weighted cross entropy loss, and the track status is regularly evaluated by repeatedly revisiting the abnormal area by the inspection robot;

[0159] The track status is synchronized to the visualization platform in real time, and the working status of the inspection robot, the health status of the track and abnormal areas are displayed in real time.

[0160] In this embodiment, the track status is synchronized to the visualization platform in real time, which can intuitively display the working status of the inspection robot, the health status of the track, and the location and details of any abnormal areas, making it easier for users to understand and manage.

[0161] The present invention adopts a point cloud segmentation network based on deep learning to segment the track line, uses a large amount of track point cloud data to train the neural network, and obtains the point cloud segmentation model more efficiently. The segmentation algorithm using deep learning can accurately identify track components, no longer relying on prior information about the track geometry, thereby improving the accuracy of track monitoring.

[0162] Multi-source rail transit data is collected in the form of multi-source data fusion to generate high-quality panoramic images. The improved TinyYOLOv3 neural network model is used for feature extraction to achieve high-precision detection and real-time monitoring of track defects. Adaptive sampling and dynamic adjustment mechanisms are adopted, combined with sliding window mode and edge detection algorithm to optimize the identification and classification of abnormal areas, and intuitively display the health status and abnormal locations of the track through a visualization platform. It supports 24 / 7 uninterrupted monitoring and preventive maintenance, significantly improves inspection efficiency, reduces labor costs, and ensures the safe and stable operation of rail transit. It not only enhances the robustness and adaptability of the system, but also realizes effective monitoring and management of track status.

[0163] The above embodiments are merely exemplary embodiments of the present application and are not intended to limit the scope of the present application. The scope of protection of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and scope of protection of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the scope of protection of the present application.

Claims

1. A multi-dimensional monitoring and identification method for a rail transit inspection robot, characterized in that: The following steps are involved: By setting up a laser radar and visual-inertial fusion synchronous positioning module on the inspection robot, a rail transit map is constructed, track data is obtained in real time, and the said track data is used to construct an inspection backbone network based on the rail transit map; An adaptive module is used in the inspection backbone network to segment the rail transit trunk line in real time, and a self-attention mechanism is used to obtain the similarity between the local neighborhood points of the track data, and the coordinates and data features between the track data sampling points are obtained; A track monitoring framework based on a three-dimensional point cloud is constructed according to the coordinates and data features between the track data sampling points, and an edge detection method is used to monitor the track status in real time based on the data features to obtain track status monitoring data; A neural network is used to perform traversal detection on the track status monitoring data, a dynamic adjustment and optimization recognition model for historical inspection data is established, a panoramic image data set of the traffic track is obtained in real time, and the traffic track is automatically monitored and identified.

2. A multi-dimensional monitoring and identification method for a rail transit inspection robot according to claim 1, characterized in that: By installing a laser radar and visual-inertial fusion synchronous positioning module on the inspection robot, a rail transit map is constructed and track data is obtained in real time, including: LiDAR is used to collect 3D point cloud data of the track and surrounding environment. Multiple cameras installed on the inspection robot capture image information of the surrounding environment. Combined with the inertial measurement unit (IMU), the robot's acceleration and angular velocity changes are recorded to obtain multi-source rail transit data. Denoising the multi-source rail transit data, extracting visual feature data, optimizing the multi-source rail transit data using a synchronized time reference, and obtaining time-synchronized multi-source fusion data; Extracting the pose estimation data of the inspection robot in the current frame of the multi-source fusion data, constructing a rail transit map, continuously detecting the robot's access area, performing loop closure operations, and optimizing the global rail transit map in real time; The track status is continuously monitored according to the global rail transit map to obtain real-time track data.

3. The multi-dimensional monitoring and identification method of a rail transit inspection robot according to claim 2 is characterized in that: The rail data is used to construct an inspection backbone network based on the rail transit map, including: Generate the original point cloud set (x1, x2, ..., x n ), the maximum pooling layer symmetric strategy of the PointNet network is used to process the original point cloud set (x1, x2, ..., x n ) is disordered, and the ordered original point cloud set (x1, x2, ..., x n ) as network input data; Multi-layer perceptron MLP is used to extract features and transform the original point cloud set (x1, x2, ..., x n ) is mapped to a 1024-dimensional feature space, and then the maximum pooling layer is used to process the high-dimensional features to obtain a point cloud segmentation network. The expression of the point cloud segmentation network is: f({x1,x2,…,x n })=ρ(s(h(x1),h(x2),…,h(x3))) Among them, f represents the point cloud countermeasure segmentation function, h represents the feature extraction layer, s represents the symmetric function, and ρ represents the 1024-dimensional feature extraction layer; Different scale ranges are established for the points used in the point cloud segmentation network, and several regions with different radius sizes are divided with the sampling points as the center. Each different scale region contains a different number of point clouds. The scale features of each sampling point are trained using a multi-layer perceptron (MLP) to obtain local area features. The backbone network is segmented according to the local area characteristics to obtain an inspection backbone network.

4. The multi-dimensional monitoring and identification method of a rail transit inspection robot according to claim 3 is characterized in that: In the inspection backbone network, an adaptive module is used to segment the rail transit trunk line in real time, and a self-attention mechanism is used to obtain the similarity between the local neighborhood points of the track data, including: In the inspection backbone network, N downsampled input points are extracted. s points, extract the feature set F of the downsampled point set s , for the N s The farthest sampling FPS is used for each down-sampling point to obtain relatively uniform original sampling points; The adaptive sampling module is used to process the original sampling points, and the K-NN algorithm is used to group and search the neighborhood points of each sampling point. The aggregation function is used to aggregate the features of each neighborhood to all other neighborhood points to obtain the final neighborhood aggregation feature g i,k , whose expression is: Among them, x i,k Represents the sampling point x i The kth neighboring point of i,j Represents the sampling point x i The jth neighboring point of i,k ,x i,j ) represents the relationship function, calculate the sampling point x i and its high-dimensional relationship between the neighborhood, γ(x i,j ) represents the function of the number of conversion channels, N(x i ) represents the sampling point x i The feature set of the kth neighborhood point, σ represents the aggregation function, g i,k Represents the aggregated features of the kth neighborhood point of the i-th sampling point after updating; A multi-layer MLP network with a Softmax activation function is used on k neighborhood points according to the aggregated features to obtain a mapping relationship between adjacent neighborhood points, a coordinate axis is established for the sampling points according to the mapping relationship, and a normalized weight of each coordinate axis and feature channel is obtained; The inspection backbone network is segmented in real time according to the normalized weights to obtain real-time rail transit backbone lines.

5. The multi-dimensional monitoring and identification method of a rail transit inspection robot according to claim 3 is characterized in that: Obtaining coordinates and data features between track data sampling points based on the rail transit trunk line, including: Locating the target rail transit trunk line through the sampling points on the rail transit trunk line, using the XY coordinates of the two-dimensional image to index the corresponding point cloud data, establishing the same coordinate system for all the point cloud data, and obtaining the coordinates of the track data; The local features of the point cloud data are obtained according to the normalized weights, and the area of each feature point is expanded by segmenting the inspection backbone network to obtain the global data features of the track data.

6. The multi-dimensional monitoring and identification method of a rail transit inspection robot according to claim 5, characterized in that: A track monitoring framework based on a three-dimensional point cloud is constructed based on the coordinates and data features between the track data sampling points, including: Using the coordinates and data features between the track data sampling points as high-quality point cloud data of the track in an initial state, and using the high-quality point cloud data to construct a track reference model; Acquire real-time daily inspection data of the inspection robot, and update the daily inspection data into the track reference model after each inspection to obtain real-time track status; Setting a status indicator for the track state based on the track benchmark model and historical data, and setting a threshold for the status indicator. If the threshold is exceeded, the track will be considered to be potentially dangerous; Performing data monitoring on the real-time track status, using the monitoring data as training samples to automatically detect using a machine learning algorithm, monitoring the track status in real time, and automatically marking abnormal areas; A visualization platform is built as the visualization interface of the track monitoring framework to display the overall status of the track in real time.

7. The multi-dimensional monitoring and identification method of a rail transit inspection robot according to claim 6, characterized in that: The track status is monitored in real time using an edge detection method on the data features to obtain track status monitoring data, including: Select any track in the inspection backbone network on the track monitoring framework, project each point on the track onto a two-dimensional coordinate plane, and use a KD-Tree to perform a neighbor search with any point P as the center in the two-dimensional coordinate plane to obtain M neighboring points. The neighborhood set of any point P is defined as M p , whose expression is: M p ={p0∈P|p0∈M p } Among them, p0 is one of the neighbors of P, and p0 is the neighborhood set M p one of the; The M neighborhood sets corresponding to any point P are connected with the rest of the neighborhood points with point P as the center, and a series of angles θ={θ1,θ2,…,θ m }, sort the angles θ to get the maximum value θ max ; The maximum angle between two consecutive points is used to calculate the boundary probability. If the maximum angle is greater than a set threshold, point P is defined as a boundary point. By traversing each point to determine whether it is a boundary point, all boundary points are obtained, and two straight lines of edge points are obtained by straight line fitting of the boundary points. Track boundary data is simulated in real time on the visualization interface to obtain monitoring data of the track status.

8. The multi-dimensional monitoring and identification method of a rail transit inspection robot according to claim 7, characterized in that: The track condition monitoring data is traversed and detected using a neural network, including: Preprocessing the track status monitoring data, annotating the monitoring data, and integrating the track status monitoring data into a unified framework using a multimodal fusion network; A sliding window mode is used to perform traversal detection block by block, and the data in each window is fed into a neural network for training to obtain track status anomaly data; Performing abnormality location and abnormality classification on the track state abnormality data, calculating confidence scores for the abnormality location and abnormality classification, and setting thresholds to filter out abnormal data with low confidence; Adjacent abnormal areas are merged into a whole, the detection results are visualized, and superimposed on the track visualization interface to intuitively display the location and type of the abnormal area.

9. The multi-dimensional monitoring and identification method of a rail transit inspection robot according to claim 8, characterized in that: Establish a dynamic adjustment and optimization recognition model for historical inspection data, obtain a panoramic image dataset of traffic tracks in real time, and automatically monitor and identify traffic tracks, including: The monitoring data of the abnormal area is subjected to multimodal fusion using an adaptive mechanism, and the inspection robot is revisited in the corresponding abnormal area to obtain potential recognition errors by comparing the current detection results with historical records; The corrected track data is added to the training set, the track model of the track monitoring framework is optimized, and the panoramic image information obtained by the inspection robot is processed using a panoramic image target recognition algorithm; The improved TinyYOLOv3 neural network model uses a deep learning algorithm to perform target recognition on the panoramic image information, obtain a full set of rail transit data, and monitor and identify the status of the traffic track in real time and automatically.

10. The multi-dimensional monitoring and identification method of a rail transit inspection robot according to claim 9, characterized in that: The panoramic image target recognition algorithm includes: Generating a panoramic image of the track area using the multi-source rail transit data collected by the inspection robot, and annotating historical and newly collected panoramic images; The annotated panoramic image is subjected to data preprocessing, and an improved TinyYOLOv3 neural network model is used to extract features from the multi-source rail transit data to obtain track-related features; The abnormal area of the track is calculated using weighted cross entropy loss, and the track status is regularly evaluated by repeatedly revisiting the abnormal area by the inspection robot; The track status is synchronized to the visualization platform in real time, and the working status of the inspection robot, the health status of the track and abnormal areas are displayed in real time.

Images