Foreign matter detection method and system for safe operation of railway track

Abstract

The application provides a foreign matter detection method and system for safe operation of a railway track, relates to the technical field of image data processing, and periodically collects track image sequences and track point cloud sequences through a multi-source monitoring array; extracts key images from the track image sequences, generates track key image sequences, and maps to obtain track key point cloud sequences; uses the track key image sequences to perform point cloud fitting on the track key point cloud sequences, constructs a foreign matter simulation model distribution sequence, and performs foreign matter risk assessment based on the foreign matter simulation model distribution sequence to output a track foreign matter detection result. The application solves the technical problems in the prior art that due to single detection means, there are detection blind spots and poor environmental adaptability, resulting in insufficient detection precision and the inability to effectively evaluate the influence of foreign matter risks, achieves the technical effect of improving the accuracy of foreign matter detection, and further enhances the safety and reliability of track operation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image data processing, in particular to a foreign matter detection method and system for safe operation of railway track. BACKGROUND

[0002] With the rapid development of railway transportation industry, train speed and density are increasing, and foreign matter intrusion is increasingly threatening the safety of railways. The existing foreign matter detection methods for railway track mainly include contact detection and non-contact detection. Contact detection installs pressure, capacitance and other sensors on the track, which can trigger an alarm only when the foreign matter directly contacts the detection components. However, this method has response delay, poor detection effect for small or light foreign matters, and high maintenance cost due to environmental interference. Non-contact detection uses single detection means such as video monitoring system, laser radar or infrared sensor, which avoids the contact problem, but has limited detection accuracy and cannot accurately obtain detailed information of foreign matters. Moreover, it is easily affected by environmental factors such as weather and dust, and has a high false alarm rate. In addition, these methods usually use simple threshold or feature matching algorithms for detection, which cannot effectively distinguish the actual risk of different foreign matter intrusions on train operation, and thus cannot provide data support for train emergency response decision-making, affecting the safety of train operation. SUMMARY

[0003] The present application provides a foreign matter detection method and system for safe operation of railway track, which solves the technical problem of insufficient foreign matter detection accuracy and ineffective evaluation of foreign matter risk impact due to single detection means, detection blind area and poor environmental adaptability in the prior art, and achieves the technical effects of improving the accuracy of foreign matter detection, intelligently evaluating the impact of foreign matter risk, and enhancing the safety and reliability of track operation.

[0004] In view of the above problems, on the one hand, the present application provides a foreign matter detection method for safe operation of railway track, which comprises: periodically acquiring data of railway track by using a multi-source monitoring array arranged along the railway track, to obtain a track image sequence and a track point cloud sequence; extracting key images from the track image sequence to generate a track key image sequence, and mapping to obtain a track key point cloud sequence; fitting the track key point cloud sequence by using the track key image sequence, constructing a foreign matter simulation model distribution sequence according to the optimal point cloud fitting result sequence; and performing foreign matter risk assessment based on the foreign matter simulation model distribution sequence, and outputting the end foreign matter simulation model distribution and the end foreign matter risk assessment result as the current track foreign matter detection result.

[0005] In another aspect, the application also provides a foreign matter detection system for safe operation of a railway track, comprising: a track data acquisition module for periodically acquiring synchronous data of the railway track by using a multi-source monitoring array arranged along the railway track, to obtain a track image sequence and a track point cloud sequence; a key data extraction module for extracting key images from the track image sequence, to generate a track key image sequence and map a track key point cloud sequence; a foreign matter simulation model construction module for fitting the track key point cloud sequence by using the track key image sequence, to construct a foreign matter simulation model distribution sequence according to a sequence of optimal point cloud fitting results; and a foreign matter risk assessment module for performing foreign matter risk assessment based on the foreign matter simulation model distribution sequence, to output a terminal foreign matter simulation model distribution and a terminal foreign matter risk assessment result as a current track foreign matter detection result.

[0006] The one or more technical solutions provided in the application have at least the following beneficial effects:

[0007] By using the multi-source monitoring array arranged along the railway track, synchronous data of the track is periodically acquired to obtain a track image sequence and a point cloud sequence, which reflects the track and foreign matter from multiple angles, ensuring the comprehensiveness and real-time nature of the data and providing a rich source of information for subsequent analysis. By extracting key images from the track image sequence to generate a track key image sequence and map a track key point cloud sequence, the data processing amount is reduced, the processing efficiency is improved, and the representativeness and accuracy of the data are ensured. In addition, the mapping and fusion of images and point clouds enhance the complementarity of the data, providing more accurate data support for subsequent point cloud fitting. The track key point cloud sequence is fitted by using the track key image sequence, and a foreign matter simulation model distribution sequence is constructed based on the sequence of optimal point cloud fitting results. This process converts the collected data into an intuitive three-dimensional model through point cloud processing and simulation modeling technology, realizes accurate description and positioning of foreign matter, provides visual and quantitative basis for foreign matter risk assessment, and improves the accuracy and reliability of detection. Based on the foreign matter simulation model distribution sequence, risk assessment is performed, and a terminal foreign matter simulation model distribution and a terminal foreign matter risk assessment result are output as a current track foreign matter detection result. This step not only detects the existence of foreign matter, but also further assesses the safety risk of the railway operation, providing a scientific and comprehensive reference for train operation control and maintenance decision-making, and promoting the improvement of railway operation management level.

[0008] In summary, the present application synchronously collects image and point cloud data through a multi-source monitoring array, fuses the advantages of different sensors, avoids the limitations of a single data source, and improves the comprehensiveness and accuracy of detection. Through the extraction of key images and key point clouds, unnecessary data redundancy is reduced, and data processing efficiency is improved. At the same time, the process of point cloud fitting and simulation modeling further improves the accuracy and reliability of the data, making the detection results more timely and accurate. The introduction of the foreign matter risk assessment link, combined with the simulation model distribution sequence, can comprehensively and deeply analyze the potential threat of foreign matters to railway operation and output the corresponding risk assessment results. Overall, the present application improves the accuracy of foreign matter detection and realizes the leap from detection to decision support through risk assessment, providing comprehensive protection for railway safe operation and enhancing the safety and reliability of railway track operation.

[0009] The above description is only a summary of the technical solutions of the present application. In order to more clearly understand the technical means of the present application, the following specific embodiments of the present application can be implemented in accordance with the content of the specification, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 The flowchart of the foreign matter detection method for safe operation of railway track provided by the embodiments of the present application is shown.

[0011] Figure 2 The flowchart of generating a key image sequence of the track in the foreign matter detection method for safe operation of railway track provided by the embodiments of the present application is shown.

[0012] Figure 3 The structural diagram of the foreign matter detection system for safe operation of railway track provided by the embodiments of the present application is shown.

[0013] Explanation of reference signs: track data acquisition module 10, key data extraction module 20, foreign matter simulation model construction module 30, foreign matter risk assessment module 40. DETAILED DESCRIPTION

[0014] The embodiments of the present application provide a foreign matter detection method and system for safe operation of railway track, which solves the technical problem that the existing technology has a single detection means, has a detection blind area and poor environmental adaptability, resulting in insufficient foreign matter detection accuracy and inability to effectively evaluate the impact of foreign matter risk, and achieves the technical effects of improving the accuracy of foreign matter detection, intelligently evaluating the impact of foreign matter risk, and further enhancing the safety and reliability of track operation.

[0015] Embodiment one, as shown in the following table, the embodiments of the present application provide a foreign matter detection method for safe operation of railway track, which comprises: Figure 1 ​

[0016] Step S1: Regularly acquire data of the railway track synchronously by using a multi-source monitoring array arranged along the railway track, to obtain a track image sequence and a track point cloud sequence.

[0017] Specifically, the multi-source monitoring array is a monitoring system composed of multiple different types of sensors, including at least a laser radar and a camera. The laser radar obtains distance information of an object by emitting a laser beam and receiving reflected light, thereby constructing point cloud data; the camera can take images of the track to provide visual information of the track. The track image sequence is a sequence composed of continuous images taken by a high-definition camera, reflecting the visual information of the track and its surrounding environment. The track point cloud sequence is a sequence composed of continuous point cloud data generated by the laser radar, reflecting the three-dimensional spatial information of the track and its surrounding environment.

[0018] At key positions along the railway track, such as turnouts and tunnel entrances, laser radars and high-definition cameras are installed, and the high-definition cameras are installed at the same point as the laser radars, forming a complementary monitoring network. For example, a set of sensors (laser radar + camera) can be arranged every 200 meters along the track, and the density can be increased to 100 meters in key areas (such as curves and slopes). The spatial registration of the laser radar and the camera is completed by using the chessboard calibration method to ensure that the detection areas completely coincide. The laser radar collects three-dimensional point cloud data of the track area at certain intervals and scanning frequencies, and the camera takes high-definition images at the same time point to ensure the consistency of the data in time and space. The point cloud data and image data collected by the multi-source monitoring array are arranged in the order of acquisition to form the track point cloud sequence and the track image sequence. The track point cloud sequence and the track image sequence correspond one-to-one, i.e. each image data in the track image sequence has a unique set of point cloud data in the track point cloud sequence, and both have the same timestamp and position marker.

[0019] Through multi-source data acquisition, information of the track can be obtained from different angles to provide a rich data basis for subsequent foreign object detection.

[0020] Step S2: Extract key images from the track image sequence to generate a track key image sequence, and map to obtain a track key point cloud sequence.

[0021] Specifically, key image extraction is to extract the most representative and informative image frames from a large number of track image sequences. These image frames can reflect the key state of the track and potential foreign object information. Image processing algorithms are used to extract key images from track image sequences. For example, feature recognition-based algorithms can be used, such as finding track-specific shaped components or areas with large color differences, etc. These selected key images are arranged in chronological order to form a track key image sequence. Then, according to the time stamp, the extracted key images are matched with the point cloud data at the same time. Through spatial coordinate conversion algorithms, the point cloud data is mapped into the pixel coordinate system of the key image, thereby obtaining a track key point cloud sequence.

[0022] Key image extraction can reduce data processing, focus on the most useful image information for foreign object detection, and improve processing efficiency. The mapping operation establishes the connection between image data and point cloud data, so that both types of data can be used comprehensively for more accurate foreign object detection.

[0023] Step S3: using the track key image sequence, performing point cloud fitting on the track key point cloud sequence, and constructing a foreign object simulation model distribution sequence according to the optimal point cloud fitting result sequence.

[0024] Specifically, the track key point cloud sequence is preprocessed, such as filtering, denoising, etc., to remove noise points and outliers. Point cloud fitting algorithms (such as least squares-based surface fitting or RANSAC algorithm) are used to fit the point cloud data, generating a three-dimensional model of the foreign object. For example, for the point cloud data of a suspected foreign object on the track, according to the distribution pattern of the surrounding normal track point cloud, the parameters of the fitting model are adjusted to find the optimal point cloud fitting result sequence, each fitting result data in the fitting result sequence corresponds to the foreign object point cloud data at a time. Then, according to each point cloud fitting result in the optimal point cloud fitting result sequence, a simulation model of the foreign object is constructed, obtaining foreign object simulation models at different times, and arranging these models in chronological order to generate a foreign object simulation model distribution sequence. For example, the track key point cloud sequence contains point cloud data of a foreign object, and a cuboid model is fitted by the RANSAC algorithm, the size and position of the model are consistent with the actual foreign object. According to the fitting result, a three-dimensional simulation model of the foreign object is generated and added to the foreign object simulation model distribution sequence. This sequence can describe the shape, position, etc. of the foreign object on the track.

[0025] Through point cloud fitting and simulation model construction, complex point cloud data is converted into intuitive three-dimensional models, achieving accurate description and positioning of foreign objects, not only improving the accuracy of foreign object detection, but also providing visualization and quantification for subsequent risk assessment.

[0026] Step S4: performing foreign object risk assessment based on the foreign object simulation model distribution sequence, and outputting the end foreign object simulation model distribution and the end foreign object risk assessment result as the current track foreign object detection result.

[0027] Specifically, according to the foreign object simulation model distribution sequence, a preset risk assessment algorithm (such as a rule-based algorithm or a deep learning algorithm) is used to assess the risk of the foreign object. For example, according to factors such as the position of the foreign object (whether it is in the center of the track), the size (whether it exceeds the safety threshold), etc., the risk level is assessed. The foreign object simulation model distribution and the risk assessment result are output to the railway dispatching system or the monitoring platform for the staff to make decisions. For example, the output result can display the position coordinates, type, size and risk level (such as low risk, medium risk, high risk) of the foreign object. For example, the foreign object simulation model shows that a metal object is located in the center of the track and has a large size, and the risk assessment algorithm determines it as "high risk". The simulation model and risk assessment result of the foreign object are sent to the railway dispatching center to prompt the staff to take emergency measures (such as slowing down or stopping).

[0028] Through foreign object risk assessment, the risk of foreign objects can be accurately quantified and evaluated, providing decision-making basis for railway operation management departments, thereby improving the safety and reliability of railway track operation.

[0029] Further, as shown in Figure 2 Step S2 includes:

[0030] Step S21: selecting the first track image monitored earliest in the track image sequence, setting the first track image as the first track key image, and obtaining the second track image adjacent to the first track image.

[0031] Step S22: inputting the first track image and the second track image into an image analysis plug-in for similarity comparison, and outputting the first image similarity.

[0032] Step S23: if the first image similarity is less than a predetermined similarity threshold, setting the second track image as the second track key image.

[0033] Step S24: if the first image similarity is greater than or equal to the predetermined similarity threshold, discarding the second track image.

[0034] Step S25: continue to select iteratively until the track image is traversed, obtain a plurality of track key images, and sort to obtain the track key image sequence.

[0035] Specifically, in the track image sequence, the earliest collected track image is found according to the timestamp, and is set as the first track key image. Then, the next image adjacent to the first track key image, i.e., the second track image, is obtained according to the order or index relationship of image storage. The earliest monitored image is selected as the starting point, which provides a basic image for subsequent similarity comparison, and facilitates construction of the starting part of the track key image sequence, so as to gradually screen out the key images.

[0036] The image analysis plug-in is an algorithm module for analyzing and processing images, and can calculate the image similarity between two images. The image similarity is a numerical index for measuring the similarity degree of two images. The value is usually between 0 and 1, 0 indicating complete dissimilarity, and 1 indicating complete identity. The first track image and the second track image are input into the image analysis plug-in for similarity comparison, and the first image similarity is output, which provides a quantitative basis for judging whether the images are similar, so as to determine whether the images are included in the track key image sequence.

[0037] The predetermined similarity threshold is a threshold value set in advance, which is used to judge whether two images are similar enough. If the similarity is higher than or equal to the threshold value, it is considered that there is no significant change between the images; if the similarity is lower than the threshold value, it is considered that there is a significant change between the images. The first image similarity calculated is compared with the predetermined similarity threshold. If the first image similarity is less than the predetermined similarity threshold, it indicates that the second track image has a large difference from the first track key image and contains new information (such as the appearance of foreign matter on the track or the change of the track state), and at this time, the second track image is set as the second track key image.

[0038] When the first image similarity is greater than or equal to the predetermined similarity threshold, it indicates that the second track image is very similar to the first track key image and does not contain new important information, and at this time, the second track image is directly discarded and is not included in the track key image sequence, so as to reduce unnecessary data processing.

[0039] After the second track image is discarded or the second track key image is determined, the next round of iteration selection is continued in the same way. The original first track key image or the second track key image is taken as the first track key image of the next round, and then the image adjacent thereto is obtained for similarity comparison, and the cycle is repeated until all the images in the entire track image sequence are traversed. Finally, the obtained multiple track key images are sorted according to the order of acquisition, and the track key image sequence is obtained.

[0040] For example, there are 100 images in the track image sequence, the first image (frame 1) is selected as the first track key image, and the second frame image is selected as the second track image. The similarity of the two images is calculated using the image analysis plug-in, and the first image similarity is 0.85. The predetermined similarity threshold is 0.90, and the first image similarity is 0.85, which is less than the threshold, so the second track image is set as the second track key image. Then the images adjacent to the second track key image are obtained from the track image sequence for similarity comparison, and the above iteration process is repeated, and finally 10 key images are selected, which reflect the significant changes of the track state, forming a track key image sequence.

[0041] By traversing the entire image sequence for image similarity comparison, the key images are selected from a large number of track images, reducing the data amount while retaining important information of the track state changes, not only improving the data processing efficiency, but also enhancing the pertinence and accuracy of subsequent analysis. By setting the similarity threshold, the image change detection requirements in different scenarios can be flexibly adapted, providing a high-quality data basis for subsequent foreign matter detection and analysis.

[0042] Further, the image analysis plug-in in step S22 is constructed based on a twin network, and a plurality of sample track image groups and a plurality of image similarities are used for supervised training of the twin network until the network converges.

[0043] Specifically, the twin network is a deep learning network structure, which includes two identical sub-networks that share the same structure and parameters, and respectively processes two images to be compared, such as the first track image and the second track image described above. A plurality of sample track image groups and a plurality of image similarities are used for supervised training of the twin network to obtain the image analysis plug-in. The specific process includes:

[0044] First, a plurality of sample track image groups are collected, which are image pairs for training the Siamese network. Each sample image pair contains two images, which can be similar (positive sample pair) or dissimilar (negative sample pair), and the actual image similarity of each sample image pair is known (determined by manual annotation or expert evaluation, etc.). Then, the images in the sample track image groups (supervised input) are input into the two sub-networks of the Siamese network respectively. Each sub-network contains a convolutional layer and a fully connected layer. For example, ResNet or VGG can be used as the backbone network. The convolutional layer is used to extract the features of the input image; the fully connected layer is used to encode the extracted features into a fixed-length feature vector; and the output layer uses a Sigmoid activation function to output the similarity between the two images. The predicted similarity value is compared with the known image similarity (supervised output), and the parameters of the Siamese network are adjusted by gradient descent method according to the difference between the two (usually a loss function is used to quantify this difference, such as a contrastive loss function or a binary cross-entropy loss function). This process is repeated continuously, with new sample track image groups being used for training each time, until the loss function value of the Siamese network no longer decreases significantly, reaching the state of network convergence, and the trained Siamese network is output as the image analysis plug-in.

[0045] Further, in step S3, the track key point cloud sequence is fitted by point cloud using the track key image sequence, including:

[0046] Step S31: using the image contour recognition plug-in, the contour features of the plurality of track key images in the track key image sequence are extracted, and a plurality of image contour sets are obtained.

[0047] Step S32: based on the contour features of the track inherent facilities, the plurality of image contour sets are respectively subjected to similar contour elimination, and a plurality of foreign object contour sets are obtained.

[0048] Step S33: according to the plurality of foreign object contour sets, the plurality of track key point clouds in the track key point cloud sequence are respectively subjected to point cloud mapping fitting, and a plurality of optimal point cloud fitting results are output to construct an optimal point cloud fitting result sequence.

[0049] Specifically, the image contour recognition plug-in is an algorithm module for recognizing the contour of an object in an image, which extracts contour features based on a convolutional neural network. Each track key image in the track key image sequence is input into the image contour recognition plug-in in turn. The plug-in uses a preset contour recognition algorithm to process the image and obtain a plurality of image contour sets. Each image contour set corresponds to the contour information in a track key image. The image contour recognition plug-in can accurately extract the contour features in the track key image, and these contour features help to distinguish different objects on the track and provide basic data for subsequent identification of foreign object contours.

[0050] The track inherent facility contour feature refers to the contour feature possessed by the railway track itself and the facilities normally existing on the track (such as rails, sleepers, spikes, etc.). These features are predetermined and can be obtained by analyzing the normal track image or according to the design specifications of the track. For each image contour set, the contour features therein are compared with the pre-stored track inherent facility contour features. If a contour feature is highly similar to the track inherent facility contour feature, it is considered that the contour belongs to the track inherent facility and is removed from the image contour set. For example, the similarity between the contours is calculated using a shape matching algorithm (such as Hausdorff distance). Whether the contours are similar is determined according to a similarity threshold (such as 0.8). The contours that are not similar are retained to form the foreign object contour set. By removing similar contours, the contour interference of the track inherent facility can be effectively removed, and the foreign object contour set obtained is more focused on the possible foreign objects, thereby improving the accuracy of foreign object detection.

[0051] For each foreign object contour set, the foreign object contour information is mapped to the point cloud space according to the relationship between the image coordinates and the point cloud coordinates, and the corresponding track key point cloud in the track key point cloud sequence is found. Then, a point cloud fitting algorithm (such as the ICP algorithm or the least squares-based fitting algorithm) is used to fit the mapped track key point cloud, and the result with the smallest fitting error is selected as the optimal point cloud fitting result by adjusting the fitting parameters. The point cloud mapping and fitting are performed for each foreign object contour set, and the obtained multiple optimal point cloud fitting results are arranged in order to construct the optimal point cloud fitting result sequence.

[0052] Through image contour recognition, similar contour removal, and point cloud mapping and fitting, the accurate description and positioning of foreign objects in the track key point cloud sequence are realized, the precision of foreign object detection is improved, and intuitive three-dimensional information is provided for subsequent risk assessment.

[0053] Further, step S31 comprises:

[0054] Step S311: According to the historical track foreign object detection record, a sample track image set and a sample image contour set are collected, the sample track image set and the sample image contour set are taken as training data, and are equally divided into P parts to obtain P training sets, wherein P is an integer greater than 10.

[0055] Step S312: The P training sets are used to respectively supervise the training and cross-validation of the convolutional neural network, and P image contour recognition branches are obtained, which are combined to construct an image contour recognition plug-in.

[0056] Step S313: The similarity deviation between the track key image in the track key image sequence and the previous adjacent track key image is calculated respectively, and multiple similarity difference values are obtained.

[0057] Step S314: multiply the ratio of the similarity difference value and the historical maximum similarity difference value by P and take the integer part to obtain a plurality of adaptive branch call quantities.

[0058] Step S315: based on the plurality of adaptive branch call quantities, using the image contour recognition plug-in, performing contour feature extraction on a plurality of track key images in the track key image sequence.

[0059] Specifically, the historical track foreign matter detection record is the data record accumulated in the past when detecting foreign matters on the railway track, including the previously detected foreign matter information, the track image at that time and other related data. The sample track image set is a set of track images selected from the historical track foreign matter detection record for training. The sample image contour set is a set of image contours corresponding to the sample track image set after contour extraction, which is used as the standard output for supervised training. Data is mined from the historical track foreign matter detection record, and a sufficient number of sample track image sets and corresponding sample image contour sets are collected. Then, these data are divided into P parts in an equal division manner, and each part is used as an independent training set. Here, P is an integer greater than 10 to ensure that sufficient image contour recognition branches are obtained in the subsequent process and the accuracy of contour recognition is ensured.

[0060] For each training set, the sample track image set therein is used as the input, and the sample image contour set is used as the target output, and the convolutional neural network is supervised trained. In the training process, the convolutional layer of the convolutional neural network automatically learns the feature pattern in the image, the pooling layer performs dimension reduction processing on the features, and the fully connected layer performs the final classification or regression operation. At the same time, using the cross-validation method, different subsets are selected as the validation set in each training process, for example, in the first training, the 2nd to Pth parts are used as the training set, and the 1st part is used as the validation set; in the second training, the 1st, 3rd to Pth parts are used as the training set, and the 2nd part is used as the validation set, and so on. After such training and verification, P image contour recognition branches are obtained, and these branches are combined in parallel to build the image contour recognition plug-in.

[0061] For each track key image in the track key image sequence (except the first track key image), the similarity between it and the previous adjacent track key image is calculated using the image analysis plug-in. For example, for the 2nd track key image, the similarity between it and the 1st track key image is calculated, and so on. Then, the similarity between the current track key image and the adjacent track key image is subtracted by the similarity between the adjacent track key image and the previous adjacent track key image (if there is one), to obtain a similarity deviation. These deviation values are collected to obtain a plurality of similarity difference values. For example, the similarity between image A and image B is 0.9, and the similarity between image B and image C is 0.7, so the similarity deviation is 0.2. By calculating the similarity deviation, the degree of difference in feature changes between track key images can be understood, which helps to judge the change of track state and provides a basis for subsequent adjustment of contour recognition strategy according to the image change.

[0062] First, the historical maximum similarity difference value is obtained, which is the maximum similarity difference value recorded in the previous track image analysis process. Then, for each similarity difference value, the ratio of the similarity difference value to the historical maximum similarity difference value is calculated, and then the ratio is multiplied by P and rounded. For example, if the ratio of a similarity difference value to the historical maximum similarity difference value is 0.3 and P = 15, then the calculated number of adaptive branch calls is 0.3 x 15 = 4.5, and after rounding, it is 5. In this way, a plurality of adaptive branch call numbers are obtained. The number of adaptive branch calls is determined according to the relationship between the similarity difference value and the historical data, and different numbers of branches in the image contour recognition plug-in can be flexibly called according to the actual change of the track image, so as to more accurately extract contour features of the track key image and adapt to different track image states. The greater the deviation, the more branches are selected, and the higher the contour recognition accuracy is correspondingly; the smaller the deviation, the fewer branches are selected, which can reduce unnecessary waste of computing resources on the premise of ensuring recognition accuracy and improve contour recognition efficiency.

[0063] For each track key image in the track key image sequence, a corresponding number of image contour recognition branches are called from the image contour recognition plug-in according to the adaptive branch call number corresponding to the track key image to extract contour features of the track key image. For example, if the adaptive branch call number of a track key image is 3, then 3 branches in the image contour recognition plug-in are called to extract contour features of the track key image at the same time, and each branch may extract different aspects or different degrees of contour features, and finally the results of these branches are integrated to obtain the contour features of the track key image.

[0064] By constructing an image contour recognition plug-in based on a convolutional neural network, efficient and accurate contour feature extraction is realized. The method of dynamically adjusting the number of branch calls can flexibly adjust the computing resources according to the degree of image change, improving the processing efficiency and providing high-quality contour features for subsequent foreign matter detection.

[0065] Further, step S33 comprises:

[0066] Step S331: randomly select a first foreign object contour set and obtain a first track key point cloud corresponding to the first foreign object contour set.

[0067] Step S332: perform point cloud random fitting on the first track key point cloud according to the first foreign object contour set, output a first point cloud fitting result, and set the ratio of the number of point clouds falling into the first foreign object contour set to the total number of point clouds in the first track key point cloud as the first fitting accuracy.

[0068] Step S333: perform point cloud random fitting on the first track key point cloud again according to the first foreign object contour set, output a second point cloud fitting result, and calculate the second fitting accuracy.

[0069] Step S334: perform iterative fitting until a predetermined fitting number is reached, output the point cloud fitting result corresponding to the maximum fitting accuracy as the first optimal point cloud fitting result, and add it to the plurality of optimal point cloud fitting results.

[0070] Specifically, one of the plurality of foreign object contour sets is randomly selected and defined as the first foreign object contour set. Then, according to the timestamp of the track key image corresponding to the first foreign object contour set, the first track key point cloud corresponding to the first foreign object contour set is found. For example, the first foreign object contour set comes from the 5th track key image, and the corresponding track key point cloud is the 5th frame of point cloud data.

[0071] According to the shape and features of the first foreign object contour set, a random sample consensus algorithm (RANSAC) is used to perform point cloud random fitting on the first track key point cloud. A part of the points in the point cloud is randomly selected for fitting to obtain a preliminary point cloud fitting result, i.e., the first point cloud fitting result. Then, the coordinates of each point cloud point are judged to determine whether it is within the range defined by the foreign object contour set, and the number of point clouds falling into the first foreign object contour set is calculated. The number of point clouds falling into the first foreign object contour set is divided by the total number of point clouds in the first track key point cloud to obtain the first fitting accuracy. For example, there are 1000 points in the first track key point cloud, of which 300 points fall into the first foreign object contour set, so the first fitting accuracy is 300 / 1000=0.3.

[0072] Again using the first foreign object contour set as the target shape, the first track key point cloud is randomly fitted according to the point cloud fitting algorithm described above. Since each fitting is random, different fitting results will be obtained, i.e. the second point cloud fitting result. Then, according to the method of calculating the first fitting accuracy, the second fitting accuracy is calculated, i.e. the number of point clouds falling into the first foreign object contour set is counted again and divided by the total number of point clouds in the first track key point cloud.

[0073] Iterative fitting is continuously performed in the manner of random fitting of point clouds, calculation of fitting accuracy, and random fitting of point clouds again. When the number of iterations reaches the predetermined number of fittings, the iteration is stopped. Then, among all the obtained fitting results, the one with the highest fitting accuracy is found, which is set as the first optimal point cloud fitting result and added to the multiple optimal point cloud fitting results.

[0074] The random fitting method can effectively handle noise and outliers in the point cloud and improve the robustness of the fitting. Through multiple random fitting and iterative optimization, the result with the highest fitting accuracy is finally selected as the optimal point cloud fitting result, ensuring the accuracy and reliability of the point cloud fitting result, so as to accurately describe the three-dimensional shape and position of the foreign object and provide high-quality three-dimensional models for subsequent risk assessment.

[0075] Further, the step S3 of constructing the foreign object simulation model distribution sequence according to the optimal point cloud fitting result sequence comprises:

[0076] Step S34: Based on the multiple foreign object contour sets, color feature extraction is performed on the foreign object images in the multiple track key images respectively, and multiple foreign object color distribution sets are obtained.

[0077] Step S35: Three-dimensional modeling is performed according to the multiple optimal point cloud fitting results, and multiple foreign object three-dimensional model sets are obtained.

[0078] Step S36: Using the multiple foreign object color distribution sets, color mapping rendering is performed on the multiple foreign object three-dimensional model sets respectively, and multiple foreign object simulation model distributions are obtained, and a foreign object simulation model distribution sequence is constructed.

[0079] Specifically, for each foreign object contour set, the foreign object image part is located in the corresponding multiple track key images. Then, a color feature extraction algorithm is used, such as a color feature extraction algorithm based on histogram statistics, to count the distribution proportion of different colors in the foreign object image and other information, so as to obtain multiple foreign object color distribution sets. Each foreign object color distribution set corresponds to a foreign object contour set and reflects the color features of the foreign object in the track key image. In actual implementation, the OpenCV image processing library can be used for color feature extraction.

[0080] According to each optimal point cloud fitting result, modeling is performed using a three-dimensional modeling software (such as Blender). For example, a three-dimensional reconstruction algorithm based on point clouds can be used to construct the three-dimensional shape of the foreign object according to the spatial distribution and geometric relationship of the point clouds, thereby obtaining a plurality of foreign object three-dimensional model sets. Each foreign object three-dimensional model set corresponds to an optimal point cloud fitting result and represents the three-dimensional geometric structure of the foreign object.

[0081] For each foreign object three-dimensional model set and the corresponding foreign object color distribution set, color mapping is performed using a rendering engine (such as OpenGL or Vulkan) or a three-dimensional modeling software to map the color information in the foreign object color distribution set onto the surface of the foreign object three-dimensional model set according to certain rules (such as according to the normal direction of the model surface, texture coordinates, etc.), thereby obtaining a plurality of foreign object simulation model distributions. These foreign object simulation model distributions are more similar in appearance to actual foreign objects, and finally these foreign object simulation model distributions are constructed into a foreign object simulation model distribution sequence in order.

[0082] Through color feature extraction, three-dimensional modeling, and color mapping rendering, the two-dimensional color information and three-dimensional shape information of the foreign object are combined to generate a foreign object simulation model distribution sequence with color information, thereby accurately describing the three-dimensional shape and position of the foreign object and enhancing the visualization effect of the model through color information, making the identification and analysis of foreign objects more intuitive. In addition, the foreign object simulation model distribution sequence can provide more abundant information for subsequent risk assessment, so as to more accurately judge the type and potential threat of the foreign object, thereby significantly improving the accuracy of railway track foreign object detection.

[0083] Further, the step S34 of obtaining a plurality of foreign object color distribution sets further includes:

[0084] Step S34-1: Collecting the current real-time light intensity, real-time weather conditions, and current time period of the railway track.

[0085] Step S34-2: Color correcting a plurality of extracted foreign object color distribution sets according to the real-time light intensity, real-time weather conditions, and current time period, respectively, to obtain a plurality of foreign object color distribution sets.

[0086] Further, the step S34-2 includes:

[0087] Step one: Combined with historical foreign object detection image records, sample image extraction is performed according to the current time period, and high-frequency image clustering is performed to divide and determine a plurality of color correction regions, wherein the light intensity deviation of each color correction region is the same by default.

[0088] Step two: based on the convolutional neural network, a plurality of color correction branches are established according to the plurality of color correction regions, and a region-branch matcher is constructed in combination with the plurality of color correction regions.

[0089] Step three: using the region-branch matcher, a color correction branch of an adaptive region is selected based on the foreign object position coordinates of each foreign object color distribution, and a plurality of foreign object color distribution sets are color corrected according to the real-time illumination intensity and real-time meteorological conditions, and the plurality of foreign object color distribution sets are output.

[0090] Specifically, the real-time illumination intensity refers to the degree of light intensity in the environment where the railway track is located at the current time, which can be measured by light sensors and other devices. The illumination intensity will affect the presentation of object color. The real-time meteorological conditions include the current weather conditions (such as sunny, cloudy, rainy, etc.), atmospheric visibility and other meteorological factors, which will also affect the visual effect of object color. The current time period represents the specific time interval of image acquisition, such as morning, afternoon or specific time period. The illumination angle and intensity are different in different time periods, which will affect the presentation of object color. The light sensor and meteorological monitoring device are set along the railway track. The light sensor can obtain the current illumination intensity value in real time, and the meteorological monitoring device can monitor the current meteorological conditions, such as weather type, visibility and other related parameters. At the same time, the time stamp of track image acquisition is extracted to determine the current time period.

[0091] The color correction region is a plurality of regions divided by image clustering technology from the sample image, each region having similar illumination and color deviation characteristics. The color correction branch is a convolutional neural network submodel trained based on different color correction regions, which is specially used for image color restoration under specific illumination conditions. The region-branch matcher is an intelligent matching module, which can automatically select the most adaptive color correction branch for processing according to the position of the foreign object in the image and the current environmental conditions.

[0092] For each set of foreign object color distribution that has been preliminarily extracted, color correction is performed according to the collected real-time light intensity, real-time meteorological conditions and current time period, and self-adaptive adjustment is performed relying on the historical image database and machine learning model. Specifically, first, an image sample database is constructed in combination with historical foreign object detection image records, and the samples are derived from the track monitoring images collected along the railway track at different time periods and under different meteorological conditions in the past year. For the shooting time period (such as evening) of the current track detection image, the real-time light intensity (such as 80 Lux) and the real-time meteorological conditions (such as fog and haze), the image sample set under the corresponding conditions is extracted from the sample database, and the sample quantity is not less than 1000. The extracted image sample set is preprocessed, including image size unification (such as adjustment to 512x512), color gamut conversion (RGB to HSV) and denoising operation, and then the high-frequency feature vector of each image is obtained by using a high-frequency component extraction algorithm (such as Laplacian pyramid or Fourier transform). The high-frequency vectors of all images are input into the K-means clustering algorithm (K generally takes a value of 10-20), and the images are divided into several color correction regions according to the image light characteristics. The light intensity deviation threshold in each region is set to not more than ±15 Lux, and the color offset mean value is set within the HSV chroma range of ±5%, so as to ensure the color consistency and region representativeness of the clustering results.

[0093] For each color correction region divided in the foregoing, a convolutional neural network (CNN) model is separately established as the color correction branch of the region. The training data is the original image and the corresponding artificial color correction image (which can be generated by artificial color adjustment or reference to real object images) in the region. The network structure preferably uses a shallow U-Net or ResNet-18 backbone network, the input is the original image, and the output is the color correction image. During the training process, the loss function includes color deviation loss and structural similarity loss, the optimizer selects the Adam optimizer, the learning rate is set to 0.001, and the training rounds are more than 100 rounds to ensure network convergence, and several color correction branches corresponding to several color correction regions are obtained. The several color correction branches are saved in a modularized manner, and a region-branch matcher is constructed for automatically matching the corresponding color correction branch according to the position coordinates of the foreign object in the image. The internal rules of the matcher are based on the intersection calculation of image coordinates and region masks to ensure accurate branch calling.

[0094] In the execution of the track foreign matter detection task, the foreign matter position coordinates of each foreign matter color distribution region are extracted from the track image. The region-branch matcher is called to select the color correction branch model corresponding to the image region where the foreign matter is located. The image region containing the foreign matter color distribution is cropped to a standard input size (such as 128x128) and input into the matched color correction branch to perform forward inference and output the color-corrected foreign matter image segment. The color information in the output image is extracted to obtain the HSV mean value and distribution standard deviation, which are used to construct the final foreign matter color distribution set. This process can be performed in parallel for multiple foreign matters, and the final uniform illumination and color restored foreign matter color distribution data is obtained to provide real texture support for subsequent three-dimensional simulation modeling.

[0095] Further, step S4 includes:

[0096] Step S41: set the latest monitored foreign matter simulation model distribution in the foreign matter simulation model distribution sequence as the terminal foreign matter simulation model distribution, and obtain a plurality of terminal foreign matter distribution features, wherein the terminal foreign matter has a foreign matter structure feature and a foreign matter position coordinate.

[0097] Step S42: based on the foreign matter simulation model distribution sequence, respectively analyze the moving trend of the plurality of terminal foreign matter distribution features to determine a plurality of foreign matter moving trends.

[0098] Step S43: according to the plurality of foreign matter structure features, the plurality of foreign matter position coordinates and the plurality of foreign matter moving trends, respectively evaluate the risk of the plurality of terminal foreign matters, and output the terminal foreign matter risk evaluation result.

[0099] Specifically, the terminal foreign matter simulation model distribution is the last monitored foreign matter simulation model distribution in the foreign matter simulation model distribution sequence, reflecting the state of the foreign matter at the last monitoring. The terminal foreign matter distribution feature is some characteristics of the foreign matter obtained from the terminal foreign matter simulation model distribution, including the foreign matter structure feature (such as shape, size, etc.) and the foreign matter position coordinate (position on the track). In the foreign matter simulation model distribution sequence, the latest monitored foreign matter simulation model distribution is determined according to the time stamp, which is set as the terminal foreign matter simulation model distribution. Then the foreign matter structure feature and the foreign matter position coordinate and other information marked by the terminal foreign matter simulation model distribution are extracted to obtain a plurality of terminal foreign matter distribution features.

[0100] The movement trend analysis refers to a process of analyzing the position change of the foreign object at different times to determine the direction, speed, and other trends of the movement of the foreign object. Based on the distribution sequence of the foreign object simulation model, the position coordinates of the foreign object in each end foreign object distribution feature are compared and analyzed. For example, the current end foreign object position coordinates are compared with the foreign object position coordinates at previous monitoring time. If the foreign object position coordinates gradually increase in a certain direction of the track, it can be determined that the foreign object has a movement trend in this direction. By calculating the difference of the position coordinates at different times and the time interval, the movement speed and other information of the foreign object can be determined, so as to determine the movement trends of multiple foreign objects. Mathematical calculation methods can be used, such as calculating the average value of the difference of the position coordinates at adjacent times to represent the movement speed.

[0101] For each end foreign object, risk assessment is performed according to the corresponding foreign object structure feature, foreign object position coordinates, and foreign object movement trend. For example, if the foreign object structure feature indicates that it is a large and sharp object, the position coordinates are located at a critical part of the track (such as a rail joint), and the movement trend is to move towards the center of the track, the risk degree of this foreign object will be high. According to the pre-set risk assessment rules, the weight of each factor can be set, and then the end foreign object risk assessment result can be output by weighted calculation. For example, the structure feature weight is 0.4, the position coordinate weight is 0.3, and the movement trend weight is 0.3. According to the actual situation of each factor, the score (0-1) is given, and then the total score is calculated to obtain the risk assessment result.

[0102] Through the analysis of the end foreign object simulation model distribution and the risk assessment, the comprehensive assessment of the potential threat of the foreign object is realized. Through the movement trend analysis, the movement direction and speed of the foreign object can be predicted, so that measures can be taken in advance. Combined with the structure feature, position coordinates, and movement trend of the foreign object, the risk assessment can provide scientific and accurate decision support for railway operation, not only improve the accuracy of foreign object detection, but also significantly enhance the safety and reliability of railway operation, and provide scientific basis for railway operation safety management, so as to take timely measures to deal with possible dangers.

[0103] In summary, the foreign object detection method for safe operation of railway track provided by the embodiments of the present application has the following beneficial effects:

[0104] The embodiment of the application synchronously collects image and point cloud data through a multi-source monitoring array, fuses the advantages of different sensors, avoids the limitations of a single data source, and improves the comprehensiveness and accuracy of detection. Through the extraction of key images and key point clouds, unnecessary data redundancy is reduced, and the data processing efficiency is improved. At the same time, the process of point cloud fitting and simulation modeling further improves the accuracy and reliability of the data, so that the detection result is more timely and accurate. The foreign matter risk assessment link is introduced, combined with the simulation model distribution sequence, the structure characteristics, position coordinates and movement trend of the foreign matter are comprehensively considered, and the potential threat of the foreign matter to the train operation is predicted in advance. Overall, the embodiment of the application improves the accuracy of foreign matter detection, and realizes the leap from detection to decision support through risk assessment, provides comprehensive protection for safe operation of the railway, and enhances the safety and reliability of the railway track operation.

[0105] Embodiment two, as shown in the same inventive concept as the preceding embodiment one, the embodiment of the application provides a foreign matter detection system for safe operation of a railway track, the system comprises: Figure 3

[0106] The track data acquisition module 10 is used for periodically acquiring synchronous data of the railway track by using the multi-source monitoring array arranged along the railway track, and acquiring a track image sequence and a track point cloud sequence.

[0107] The key data extraction module 20 is used for extracting key images from the track image sequence, generating a track key image sequence, and mapping a track key point cloud sequence.

[0108] The foreign matter simulation model construction module 30 is used for fitting the track key point cloud sequence by using the track key image sequence, and constructing a foreign matter simulation model distribution sequence according to an optimal point cloud fitting result sequence.

[0109] The foreign matter risk assessment module 40 is used for assessing the risk of foreign matters based on the foreign matter simulation model distribution sequence, and outputting a terminal foreign matter simulation model distribution and a terminal foreign matter risk assessment result as a current track foreign matter detection result.

[0110] Further, the key data extraction module 20 of the embodiment of the application is further used to perform the following steps:

[0111] ​selecting a first track image earliest monitored in the track image sequence, setting the first track image as a first track key image, and obtaining a second track image adjacent to the first track image; inputting the first track image and the second track image into an image analysis plug-in for similarity comparison, and outputting a first image similarity; if the first image similarity is less than a predetermined similarity threshold, setting the second track image as a second track key image; if the first image similarity is greater than or equal to the predetermined similarity threshold, discarding the second track image; continuing iteration until the track image is traversed, obtaining a plurality of track key images, and obtaining the track key image sequence after sorting.

[0112] Further, the image analysis plug-in is constructed based on a twin network, and the twin network is supervised trained by using a plurality of sample track image groups and a plurality of image similarities until the network converges.

[0113] Further, the foreign matter simulation model construction module 30 further comprises:

[0114] a contour feature extraction module, configured to perform contour feature extraction on a plurality of track key images in the track key image sequence by using an image contour recognition plug-in, and obtain a plurality of image contour sets.

[0115] a similar contour elimination module, configured to respectively eliminate similar contours from the plurality of image contour sets based on track inherent facility contour features, and obtain a plurality of foreign matter contour sets.

[0116] a point cloud mapping fitting module, configured to respectively perform point cloud mapping fitting on a plurality of track key point clouds in the track key point cloud sequence according to the plurality of foreign matter contour sets, and output a plurality of optimal point cloud fitting results to construct an optimal point cloud fitting result sequence.

[0117] Further, the contour feature extraction module is further configured to perform the following steps:

[0118] According to historical track foreign matter detection records, a sample track image set and a sample image contour set are collected, the sample track image set and the sample image contour set are taken as training data, and are equally divided into P parts to obtain P training sets, wherein P is an integer greater than 10; the P training sets are respectively used to supervise training and cross-validation of a convolutional neural network, P image contour recognition branches are obtained, and an image contour recognition plug-in is constructed by combination; a similarity deviation between a track key image and a previous adjacent track key image in the track key image sequence is calculated respectively, and a plurality of similarity difference values are obtained; a ratio of the similarity difference value to a historical maximum similarity difference value is multiplied by P to obtain a plurality of adaptive branch calling quantities; based on the plurality of adaptive branch calling quantities, the image contour recognition plug-in is used to perform contour feature extraction on the plurality of track key images in the track key image sequence.

[0119] Further, the point cloud mapping fitting module is further configured to perform the following steps:

[0120] randomly selecting a first foreign object contour set, and obtaining a first track key point cloud corresponding to the first foreign object contour set; performing point cloud random fitting on the first track key point cloud according to the first foreign object contour set, outputting a first point cloud fitting result, and setting a ratio of a number of point clouds falling into the first foreign object contour set to a total number of point clouds in the first track key point cloud as a first fitting accuracy; performing point cloud random fitting on the first track key point cloud according to the first foreign object contour set again, outputting a second point cloud fitting result, and calculating a second fitting accuracy; performing iterative fitting until a predetermined fitting number is reached, outputting a point cloud fitting result corresponding to a maximum fitting accuracy as a first optimal point cloud fitting result, and adding the first optimal point cloud fitting result to the plurality of optimal point cloud fitting results.

[0121] Further, the foreign object simulation model construction module 30 of the embodiment of the application further comprises:

[0122] The color feature extraction module is configured to perform color feature extraction on foreign object images in a plurality of track key images based on the plurality of foreign object contour sets, and obtain a plurality of foreign object color distribution sets.

[0123] The three-dimensional modeling module is configured to perform three-dimensional modeling according to the plurality of optimal point cloud fitting results, and obtain a plurality of foreign object three-dimensional model sets.

[0124] The color mapping rendering module is configured to perform color mapping rendering on the plurality of foreign object three-dimensional model sets respectively by using the plurality of foreign object color distribution sets, obtain a plurality of foreign object simulation model distributions, and construct a foreign object simulation model distribution sequence.

[0125] Further, the color feature extraction module is further configured to perform the following steps:

[0126] Collect the current real-time light intensity, real-time meteorological conditions and the current time period of the railway track; according to the real-time light intensity, real-time meteorological conditions and the current time period, color correction is performed on the plurality of extracted foreign matter color distribution sets respectively, to obtain a plurality of foreign matter color distribution sets; wherein, according to the real-time light intensity, real-time meteorological conditions and the current time period, color correction is performed on the plurality of extracted foreign matter color distribution sets respectively, including: combining the historical foreign matter detection image record, sample image extraction is performed according to the current time period, high-frequency image clustering is performed, and a plurality of color correction regions are determined, wherein the light intensity deviation of each color correction region is the same by default; based on the convolutional neural network, a plurality of color correction branches are established according to the plurality of color correction regions, and a region-branch matcher is constructed in combination with the plurality of color correction regions; using the region-branch matcher, the color correction branch of the adaptive region is selected based on the foreign matter position coordinates of each foreign matter color distribution, and color correction is performed on the plurality of extracted foreign matter color distribution sets according to the real-time light intensity and real-time meteorological conditions, to output a plurality of foreign matter color distribution sets.

[0127] Further, the foreign matter risk assessment module 40 of the embodiment of the present application is also used to perform the following steps:

[0128] The latest monitored foreign matter simulation model distribution in the foreign matter simulation model distribution sequence is set as the terminal foreign matter simulation model distribution, and a plurality of terminal foreign matter distribution characteristics are obtained, wherein the terminal foreign matter has foreign matter structure characteristics and foreign matter position coordinates; based on the foreign matter simulation model distribution sequence, the plurality of terminal foreign matter distribution characteristics are respectively subjected to moving trend analysis to determine a plurality of foreign matter moving trends; according to the plurality of foreign matter structure characteristics, the plurality of foreign matter position coordinates and the plurality of foreign matter moving trends, the plurality of terminal foreign matters are respectively subjected to risk assessment, and a terminal foreign matter risk assessment result is output.

[0129] Through the foregoing detailed description of the foreign matter detection method for safe operation of the railway track, those skilled in the art can clearly understand the foreign matter detection system for safe operation of the railway track in the embodiment. For the system disclosed in Embodiment Two, since it corresponds to the method disclosed in Embodiment One, it has corresponding functional modules and beneficial effects, and the related parts can be referred to the method part description.

[0130] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for foreign object detection for safe operation of a railway track, characterized in that, The method comprises: Periodically acquiring data of the railway track by using a multi-source monitoring array arranged along the railway track to obtain a track image sequence and a track point cloud sequence; Extracting key images from the track image sequence to generate a track key image sequence and obtain a track key point cloud sequence; Fitting the track key point cloud sequence by using the track key image sequence, and constructing a foreign object simulation model distribution sequence according to an optimal point cloud fitting result sequence; Performing foreign object risk assessment based on the foreign object simulation model distribution sequence, and outputting an end foreign object simulation model distribution and an end foreign object risk assessment result as a current track foreign object detection result; Performing foreign object risk assessment based on the foreign object simulation model distribution sequence, and outputting an end foreign object simulation model distribution and an end foreign object risk assessment result, comprising: Setting the latest monitored foreign object simulation model distribution in the foreign object simulation model distribution sequence as the end foreign object simulation model distribution, and obtaining a plurality of end foreign object distribution characteristics, wherein the end foreign object has a foreign object structure characteristic and a foreign object position coordinate; Respectively performing mobile trend analysis on the plurality of end foreign object distribution characteristics based on the foreign object simulation model distribution sequence to determine a plurality of foreign object mobile trends; Respectively performing risk assessment on a plurality of end foreign objects according to a plurality of foreign object structure characteristics, a plurality of foreign object position coordinates and a plurality of foreign object mobile trends, and outputting an end foreign object risk assessment result.

2. The foreign object detection method for safe operation of a railway track according to claim 1, characterized by, Extracting key images from the track image sequence to generate a track key image sequence, comprising: Selecting a first track image monitored earliest in the track image sequence, setting the first track image as a first track key image, and obtaining a second track image adjacent to the first track image; Inputting the first track image and the second track image into an image analysis plug-in for similarity comparison, and outputting a first image similarity; If the first image similarity is less than a predetermined similarity threshold, setting the second track image as a second track key image; If the first image similarity is greater than or equal to the predetermined similarity threshold, discarding the second track image; Continuing iteration until the track image is traversed, obtaining a plurality of track key images, and obtaining the track key image sequence after sorting.

3. The method for foreign object detection for safe operation of railway tracks according to claim 2, characterized in that, The image analysis plug-in is constructed based on a twin network, and a plurality of sample track image groups and a plurality of image similarities are used to supervise training of the twin network until the network converges.

4. The method for foreign object detection for safe operation of railway tracks according to claim 1, characterized in that, Fitting the track key point cloud sequence by using the track key image sequence, comprising: Extracting contour features of a plurality of track key images in the track key image sequence by using an image contour recognition plug-in to obtain a plurality of image contour sets; Respectively performing similar contour elimination on the plurality of image contour sets based on track inherent facility contour features to obtain a plurality of foreign object contour sets; Respectively fitting the plurality of track key point clouds in the track key point cloud sequence based on the plurality of foreign object contour sets to output an optimal point cloud fitting result sequence.

5. The method for foreign object detection for safe operation of railway tracks according to claim 4, characterized in that, The image contour recognition plug-in is used for contour feature extraction on the plurality of track key images in the track key image sequence, comprising: According to the historical track foreign matter detection record, a sample track image set and a sample image contour set are collected, the sample track image set and the sample image contour set are taken as training data, and are equally divided into P parts to obtain P training sets, wherein P is an integer greater than 10; The P training sets are used to supervise the training and cross-validation of the convolutional neural network respectively, P image contour recognition branches are obtained, and an image contour recognition plug-in is constructed by combination; The similarity deviation of the track key image and the adjacent track key image is calculated respectively to obtain a plurality of similarity difference values; The ratio of the similarity difference value to the historical maximum similarity difference value is multiplied by P to obtain a plurality of adaptive branch calling quantities; Based on the plurality of adaptive branch calling quantities, the image contour recognition plug-in is used for contour feature extraction on the plurality of track key images in the track key image sequence.

6. The method for foreign object detection for safe operation of railway tracks according to claim 5, characterized in that, According to the plurality of foreign matter contour sets, point cloud mapping fitting is performed on the plurality of track key point clouds in the track key point cloud sequence, comprising: Randomly selecting a first foreign matter contour set and obtaining a first track key point cloud corresponding to the first foreign matter contour set; According to the first foreign matter contour set, the first track key point cloud is randomly fitted, and the first point cloud fitting result is output, and the ratio of the number of point clouds falling into the first foreign matter contour set to the total number of point clouds in the first track key point cloud is set as the first fitting accuracy; According to the first foreign matter contour set again, the first track key point cloud is randomly fitted, and the second point cloud fitting result is output, and the second fitting accuracy is calculated; Iterative fitting is performed until a predetermined fitting number is reached, and the point cloud fitting result corresponding to the maximum fitting accuracy is output as the first optimal point cloud fitting result, which is added to the plurality of optimal point cloud fitting results.

7. The method for foreign object detection for safe operation of railway tracks according to claim 6, characterized in that, According to the optimal point cloud fitting result sequence, a foreign matter simulation model distribution sequence is constructed, comprising: Based on the plurality of foreign matter contour sets, color feature extraction is performed on the foreign matter images in the plurality of track key images to obtain a plurality of foreign matter color distribution sets; According to the plurality of optimal point cloud fitting results, three-dimensional modeling is performed to obtain a plurality of foreign matter three-dimensional model sets; Using the plurality of foreign matter color distribution sets, color mapping rendering is performed on the plurality of foreign matter three-dimensional model sets to obtain a plurality of foreign matter simulation model distributions, and a foreign matter simulation model distribution sequence is constructed.

8. The method for foreign object detection for safe operation of railway tracks according to claim 7, characterized in that, The plurality of foreign matter color distribution sets are obtained, and further comprising: Collecting the current real-time light intensity, real-time weather conditions and current time period of the railway track; According to the real-time light intensity, real-time weather conditions and current time period, color correction is performed on the plurality of extracted foreign matter color distribution sets to obtain a plurality of foreign matter color distribution sets; The color correction according to the real-time light intensity, real-time weather conditions and current time period, comprising: In combination with the historical foreign matter detection image record, sample image extraction is performed according to the current time period, and high-frequency image clustering is performed to divide and determine a plurality of color correction regions, wherein the light intensity deviation of each color correction region is the same by default; Based on the convolutional neural network, a plurality of color correction branches are established according to the plurality of color correction regions, and a region-branch matcher is constructed in combination with the plurality of color correction regions; Using the region-branch matcher, the color correction branch of the adaptive region is selected based on the foreign matter position coordinates of each foreign matter color distribution, and the color correction is performed on a plurality of extracted foreign matter color distribution sets according to the real-time light intensity and real-time weather conditions, and a plurality of foreign matter color distribution sets are output.

9. A foreign object detection system for safe operation of a railway track, characterized in that, The system is used to execute the foreign matter detection method for safe operation of a railway track according to any one of claims 1-8, comprising: a track data acquisition module for periodically acquiring synchronous data of the railway track by using a multi-source monitoring array arranged along the railway track, and acquiring a track image sequence and a track point cloud sequence; a key data extraction module for extracting key images from the track image sequence to generate a track key image sequence, and mapping to acquire a track key point cloud sequence; a foreign matter simulation model construction module for fitting a point cloud based on the track key image sequence and the track key point cloud sequence, and constructing a foreign matter simulation model distribution sequence based on an optimal point cloud fitting result sequence; a foreign matter risk assessment module for performing foreign matter risk assessment based on the foreign matter simulation model distribution sequence, and outputting a terminal foreign matter simulation model distribution and a terminal foreign matter risk assessment result as a current track foreign matter detection result.

Images