Efficient detection and treatment of track damage based on deep learning

Abstract

The method for efficiently detecting and processing rail damage based on deep learning includes: image acquisition, preprocessing the acquired image data set to improve image quality, using image annotation tools to mark the positions of various types of damage in the track acquisition image data set, clustering the annotation frames through distance measurement, preliminary extraction of image features, feature map segmentation, feature reorganization, calculation of the degree of association between blocks, selection of associated areas, associated feature extraction, setting of multiple categories of rail damage types, establishment of multi-task data sets, establishment of multi-task models, determination of rail damage types, result evaluation, etc. The present invention uses deep learning to achieve efficient detection of rail damage, and improves detection precision and accuracy.

Description

Technical Field

[0001] The present invention relates to the field of rail transit detection technology, focusing on efficient detection and treatment of rail surface damage to ensure the safety and stability of rail infrastructure. Background Art

[0002] Minor track system fatigue disease can cause vibration and noise during train operation, affecting passengers' comfort experience, while the continuous deterioration of track disease can lead to serious abnormal vibration between wheels and rails. Damage or loss of fasteners can even cause the track to loosen or deviate, leading to major safety accidents such as train derailment. As the service life of the track continues to increase, the possibility of sudden accidents caused by the rapid development of track diseases is also increasing. The traditional regular maintenance method is difficult to predict sudden safety accidents caused by track diseases, and the regular replacement strategy cannot accumulate more track service degradation cases, and thus cannot effectively positively feedback to the design stage of the track infrastructure to achieve the goal of increasing its service life. Therefore, the traditional maintenance method can no longer meet the safety inspection and early warning needs of the track line infrastructure.

[0003] At present, the detection tools for rail damage often use GTC-80 large rail flaw detection vehicles and GCT-8C small hand-pushed flaw detectors to detect rail base materials. However, the large rail flaw detection vehicle is a high-speed rail flaw detection equipment that can simultaneously detect two rails in service; due to its high cost, it is mainly used in track detection such as national railways and subways. Its work efficiency is high, but the original system cannot be self-learning and optimized, and the detection effect of small-sized rail head nuclear damage is poor, which is prone to missed detection and false alarms; small rail flaw detectors are also called rail flaw detection trolleys, but the flaw detection trolleys need to be pushed forward by staff, and there will be missed reports while walking. Therefore, the data collected by flaw detection generally need to be replayed in a secondary playback mode to detect rail damage. The first time is for the line flaw detection staff to judge whether the section being detected is damaged on the spot based on the display of the A-type image wave; the second time is to hand over the stored B-type image data to the data center for playback analysis and re-check whether there is damage. It can be seen that there are problems of heavy workload and low efficiency. Therefore, large flaw detection vehicles are expensive, and small hand-pushed flaw detectors are inefficient, which is not conducive to the detection of track damage.

[0004] Prior art: CN115239632A discloses a rail surface damage detection method that integrates inspection images and ultrasonic images, which includes the following steps: S1, searching for rail inspection images and ultrasonic B-display images at the same mileage position, naming them according to the mileage number and saving them as jpg format images; S2, constructing a rail inspection image data set; S3, filtering out clutter in the ultrasonic B-display image; S4, constructing a rail surface segmentation algorithm to extract rail surface images; S5, building a CUFuse model; S6, dividing the rail inspection image data set into a training set and a test set at a ratio of 8:2, and inputting the data in the training set into the CUFuse model for training; S7, using the test set to test the CUFuse model; S8, encapsulating the trained CUFuse model as a plug-in and installing it in the RUD-15 double-track rail ultrasonic flaw detector. CN117036234A discloses a hybrid rail ultrasonic B-display damage identification method, system and storage medium, including classification learning of various damage maps of rail ultrasonic B-display images, using collected data sample sets to train recognition models, selecting the optimal model by comparing all parameters, and then combining the most appropriate data enhancement combination, and finally mounting it on the system and deploying it on a hardware device or storage medium to perform damage identification operations. After the system has identified the damage map, it displays all damage data identified in the ultrasonic video in the form of a list. CN112200225A discloses a rail damage B-display image recognition method based on a deep convolutional neural network, including searching and saving four types of wave conditions: welds, normal screw holes, abnormal screw holes, and surface damage; filling the original image and expanding the image size; cropping the filled data with the B-display damage image as the center; cropping the data symmetrically along the symmetry axis of the left and right rails into two parts horizontally, and cropping them into four parts along the longitudinal direction, and finally obtaining a 416×416 standard image; labeling the data, selecting the detection object to be trained, and saving it as a label file , make the obtained label files and image files into a standard data set; build the YOLO-UAV+min model; set the number of anchor boxes that need to be clustered, perform K-means clustering analysis on the data set used, obtain the prior box, and input the prior box into the YOLO-UAV+min model; configure network parameters; use the YOLO-UAV+min model for training; package the trained YOLO-UAV+min model as a plug-in and install it in the rail ultrasonic flaw detector for intelligent recognition of rail damage B-display images in daily rail flaw detection operations. Although this method improves the detection accuracy and detection speed in the target detection method, and verifies the real-time effectiveness of target rail damage detection, this method has the problem of difficulty in extracting small-size features when detecting target rail flaws.CN111402209A discloses a high-speed railway rail damage detection method based on U-Net, comprising the following steps: building a real scene for high-speed railway rail damage detection, obtaining a detection image of rail damage; marking the rail damage area in each collected original image and saving the annotation; expanding the training sample through data enhancement operations of translation, rotation and scaling; setting the parameters of the rail damage detection network model, inputting an image and its corresponding segmentation map, training the detection network through a deep learning algorithm platform, and modifying the detection network model parameters according to the evaluation index of the training network until the evaluation index is obtained; marking the original input image with a rectangular frame based on the segmentation map calculated by the U-Net network, and if the collected image contains defects, prompting the relevant processing results in the computer, and marking the defective workpiece into the warehouse.

[0005] The above-mentioned existing technologies are mainly divided into the following categories, namely, defect detection technology based on machine vision, defect detection technology based on laser scanning, and defect detection technology based on vibration response signals. However, the focus of the above-mentioned methods is often on the extraction of single track defect features based on data processing, and they do not conduct a comprehensive diagnosis of common track diseases, and their diagnostic results are not satisfactory. At the same time, the existing detection technology has not achieved accurate and rapid identification effects, which is a very severe test for the current situation of the rapid development of urban rail transit in my country. Therefore, it is still urgent to pay attention to the research on online automatic detection technology of track defects. Summary of the invention

[0006] In order to solve the defects in the prior art, the present invention discloses an efficient detection and processing method for track damage based on deep learning, and its technical solution is as follows:

[0007] A deep learning-based efficient rail damage detection and processing method comprises the following steps:

[0008] Step S1: Image acquisition: An image acquisition module suitable for the complex multi-curved rail surface is used. A binocular vision camera group and a special-section LED combination light source are set on the gimbal of the drone. The binocular vision camera group includes two cameras for capturing images of the same scene from different angles to achieve depth perception. The special-section LED combination light source adopts an LED light source, and its cross-sectional shape only needs to ensure that it can optimize the light distribution to provide uniform illumination. The specific structure of the special-section is not limited, as long as it meets the above requirements, and the LED shape angle can be dynamically controlled according to the track site environment to ensure the synchronization of image acquisition and uniform illumination, so as to ensure that the collected track surface image is complete and high-quality;

[0009] Step S2: preprocessing the collected image data set to improve the image quality: using the UAV to collect periodic orbit state data according to the mission monitoring orbit interval, and after preprocessing, forming a preprocessed historical orbit monitoring data set and storing it in the database by category; collecting the current image of the monitoring orbit interval to form a real-time monitoring data set; and after preprocessing, forming a real-time monitoring data set and storing it in the database by category;

[0010] Step S3: using an image annotation tool to annotate the positions of various types of damage in the track acquisition image data set, and clustering the annotation boxes by distance measurement;

[0011] Step S4: Preliminary extraction of image features: Use deep convolution with a convolution kernel of 3*3 to extract image features, and obtain a preliminary feature map IMG with a dimension of [H, W, C], where H is the height of the feature map, W is the width of the feature map, and C is the number of channels of the feature map;

[0012] Step S5: Feature map segmentation: The feature map is segmented into M*M feature blocks, each block has a size of [H / M, W / M, C], and a total of M 2 blocks, which is convenient for subsequent reorganization;

[0013] Step S6: Feature reorganization: Arrange all feature blocks into a row, and the reorganized dimension is [M 2 ,HW / M 2 ,C], which is used as the input for the subsequent calculation of the correlation between blocks. The reorganized feature map is recorded as X=[x1,x2,....x M 2 ]; where x1 is the feature of the first block, x M 2 It is the Mth 2 Characteristics of the block;

[0014] Step S7: Calculate the degree of association between blocks:

[0015] R=XW q (W k ) T X T

[0016] Among them, R is the degree of correlation between blocks, and the dimension is [M 2 ,C], X is the reorganized feature map in step S6, the superscript T is the transposition operation, W q and W k They are parameter matrices, respectively. The superscripts q and k represent the query and key variables in the attention mechanism used to calculate the degree of association. The parameter matrices are obtained by model training.

[0017] Step S8: Selecting related regions: For each block, according to the correlation degree between R blocks, only the t blocks that are most related to it are selected and considered to be related. The inter-block correlation matrix is ​​recorded as RI, and the dimension is [M 2 ,t], where RI i,j， represents the jth associated block of the ith block;

[0018] Step S9: partial attention-based correlation feature extraction: For each block, associate it with the most relevant t blocks to reduce unnecessary calculations. The formula is as follows:

[0019] Q=XW q

[0020] K=FLITER(X,W k ,RI)

[0021] V=FLITER(X,W v ,RI)

[0022] F=ɡ(QK T )V

[0023] Among them, Q, K, and V are the query, key, and value variables required for partial attention calculation respectively; W v is a parameter matrix whose parameters are obtained by model training; the FLITER function filters out other blocks that are not related to each block, and only retains t related blocks for each block. The inter-block correlation matrix is ​​recorded as RI; ɡ is the softmax function, and F is the extracted image feature, which gives different importance to each pixel of the image, ensuring the adequacy of feature extraction;

[0024] Step S10: Setting multiple categories of track damage types: The track damage types are divided into N categories, namely [Class1, Class2, ..., Class N ];

[0025] Step S11: Establish a multi-task data set: The overall data set includes N types of track damage images, denoted as DATA=[D1,D2,...,D N ]; establish a multi-task dataset MDATA, a total of N; for the nth multi-task dataset MDATAn, the positive sample is [Dn], and the negative sample is all other non-n type track damage images [D1,D2,...Dn-1,Dn+1,...,D N ], all the images in the multi-task datasets are the same, but the positive and negative sample categories are different;

[0026] Step S12: Establish a multi-task model: Use a multi-layer perceptron to establish N classification models, each of which is trained for only one category of classification tasks, that is, the data set of classification model n is MDATAn, and only the nth category of track damage is judged, and the judgment results are also only two results: whether it belongs to the nth category of track damage; after inputting the image into N models, N classification results will be obtained, and each classification result is a decimal of 0-1;

[0027] Step S13: Determine the type of track damage: Set a threshold for each classification model. The threshold of each classification model is different and is adjusted according to the model learning results and manual experience. If the classification result is greater than the threshold, it means that there is such type of track damage. Each image can have multiple types of damage. When damage occurs, the judgment result of the damage type should be notified to the relevant person in charge to ensure the safety of track operation.

[0028] The present invention also discloses a deep learning-based efficient track damage detection and processing system, which is based on the above-mentioned deep learning-based efficient track damage detection and processing method, and is characterized in that: the drone converts the pixel coordinates in the collected rail image to the coordinates in the camera coordinate system, and then converts the points in the camera coordinate system to the coordinates in the world physical coordinate system; the drone flies according to a pre-set flight route and collects image data of the inspection area. On the one hand, the data is transmitted to the Internet through the 5G base station and connected to the ground station monitoring center through the firewall. On the other hand, in order to ensure the reliability of the collected information, the collected image data is stored in the drone storage card without processing in case of loss; the drone system and the drone operation command system perform two-way communication according to the data interface, and transmit the collected video images back to the rail inspection damage database in real time.

[0029] Preferably: track damage includes partial collapse of rail surface, low rail joints, uneven rail welding, rail surface corrugation, fastener damage, rail head core damage, thread cracking, screw hole cracking, and rail bottom transverse cracking.

[0030] The present invention also discloses a non-volatile storage medium, characterized in that the non-volatile storage medium includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the above method.

[0031] The present invention also discloses a terminal device, characterized in that the terminal device includes: a processor, a memory, a communication interface and a bus; the processor, the memory and the communication interface are connected through the bus and complete mutual communication; the memory stores executable program code; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the above method.

[0032] The present invention also discloses an unmanned aerial vehicle, which is characterized by comprising the above-mentioned terminal device, and the unmanned aerial vehicle is used for collecting data on inspecting track damage. Beneficial Effects

[0033] 1. Abandon the use of large rail flaw detection vehicles and small hand-pushed flaw detectors to detect rail flaws, so as to improve the monitoring efficiency and accuracy of rail defects;

[0034] 2. Use multiple power supply methods for drones to ensure that the drone maintains long-term working intensity when executing commands (using wire charging on the ground to increase anti-interference; using solar panels and wireless power supply in the air to achieve energy storage);

[0035] 3. Deep learning is used to achieve efficient detection of track damage, improving detection precision and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a schematic diagram of the structure of the present invention. DETAILED DESCRIPTION

[0037] The present invention takes into account the operating conditions in which there are various typical rail fatigue damage scenarios in the rail system, such as local collapse of the rail surface, low rail joints, uneven rail welding, rail surface corrugation and fastener damage, and studies the intelligent identification and detection technology of rail diseases. It proposes a deep learning-based efficient rail damage detection and processing method, which can be applied to real-time detection and diagnosis of rail line diseases in the future.

[0038] See also Figure 1 The track damage efficient detection and processing method based on deep learning includes the following steps:

[0039] Step S1: Image acquisition: An image acquisition module suitable for complex multi-curved rail surfaces is used. A binocular vision camera group and a special-section LED combination light source are set on the gimbal of the drone. The binocular vision camera group includes two cameras for capturing images of the same scene from different angles to achieve depth perception. The special-section LED combination light source adopts an LED light source, whose cross-sectional shape can ensure optimized light distribution to provide uniform lighting, and the LED shape angle can be dynamically controlled according to the track site environment to ensure synchronization of image acquisition and uniform lighting, so as to ensure that the collected track surface image is complete and of high quality.

[0040] Data collection is completed through the mission payload carried by the UAV data acquisition system to obtain the rail surface image data set.

[0041] The data acquisition system includes an unmanned aerial vehicle system and its mission payload. The unmanned aerial vehicle system is mounted on a gimbal. The mission payload includes image acquisition equipment, multi-spectral equipment, and radar equipment. The payload is used to collect data on railway rail damage.

[0042] The images taken by the drone inspection will record the longitude and latitude coordinates and the gimbal's posture information at the time of collection. The drone flight control system uses the geodetic coordinate system for positioning, but when the drone performs mission planning, a rectangular coordinate system is required for route planning and parameter conversion. The images taken by the drone are represented by a pixel coordinate system, and it is necessary to transform the image coordinate system to the world physical coordinate system. Therefore, the pixel coordinates in the collected rail images are converted to coordinates in the camera coordinate system, and then converted from points in the camera coordinate system to coordinates in the world physical coordinate system. The drone flies according to a pre-set flight route and collects image data in the inspection area. On the one hand, it transmits the data to the Internet through a 5G base station.

[0043] It is connected to the ground station monitoring center through a firewall. On the other hand, in order to ensure the reliability of the collected information, the collected image data is stored in the drone storage card without processing to prevent loss. The drone system and the drone operation command system communicate two-way according to the data interface, and the collected video images are transmitted back to the rail inspection damage database in real time.

[0044] The image acquisition device includes a binocular vision camera group and a special-section LED combined light source to ensure synchronization and uniform illumination of image acquisition. Among them, the binocular vision camera group uses two or more cameras to capture images from different angles to obtain stereoscopic vision information.

[0045] Step S2: preprocessing the collected image data set to improve the image quality: using the UAV to collect periodic orbit status data according to the mission monitoring orbit interval, and after preprocessing, forming a preprocessed historical orbit monitoring data set and storing it in the database by category; collecting the current image of the monitoring orbit interval to form a real-time monitoring data set; and after preprocessing, forming a real-time monitoring data set and storing it in the database by category.

[0046] The preprocessing includes filtering and denoising, image sample data enhancement, outlier processing, missing value processing, and standardization processing, wherein:

[0047] Filter denoising: Gaussian filtering can eliminate noise in the image, reduce the blur of the image, better preserve edge features, and judge the denoising effect by signal-to-noise ratio and peak signal-to-noise ratio;

[0048] Among them, the Gaussian filter expression is: , where (x, y) is the point coordinates, is the standard deviation;

[0049] The signal-to-noise ratio expression is: ;

[0050] is the signal power; is the noise power; The larger the value, the clearer the image after filtering;

[0051] The peak signal-to-noise ratio expression is:

[0052] Where: n is the number of bits per pixel; is the mean square error; The larger the value, the clearer the image after filtering;

[0053] Image sample data enhancement, including image translation, image rotation, and image mirroring; wherein, the image translation is to move the image along the X direction or the Y direction, and the part that is moved out of the original image range is usually filled with 0, that is, replaced with a pure black background, so as to achieve the purpose of data enhancement, and the amplitude of the movement is set within 10% of the image size;

[0054] Image rotation takes the center of the image as the origin, and rotates the image at a certain angle to obtain images in different spatial directions;

[0055] Image mirroring includes horizontal mirroring and vertical mirroring. Horizontal mirroring uses the vertical central axis of the image as the mirroring center axis to mirror the left and right parts of the image; vertical mirroring uses the horizontal central axis of the image as the mirroring center axis to mirror the upper and lower parts of the image.

[0056] Standardization: First find the minimum value A in each attribute min and the maximum value A max , and then perform a linear transformation on the original data to map the result to [0, 1]; let the original time series be {A1, A2, … , An}, then the standardized time series { B1, B2, … , Bn}∈[0, 1], the formula is as follows: ; The new value after processing is between 0 and 1, which can convert both positive and negative attributes into positive attributes, making their directions consistent. When input into the intelligent fault diagnosis model after processing, it can also effectively avoid the forward gradient explosion problem of the neural network and improve the convergence speed of the diagnosis model.

[0057] Outlier handling

[0058] Since outliers are more likely to appear as individual cases, discarding data may result in the loss of some information. When outliers appear as individual cases, the post-filling method can be used to replace the abnormal data. When outliers do not appear as individual cases, they are discarded. In order to ensure the reliability of the data subsequently input into the intelligent fault diagnosis model, similar outliers need to be discarded or filled in for the data collected by all devices.

[0059] Missing Value Handling

[0060] During the data collection process, failures in data transmission media, failures in data collection equipment, and damage to storage media can cause some data loss. The introduction of missing values ​​in equipment data will interfere with the diagnostic performance of the equipment intelligent diagnosis model. Common methods for dealing with missing values ​​include filling and deletion. If a large piece of data is missing, the new values ​​added by the filling method may not conform to the actual situation and will also introduce new noise. Therefore, similar to dealing with outliers, when missing values ​​appear as individual cases, the post-filling method can be used to replace the missing data. When missing values ​​do not appear as individual cases, they are discarded.

[0061] Step S3: Use image annotation tools to annotate the locations of various types of damage in the track acquisition image dataset, and cluster the annotation boxes through distance measurement.

[0062] The specific process of image annotation is as follows: save the images collected on site and the image data generated by simulation in the same format; use annotation tools to annotate the rail defect parts in the data; divide the images into defect samples and normal samples according to whether there are defects or not and store them separately; when the annotation is completed, review the annotation results through data revisit detection, and modify the annotations that do not meet the requirements; convert the format of the completed annotated samples to form a common data set format; use the annotated data set for division, and construct mixed sets and real sets for subsequent model training.

[0063] In addition, after the annotation is completed through the image annotation tool, the annotation information can be saved as a file in a certain format including the category, location and other information of each track damage target. Each image corresponds to an XML file, which is then converted into a TXT file for model training. According to the above information, the coordinates of the contour points are read and filled with colors to realize the visualization of the annotation information and form a labeled image. The annotation data can be used for the training of deep learning models to help the model learn to identify and classify different types of track damage and improve the accuracy and robustness of track detection.

[0064] Track damage includes local collapse of the rail surface, low rail joints, uneven rail welding, rail surface corrugation, fastener damage, rail head core damage, thread cracking, screw hole cracking, and transverse cracks on the rail bottom.

[0065] Step S4: Preliminary extraction of image features: Use deep convolution with a convolution kernel of 3*3 to extract image features, and obtain a preliminary feature map IMG with a dimension of [H, W, C], where H is the height of the feature map, W is the width of the feature map, and C is the number of channels of the feature map;

[0066] Step S5: Feature map segmentation: The feature map is segmented into M*M feature blocks, each block has a size of [H / M, W / M, C], and a total of M 2 blocks, which is convenient for subsequent reorganization;

[0067] Step S6: Feature reorganization: Arrange all feature blocks into a row, and the reorganized dimension is [M 2 ,HW / M 2 ,C], which is used as the input for the subsequent calculation of the correlation between blocks. The reorganized feature map is recorded as X=[x1,x2,....x M 2 ]; where x1 is the feature of the first block, x M 2 It is the Mth 2 Characteristics of the block;

[0068] Step S7: Calculate the degree of association between blocks:

[0069] R=XW q (W k ) T X T

[0070] Among them, R is the degree of correlation between blocks, and the dimension is [M 2 ,C], X is the reorganized feature map in step S6, the superscript T is the transposition operation, W q and W k They are parameter matrices, respectively. The superscripts q and k represent the query and key variables in the attention mechanism used to calculate the degree of association. The parameter matrices are obtained by model training.

[0071] Step S8: Selecting related regions: For each block, according to the correlation degree between R blocks, only the t blocks that are most related to it are selected and considered to be related. The inter-block correlation matrix is ​​recorded as RI, and the dimension is [M 2 ,t], where RI i,j， represents the jth associated block of the ith block;

[0072] Step S9: partial attention-based correlation feature extraction: For each block, associate it with the most relevant t blocks to reduce unnecessary calculations. The formula is as follows:

[0073] Q=XW q

[0074] K=FLITER(X,Wk ,RI)

[0075] V=FLITER(X,W v ,RI)

[0076] F=ɡ(QK T )V

[0077] Among them, Q, K, and V are the query, key, and value variables required for partial attention calculation respectively; W v is a parameter matrix whose parameters are obtained by model training; the FLITER function filters out other blocks that are not related to each block, and only retains t related blocks for each block. The inter-block correlation matrix is ​​recorded as RI; ɡ is the softmax function, and F is the extracted image feature, which gives different importance to each pixel of the image, ensuring the adequacy of feature extraction;

[0078] Step S10: Setting multiple categories of track damage types: The track damage types are divided into N categories, namely [C1, C2, ..., C N ]; such as partial collapse of rail surface, low rail joint, uneven rail welding, rail surface corrugation, fastener damage, rail head core damage, thread cracking, screw hole cracking, rail bottom transverse cracking and other damage types included in the existing rail damage database.

[0079] Step S11: Establish a multi-task data set: The overall data set includes N types of track damage images, denoted as DATA=[D1,D2,...,D N ]; establish a multi-task dataset MDATA, a total of N; for the nth multi-task dataset MDATAn, the positive sample is [Dn], and the negative sample is all other non-n type track damage images [D1,D2,...Dn-1,Dn+1,...,D N ], all the images in the multi-task datasets are the same, but the positive and negative sample categories are different;

[0080] Step S12: Establish a multi-task model: Use a multi-layer perceptron to establish N classification models, each of which is trained for only one category of classification tasks, that is, the data set of classification model n is MDATAn, and only the nth category of track damage is judged, and the judgment results are also only two results: whether it belongs to the nth category of track damage; after inputting the image into N models, N classification results will be obtained, and each classification result is a decimal of 0-1;

[0081] Step S13: Determine the type of track damage: set a threshold for each classification model. The threshold of each classification model is different and is adjusted according to the model learning results and manual experience. If the classification result is greater than the threshold, it means that the type of track damage exists.

[0082] Step S14: Result evaluation: Use evaluation indicators: accuracy, recall and ROC curve to evaluate the detection results. These indicators can quantify the performance of the model and ensure the effectiveness of the detection method.

[0083] Step S15: When damage occurs, the determination result of the damage type should be notified to the relevant person in charge to ensure the safety of track operation. Example 2

[0084] A drone, including the above-mentioned terminal device, is used for data collection for inspecting track damage. The drone adopts a wireless + photovoltaic charging combination to increase endurance and provides a tool for detecting railway track damage. It includes a wireless charging module, which includes an energy transmitting module, an energy receiving module and a coupling module. The energy transmitting module includes an industrial frequency voltage, a high-frequency DC / AC converter, a transmitting part and its compensation network; the energy receiving module includes a receiving part and its compensation network, and an AC / DC inverter.

[0085] Working principle: The energy transmitting module converts the industrial frequency AC voltage signal into high-frequency AC power after passing through the full-bridge rectifier filter module and then through the high-frequency DC / AC converter. After that, it passes through the compensation network and is input into the transmitting coil in the coupling module to form an alternating magnetic field. When its resonant frequency is the same as the self-resonant frequency of the transmitting coil, its transmission efficiency is maximum and the line impedance is minimum. The electric energy is efficiently transmitted to the receiving coil, and then passes through the receiving part, compensation network, AC / DC inverter and LC filter of the energy receiving module to generate a DC signal, which is then passed through the power management module to prepare for charging the drone battery. Since magnetic leakage and detuning problems may occur during wireless charging of the drone, an energy replenishment network is added to the output end of the energy transmitting module and the input end of the energy receiving module; the energy replenishment structure is as follows: it includes a primary side and a secondary side, and the primary side equivalent circuit includes: the high-frequency AC signal output by the high-frequency DC / AC inverter is connected in series with the circuit internal resistance RS, the primary coil compensation inductance La, the transmitting end series compensation capacitor Ca, the primary coil inductance L1, the transmitting coil inductance resistance R1, and the transmitting coil compensation inductance resistance R2; in addition, the parallel compensation capacitor C1 of the transmitting end is connected to the output end of the primary coil compensation inductance La and the output end of the transmitting coil inductance resistance R1; the secondary side equivalent circuit includes: the secondary coil inductance L2, the receiving end series compensation capacitor Cb, the secondary coil compensation inductance Lb, the receiving end load resistance, the receiving coil compensation inductance resistance Rb, and the receiving coil inductance resistance R2 connected in series in sequence; in addition, the receiving end parallel compensation capacitor C2 is connected to the receiving end at the input end of the secondary coil compensation inductance Lb and the output end of the receiving coil compensation inductance resistance Rb. Compared with no energy supplement network, the addition of the energy supplement network structure has better output current characteristics when the load and coupling change, because the output current of the energy supplement network is independent of the load and is proportional to the mutual inductance.

[0086] In view of the requirement of wireless power transmission rate, the current transmitted on the coupling coil is relatively large, and the frequency used on the coil is relatively high, so the wire diameter can be determined according to the current density and skin depth. Since it works in the range of tens to hundreds of kHz, its transmission efficiency is proportional to the frequency. However, as the frequency increases, the skin effect of the system also increases, which will reduce the effective transmission area of ​​the coil and increase the impedance, causing the loss of the coil to increase, and causing the wire to heat up, affecting the characteristics of the coil, thereby affecting the transmission efficiency of the system. Therefore, when selecting the wire, it is necessary to reduce the impact of the skin effect on the wire.

[0087] The drone has a coil offset problem during charging, so a double-coil design is used at the transmitter end. In order to avoid magnetic field leakage during power transmission, which affects the power transmission efficiency, a ferrite shielding material is added to the coil to gather the magnetic flux generated by the coil, and also plays a good role in suppressing magnetic flux leakage.

[0088] In order to ensure that the UAV can operate in the air for a long time and increase its endurance, the present invention also arranges photovoltaic cells on the UAV and lays a solar cell array on the wings of the UAV. It also includes an energy manager and a battery to form a part of the energy system. The energy manager manages the charging and discharging of the battery, including whether to charge the battery wirelessly or to supplement it with solar energy. At the same time, the energy manager is also responsible for reasonably allocating the battery energy to the UAV's own power load needs and the task load set on the gimbal, including various data acquisition sensors such as cameras.

[0089] The energy system of the UAV adopts a distributed power supply structure. According to the regional division of photovoltaic cells and batteries in the UAV, the photovoltaic cells and batteries are divided into multiple independent sub-units. Multiple DC / DC converters with parallel outputs are used to control each unit nearby to avoid long-distance conduction losses when a centralized structure is adopted. At the same time, each converter can be redundant to improve the reliability of power supply. Multiple photovoltaic MPPT modules are used to establish a power control strategy for the distributed photovoltaic cells on the solar UAV. The operating status of the photovoltaic cells and batteries is judged to optimize the power distribution and stabilize the operation of the energy system; combined with the fuzzy control strategy, the power transmission quota of the converter is adjusted to improve the operating efficiency of the energy system.

[0090] The present invention proposes an efficient detection and processing method for rail damage based on deep learning. This method realizes all-round and accurate detection of rail surface damage by integrating two-dimensional image acquisition. This invention designs an image acquisition system suitable for complex multi-curved rail surfaces, which consists of a binocular vision camera group and a special-section LED combination light source to ensure the synchronization of image acquisition and uniform illumination. At the same time, a comprehensive evaluation system for the image acquisition system is established to ensure that the collected rail surface images are complete and of high quality. Weights are assigned to different features through the attention module to highlight the features that are more important for defect classification. This strategy not only improves the accuracy of defect classification, but also enhances the generalization ability of the system.

[0091] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. An efficient rail damage detection and processing method based on deep learning, characterized by: Step S1: Image acquisition: An image acquisition module suitable for the complex multi-curved rail surface is provided. A binocular vision camera group and a special-section LED combination light source are set on the gimbal of the UAV, wherein the binocular vision camera group includes two cameras for capturing images of the same scene from different angles to achieve depth perception; the special-section LED combination light source adopts an LED light source, whose cross-sectional shape can ensure optimized light distribution to provide uniform illumination, and the LED shape angle can be dynamically controlled according to the track site environment to ensure synchronization of image acquisition and uniform illumination; Step S2: preprocessing the collected image data set: using the drone to collect periodic orbit status data according to the mission monitoring orbit interval, and after preprocessing, forming a preprocessed historical orbit monitoring data set and storing it in a database by category; The current image of the monitored track section is collected to form a real-time monitoring data set; and after preprocessing, the real-time monitoring data set is formed and classified and stored in the database; Step S3: using an image annotation tool to annotate the positions of various types of damage in the track acquisition image data set, and clustering the annotation boxes by distance measurement; Step S4: Preliminary extraction of image features: Use deep convolution with a convolution kernel of 3*3 to extract image features, and obtain a preliminary feature map IMG with a dimension of [H, W, C], where H is the height of the feature map, W is the width of the feature map, and C is the number of channels of the feature map; Step S5: Feature map segmentation: The feature map is segmented into M*M feature blocks, each block has a size of [H / M, W / M, C], and a total of M 2 blocks, which is convenient for subsequent reorganization; Step S6: Feature reorganization: Arrange all feature blocks into a row, and the reorganized dimension is [M 2 ,HW / M 2 ,C], which is used as the input for the subsequent calculation of the correlation between blocks. The reorganized feature map is recorded as X=[x1,x2,....x M 2 ]; where x1 is the feature of the first block, x M 2 It is the Mth 2 Characteristics of the block; Step S7: Calculate the degree of association between blocks: R=XW q (W k ) T X T Among them, R is the degree of correlation between blocks, and the dimension is [M 2 ,C], X is the reorganized feature map in step S6, the superscript T is the transposition operation, W q and W k They are parameter matrices, respectively. The superscripts q and k represent the query and key variables in the attention mechanism used to calculate the degree of association. The parameter matrices are obtained by model training. Step S8: Selecting related regions: For each block, according to the correlation degree between R blocks, only the t blocks that are most related to it are selected and considered to be related. The inter-block correlation matrix is ​​recorded as RI, and the dimension is [M 2 ,t], where RI i,j， represents the jth associated block of the i-th block; Step S9: partial attention-based correlation feature extraction: For each block, associate it with the most relevant t blocks to reduce unnecessary calculations. The formula is as follows: Q=XW q K=FLITER(X,W k ,RI) V=FLITER(X,W v ,RI) F=ɡ(QK T )V Among them, Q, K, and V are the query, key, and value variables required for partial attention calculation respectively; W v is a parameter matrix whose parameters are obtained by model training; the FLITER function filters out other blocks that are not related to each block, and only retains t related blocks for each block. The inter-block correlation matrix is ​​recorded as RI; ɡ is the softmax function, and F is the extracted image feature; Step S10: Setting multiple categories of track damage types: The track damage types are divided into N categories, namely [Class1, Class2, ..., Class N ]; Step S11: Establish a multi-task data set: The overall data set includes N types of track damage images, denoted as DATA=[D1,D2,...,D N ]; establish a multi-task dataset MDATA, a total of N; for the nth multi-task dataset MDATAn, the positive sample is [Dn], and the negative sample is all other non-n type track damage images [D1,D2,...Dn-1,Dn+1,...,D N ], all the images in the multi-task dataset are the same, but the positive and negative sample categories are different; Step S12: Establish a multi-task model: Use a multi-layer perceptron to establish N classification models, each of which is trained for only one category of classification tasks, that is, the data set of classification model n is MDATAn, and only the nth category of track damage is judged, and the judgment results are also only two results: whether it belongs to the nth category of track damage; after inputting the image into N models, N classification results will be obtained, and each classification result is a decimal of 0-1; Step S13: Determine the type of track damage: set a threshold for each classification model. The threshold of each classification model is different and is adjusted according to the model learning results and manual experience. If the classification result is greater than the threshold, it means that the type of track damage exists. Step S14: Result evaluation: Use evaluation indicators: accuracy, recall and ROC curve to evaluate the detection results; Step S15: When damage occurs, the determination result of the damage type is notified to the relevant person in charge to ensure the safety of track operation.

2. The method for efficiently detecting and processing rail damage based on deep learning according to claim 1 is characterized by: The preprocessing includes filtering and denoising, image sample data enhancement, outlier processing, missing value processing, and standardization processing, wherein: Filter denoising: Gaussian filtering can eliminate noise in the image, reduce the blur of the image, better preserve edge features, and judge the denoising effect by signal-to-noise ratio and peak signal-to-noise ratio; Among them, the Gaussian filter expression is: , where (x, y) is the point coordinates, is the standard deviation; The signal-to-noise ratio expression is: ; is the signal power; is the noise power; The larger the value, the clearer the image after filtering; The peak signal-to-noise ratio expression is: ; Where: n is the number of bits per pixel; is the mean square error; The larger the value, the clearer the image after filtering; Image sample data enhancement, including image translation, image rotation, and image mirroring; wherein, the image translation is to move the image along the X direction or the Y direction, and the part that is out of the original image range is replaced with a pure black background, so as to achieve the purpose of data enhancement, and the amplitude of the movement is set within 10% of the image size; Image rotation takes the center of the image as the origin, and rotates the image at a certain angle to obtain images in different spatial directions; Image mirroring includes horizontal mirroring and vertical mirroring. Horizontal mirroring uses the vertical central axis of the image as the mirroring center axis to mirror the left and right parts of the image; vertical mirroring uses the horizontal central axis of the image as the mirroring center axis to mirror the upper and lower parts of the image. Outlier processing: When outliers appear in individual cases, the post-fill method is used to replace the abnormal data; when outliers do not appear in individual cases, they are discarded; Missing value processing: Similar to the outlier processing, when missing values ​​appear in individual cases, the missing data is replaced by post-filling method; when missing values ​​do not appear in individual cases, they are discarded; Standardization: First find the minimum value A in each attribute min and the maximum value A max , and then perform a linear transformation on the original data to map the result to [0, 1]; let the original time series be {A1, A2, … , An}, then the standardized time series { B1, B2, … , Bn}∈[0, 1], the formula is as follows: ; The new value after processing is between 0 and 1, converting both positive and negative attributes into positive attributes so that their directions of action are consistent.

3. The method for efficiently detecting and processing rail damage based on deep learning according to claim 1 is characterized by: The specific process of image annotation is as follows: save the image data collected on site and the image data generated by simulation in the same format; annotate the damaged part of the track in the data using an annotation tool; divide the images into diseased samples and normal samples according to whether there is track damage or not and store them separately; after the annotation is completed, review the annotation results through data return detection, and modify the annotations that do not meet the requirements; convert the format of the completed annotated samples to form a common data set format; divide the annotated data set to construct a mixed set and a real set for subsequent model training; After the annotation is completed by the image annotation tool, the annotation information is saved in a certain format including the category and location information of each track damage target. Each image corresponds to an XML file, which is then converted into a TXT file for training. The coordinates of the contour points are read and filled with colors based on the above information to achieve a visual display of the annotation information and form a labeled image. The labeled data is used for deep learning training to help the model learn to identify and classify different types of track damage and improve the accuracy and robustness of track detection.

4. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored program, wherein when the program is executed, the device where the non-volatile storage medium is located is controlled to execute the method according to any one of claims 1 to 3.

5. A terminal device, characterized in that: The terminal device includes: a processor, a memory, a communication interface and a bus; the processor, the memory and the communication interface are connected through the bus and communicate with each other; the memory stores executable program code; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the method described in claim 1 above.

6. A drone, characterized by: Including the terminal device as described in claim 5, the drone is used for collecting data on inspecting track damage.

Images