Track defect detection method, device and electronic equipment based on deep learning
The deep learning-based track defect detection method and device address inefficiencies in manual inspection by automating defect recognition and positioning, improving accuracy and reducing labor costs in railway track maintenance.
Patent Information
- Application Number
- CN202411436921.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-15
- Publication Date
- 2024-12-27
- Estimated Expiration
- 2044-10-15
AI Technical Summary
Current manual inspection methods for railway track defects are inefficient, subjective, and labor-intensive, failing to ensure the safety and efficiency of train operations due to the limitations of traditional defect detection techniques.
A track defect detection method and device based on deep learning, utilizing a TGL Net model with an image labeling segmentation module, image feature interaction module, feature fusion pyramid network, and classification regression module to automatically segment, classify, and regress track surface defects, enhancing accuracy and efficiency.
The method significantly improves the detection accuracy and efficiency of railway track defects, reducing labor costs while ensuring precise defect recognition and positioning, thereby enhancing train safety and operational efficiency.
Smart Images
Figure CN119007174B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and more specifically, to a track defect detection method, device and electronic equipment based on deep learning. Background Art
[0002] In recent years, railways have become one of the important transportation carriers. As a key infrastructure of railways, it is crucial to ensure that the tracks are in good condition for safe railway operation. With the expansion of railway transportation, vehicles are subjected to frequent force and weight changes during operation, wheels wear the tracks, and track surface defects gradually emerge, including peeling, collapse, wear, block drop and other defects, which pose a major threat to train safety and bring major safety hazards to railway transportation.
[0003] Therefore, track defect detection is the key to ensure the safe operation of trains. Currently, manual inspection is a common defect detection method. Inspection workers mainly use visual inspection or basic measuring equipment to check the track condition on the railway track. The manual method is simple, but has limitations, is subjective and inefficient.
[0004] With respect to the above-mentioned problems existing in the related technologies, no effective solutions have been found so far. Summary of the invention
[0005] The present application provides a track defect detection method, device and electronic equipment based on deep learning to solve the above-mentioned technical problems existing in the related technology.
[0006] According to an embodiment of the present application, a track defect detection method based on deep learning is provided, comprising: obtaining a global track image to be detected; inputting the global track image into a TGL Net model for track surface defect detection; the TGL Net model comprises an image labeling segmentation module, an image feature interaction module, a feature fusion pyramid network and a classification regression module; the image labeling segmentation module segments the global track image into a plurality of non-overlapping local images; extracting feature information of the local images through the image feature interaction module; performing feature fusion on the feature information through the feature fusion pyramid network; the classification regression module classifies and regresses the feature maps output by the feature fusion pyramid network respectively to obtain defect information of the local images; and maps the defect information in the local images to the global track image.
[0007] According to another embodiment of the present application, a track defect detection device based on deep learning is provided, including: an acquisition module, used to acquire a global track image to be detected; a detection module, used to input the global track image into a TGL Net model for track surface defect detection; the TGL Net model includes an image labeling and segmentation module, an image feature interaction module, a feature fusion pyramid network and a classification and regression module; the image labeling and segmentation module segments the global track image into multiple non-overlapping local images; the feature information of the local images is extracted through the image feature interaction module; the feature information is feature fused through the feature fusion pyramid network; the classification and regression module classifies and regresses the feature maps output by the feature fusion pyramid network respectively to obtain the defect information of the local images; a mapping module is used to map the defect information in the local images to the global track image.
[0008] According to another embodiment of the present application, a computer storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned device embodiments when running.
[0009] According to another embodiment of the present application, there is also provided an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; wherein: the memory is used to store computer programs; the processor is used to execute the steps in the above method by running the program stored in the memory.
[0010] According to another embodiment of the present application, a computer program product including instructions is also provided. When the computer program product is run on a computer, the computer is enabled to execute the steps in the above method.
[0011] Through the embodiments of the present application, a global track image to be detected is obtained; the global track image is input into the TGL Net model for track surface defect detection; the global track image is divided into multiple non-overlapping local images, and features are extracted for each local image to capture key information such as texture and shape of the track surface, and feature fusion is performed on the extracted features. The fused feature maps are classified and regressed respectively, and the defect information of each local image is output. The defects in the local images are mapped to the global track image, and automatic recognition and positioning of the global track image is realized based on deep learning, which solves the limitations of manual inspections and the subjective and inefficient technical problems, which not only saves the labor costs of enterprises, but also greatly improves the efficiency and accuracy of defect detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0013] Figure 1 is a hardware structure block diagram of a computer according to an embodiment of the present application;
[0014] Figure 2 is a flow chart of a track defect detection method based on deep learning according to an embodiment of the present application;
[0015] Figure 3 is a flow chart of global track defect detection based on deep learning in an embodiment of the present application;
[0016] Figure 4 is an overall network structure diagram of an embodiment of the present application;
[0017] Figure 5 is a structural diagram of an image feature interaction module in an embodiment of the present application;
[0018] Figure 6 It is a structural block diagram of a track defect detection device based on deep learning in an embodiment of the present application. DETAILED DESCRIPTION
[0019] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only embodiments of a part of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.
[0020] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0021] Example 1
[0022] The method embodiment provided in the first embodiment of the present application can be executed in a server, a computer, a mobile phone, or a similar computing device. Taking running on a computer as an example, Figure 1 is a hardware structure block diagram of a computer in an embodiment of the present application. Figure 1 As shown, the computer may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the computer may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.
[0023] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as a computer program corresponding to a track defect detection method in an embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0024] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0025] In this embodiment, a track defect detection method based on deep learning is provided. Figure 2is a flow chart of a track defect detection method based on deep learning according to an embodiment of the present application, such as Figure 2 As shown, the process includes the following steps:
[0026] Step S10, obtaining a global track image to be detected;
[0027] Step S20, inputting the global track image into the TGL Net model for track surface defect detection; the TGL Net model includes an image label segmentation module, an image feature interaction module, a feature fusion pyramid network and a classification regression module; the image label segmentation module segments the global track image into a plurality of non-overlapping local images; the image feature interaction module extracts feature information of the local images; the feature fusion pyramid network performs feature fusion on the feature information; the classification regression module classifies and regresses the feature maps output by the feature fusion pyramid network to obtain defect information of the local images;
[0028] In this embodiment, the TGL Net model is a track surface defect detection model. The model includes an image labeling and segmentation module, an image feature interaction module, a feature fusion pyramid network, and a classification and regression module. First, the input global track image is segmented into multiple local images of the same size and non-overlapping, and features are extracted for each local image to capture key information such as the texture and shape of the track surface. The extracted features are fused, and the fused feature maps are classified and regressed respectively, where the classification task is used to determine whether there are defects and the type of defects, and the regression task is used to determine the specific location and size of the defects, and finally the defect information of each local image is output. Detect multiple prediction boxes in a local image, and only take the prediction result with the highest confidence score for multiple prediction results of the same prediction box.
[0029] Step S30, mapping the defect information in the local image to the global track image.
[0030] The defect information detected in the local image is mapped back to the global track image. After the mapping is completed, all detected defects will be marked on the global track image for subsequent analysis and processing.
[0031] Through this application, a global track image to be inspected is obtained; the global track image is input into the TGL Net model for track surface defect detection; the long-span global track image is divided into multiple non-overlapping local images, and features are extracted for each local image to capture key information such as texture and shape of the track surface, and the extracted features are fused. The fused feature maps are classified and regressed respectively, and the defect information of each local image is output. The defects in the local images are mapped to the global track image, and automatic recognition and positioning of the global track image is realized based on deep learning, which solves the limitations of manual inspections and the subjective and inefficient technical problems, which not only saves the labor costs of enterprises, but also greatly improves the efficiency and accuracy of defect detection.
[0032] In one implementation of this embodiment, the image marking segmentation module segments the global track image into a plurality of non-overlapping local images including:
[0033] The initial global track image enters the image labeling and segmentation module to preprocess the image. Figure 4 The image tag segmentation module includes a patch segmentation unit Patch Partition and a token embedding unit Token Embedding. The patch segmentation unit divides the input global track image H*W*3 into a set of non-overlapping local images (patches), where H and W are the sizes of the global track image. For example, the size of each patch is 8 x 8, the feature dimension of each patch is 8 x8 x 3=192, and the number of patch blocks is H / 8*W / 8. The token embedding unit then converts the feature dimension of the divided local image into a fixed dimension C that the model can handle. C is generally set to 96 and is marked as tokens.
[0034] This implementation splits the initial track image into multiple non-overlapping patches, encodes these patches into high-dimensional embeddings through linear projection, and then uses them as input to the feature fusion pyramid network. The track image is pre-processed using a patch segmentation unit and a label embedding unit to split the image into smaller and more manageable patches, and these patches are represented as more informative feature vectors through embedding techniques.
[0035] With the development of artificial intelligence, convolutional neural networks (CNN) have been widely used in defect detection tasks. This is because the inherent inductive biases of convolutional neural networks (such as spatial locality and translation equivariance) are good at capturing local patterns in defect features and can extract high-level features in images. However, high-speed tracks need to obtain long-distance semantic information. Traditional CNNs are limited by local receptive fields and are difficult to effectively capture long-distance track information. In response to this, this embodiment uses the Transformer structure as the feature extraction backbone network in the image feature interaction module. Compared with the CNN model, the self-attention-based Transformer model performs better in capturing the global information of the image.
[0036] In this embodiment, extracting the feature information of the local image by the image feature interaction module includes:
[0037] A21, dividing the local image into a plurality of windows at a first feature extraction layer, and performing self-attention calculation in each window;
[0038] A22, in the second feature extraction layer, the moving window partition generates a new window so that the new window contains the elements originally in different windows.
[0039] Reference Figure 5 In the first feature extraction layer (Layer 1) of the image feature interaction module, the local image is divided into multiple (such as 4) fixed-size and non-overlapping windows, and multi-head self-attention calculations are performed independently in each window, limiting the self-attention calculations to each window. Since there are relatively few elements in each window, the amount of calculations is significantly reduced. In the second feature extraction layer (Layer 1+1), the window is displaced on the basis of the first layer to form a new window (for example, all windows are moved two unit lengths to the lower right, and after the movement, 9 new windows are generated in the window partition). The new window will contain elements from different windows before, so that the self-attention calculation in the new window crosses the boundaries of the original windows of the first layer, builds connections between windows, provides information interaction between windows, and realizes global feature extraction of the image. In this embodiment, the following formula is used for self-attention calculation: , where Q represents the query matrix Query, K represents the key matrix Key, V represents the value matrix Value, that is, the extracted information, and B is the relative position offset. ; Q, K, V , d is the query or key dimension; is the number of patches per unit window; the self-attention mechanism first performs a dot product operation on the query and the key, then uses the SoftMax function to extract the attention ratio and multiply it by the value to obtain a specific feature map. When calculating the similarity, a relative position deviation is added to each head. , B can be used as a bias term to adjust the dot product result between Q and K, which can better focus on different data features, help the model maintain stable performance on different data sets, and improve the generalization ability of the model.
[0040] The traditional Transformer architecture relies on a global attention mechanism, which often results in large amounts of computation and increases computational complexity. In contrast, the image feature interaction module in this implementation limits attention computation to each window. This strategy not only optimizes the computation process, but also effectively reduces the overall amount of computation, demonstrating the efficiency and practicality of TGL Net in processing image features.
[0041] In the iterative process of the image feature interaction module, the original four windows are shifted to generate nine windows. However, increasing the number of windows will increase the computational requirements of the model. In order to avoid a significant increase in the amount of computation due to the increase in the number of windows, this implementation method re-splices the windows into four regions through cyclic shift, so that the amount of computation is consistent with the first layer. Figure 5 , move the top data of the window partition to the bottom, and then move the leftmost data to the rightmost. In addition, the new window may be composed of several non-adjacent sub-windows in the feature map, so the attention mask is used to limit the self-attention calculation to each sub-window, further reducing the computational cost.
[0042] This embodiment adds a global self-attention module to the target detection model, utilizing the Transformer's ability in global context modeling to improve the efficiency and accuracy of computer vision technology in rail surface defect detection. The global features of the defect image are obtained through the image feature interaction module, and cross-window information interaction is achieved while maintaining efficient calculation in non-overlapping windows.
[0043] In one implementation of this embodiment, the feature fusion of the feature information through the feature fusion pyramid network includes: using the global feature information output by the image feature interaction module as the input layer of the feature fusion pyramid network, fusing two adjacent low-level features, and gradually incorporating high-level features into the fusion process. This implementation gradually introduces more context information from low to high levels through the feature fusion pyramid network to achieve a more refined feature representation.
[0044] In another implementation of this embodiment, the classification and regression module receives the three-layer feature map output by the feature fusion pyramid network, and the classification and regression steps are independent of each other. Predictions are made separately based on different feature information output by the classification and regression to adapt to defects of different categories, scales and shapes.
[0045] In the classification-regression module, classification and regression are independent of each other. In common detectors, there are two types of detection heads: coupled heads and decoupled heads. The difference lies in whether the classification and regression tasks share the same feature representation. For the coupled head, classification and regression tasks are performed together. Since the content of the trained convolutional layer is unknown, the captured local features are suitable for classification rather than regression. The classification task only needs to respond to object-specific features, while the regression task needs to consider global features to locate the boundaries of defects of various shapes and scales, which may lead to mutual interference between tasks.
[0046] In the decoupling head, classification and regression are independent of each other, and different feature representations are predicted separately, which can better accommodate defects of different categories, scales, and shapes, thereby improving the generalization ability of the model. Therefore, considering that the distribution and types of surface defects of railway tracks are different, the highly random distribution and shape of defects bring greater challenges to track defect detection. On fast tracks, the size of defects is smaller, but more obvious, because fast trains travel faster and cause greater damage to the tracks. On the contrary, on ordinary lanes or heavy lanes, the travel speed is slower than that of high-speed trains, and long-term contact with the track leaves a large area of wear. Railway curves are usually one of the high-incidence areas of track defects. When the train passes through the curve, the track is subjected to additional pressure, which can easily cause the rail to wear and defects. Due to the high randomness of the defect distribution and shape, this embodiment uses a decoupled method to classify and regress track surface defects. The decoupling head enables the model to detect defects more accurately, adapt to complex track environments, improve training efficiency, and enhance the interpretability of the model, which can effectively improve the accuracy and robustness of track surface defect detection.
[0047] In this embodiment, given that most local images usually only contain background, this embodiment predicts the confidence score of each local image containing defects through the detection head. The score is 1 when the local image contains defects and 0 when it only contains background. When there are no defects in the local image, the detection head does not need to waste resources to predict an accurate bounding box, thereby improving the learning efficiency of the model. Finally, according to the patch segmentation strategy used, the patch prediction results are assigned back to the original image. The model reduces unnecessary analysis of the background and optimizes the allocation of computing resources.
[0048] In one implementation of this embodiment, mapping the defect information in the local image to the global track image includes:
[0049] S31, determining a position index of the local image in the global track image, where the position index includes a row index and a column index;
[0050] S32, obtaining a defect position of a defect in the local image;
[0051] S33, calculating the position of the defect in the local image in the global track image according to the defect position, the row index and the column index.
[0052] The global orbit image (W L , H L ) is divided into multiple local images (Ws, Hs), and each local image is assigned a position index (i, j), where i is the row index, indicating the position of the local image in the vertical direction, and j is the column index, indicating the position of the local image in the horizontal direction.
[0053] Through the image feature interaction module, feature fusion pyramid network and classification regression module, defects are detected in each local image to obtain the coordinate information of the defect in the local image ( , , w, h), where ( , ) is the center coordinate of the defect in the local image, w is the width of the defect, and h is the height of the defect.
[0054] Calculate the starting position of the local image in the global orbit image (i.e., the coordinates of the upper left corner of the local image). The starting position can be obtained by multiplying the position index of the local image by the width and height of the local image. For defects detected in the local image, the defect position needs to be mapped to the global image. The position of the defect is represented by the defect center coordinates, defect width, and defect height. Add the center coordinates of the local image to the starting position of the local image in the global orbit image to obtain the center coordinates of the defect in the global orbit image ( , ), the calculation formula is: ; The width and height of the defect remain unchanged during the mapping process. The position information of the defect in the global image is output, and the output results include the center coordinates, width and height of the defect in the global track image, thereby realizing the mapping of the prediction results in the local image to the global image.
[0055] In this embodiment, before the global track image is input into the TGL Net model for track surface defect detection, the method further includes:
[0056] S11, obtaining a track surface defect dataset;
[0057] S12, performing data enhancement on the track surface defect dataset to obtain a track surface defect sample set;
[0058] S13, using the track surface defect sample set to train the initial model, and after the training, evaluating the model and adjusting parameters to obtain a TGL Net model for track surface defect detection.
[0059] The track surface defect dataset RSDDs is obtained, and the original data can be preprocessed to remove images without defects in the data. After manual annotation, the dataset is augmented and expanded to obtain a sample set. The sample set is randomly divided into a training set, a validation set, and a test set according to a ratio of 7:2:1. The specific dataset composition is shown in Table 1. The image data of the training set is used for model training, the image data of the test set is used to test the training effect of the model, and the image data of the validation set is used to verify the detection ability of the model. The model continuously adjusts the network parameters according to the verification effect through a large amount of training.
[0060] Table 1:
[0061]
[0062] The TGLNet model is trained using the training set, and after the training, the model is evaluated and the parameters are adjusted to obtain the optimal TGL Net model. The experimental results show that the detection accuracy of TGL Net for two track surface defect datasets Type Ⅰ and Type Ⅱ can reach 92.3% and 68.7% respectively.
[0063] After training, the model is evaluated and parameters are adjusted to obtain the TGL Net model for track surface defect detection, including:
[0064] Calculate bounding box regression loss, confidence loss, and category loss. The loss function quantifies the error or loss between the model's predicted value and the true value.
[0065] Bounding box regression loss is used to measure the difference between the predicted defect bounding box and the actual defect bounding box. Smooth L1 Loss or GIoU Loss can be used to calculate the bounding box regression loss. The calculation formula of Smooth L1 Loss is: ,in, is the predicted bounding box, is the true bounding box, N is the number of bounding boxes, and IoU is the intersection over union ratio; the GIoU Loss calculation formula is: , where IOU is the intersection-over-union ratio of the predicted box Box and the real box GT;
[0066] Confidence loss is used to measure the accuracy of the model's prediction of the presence or absence of defects; the confidence loss is calculated as: , where N is the total number of bounding boxes. It is i The confidence level of the prediction, It is i The true confidence label.
[0067] Category loss is used to measure the prediction accuracy of the model for defect categories. Cross-Entropy Loss can be used to calculate category loss. The calculation formula for category loss is: ,in, is the probability of the positive class (i.e., defect existence) of the i-th prediction, is the true label of the i-th sample.
[0068] Performing a weighted summation on the bounding box regression loss, the confidence loss, and the category loss to obtain a total loss;
[0069] Total loss function is the bounding box regression loss , confidence loss and category loss The weighted sum of these three losses is calculated as follows:
[0070] , where by assigning appropriate weights to different parts in the total loss function , , , which can balance the model’s learning on these different tasks, allowing the model to both accurately classify defects and precisely locate the locations of defects.
[0071] The loss function guides model optimization by measuring the gap between model predictions and actual annotations, thereby improving the performance and accuracy of the model.
[0072] The total loss is used to evaluate the model and adjust parameters to obtain the TGLNet model for track surface defect detection.
[0073] Model calculation average precision mAP, mAP 50 、mAP 75 、mAP S 、mAP MThe trained track defect detection model is used to test the track defect images with Params as evaluation indicators. If the model's mAP and other average accuracy indicators are greater than or equal to 90%, the model is considered to be able to accurately detect track defects, otherwise the model parameters are optimized and fine-tuned. Parameter optimization and fine-tuning, combined with the evaluation results, further optimizes and fine-tunes the track defect detection model, mainly by increasing the number of training samples, modifying the learning rate, and the number of iterations.
[0074] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0075] Example 2
[0076] In this embodiment, a track defect detection device based on deep learning is also provided to implement the above embodiments and preferred implementation modes, which have been described and will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0077] Figure 6 is a structural block diagram of a track defect detection device based on deep learning according to an embodiment of the present application, such as Figure 6 As shown, the device comprises:
[0078] An acquisition module 61 is used to acquire a global track image to be detected;
[0079] The detection module 62 is used to input the global track image into the TGL Net model for track surface defect detection; the TGL Net model includes an image label segmentation module, an image feature interaction module, a feature fusion pyramid network and a classification regression module; the image label segmentation module segments the global track image into multiple non-overlapping local images; the image feature interaction module extracts feature information of the local images; the feature fusion pyramid network performs feature fusion on the feature information; the classification regression module classifies and regresses the feature maps output by the feature fusion pyramid network to obtain defect information of the local images;
[0080] The mapping module 63 is used to map the defect information in the local image to the global track image.
[0081] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.
[0082] Example 3
[0083] An embodiment of the present application further provides a computer storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.
[0084] Optionally, in this embodiment, the computer storage medium may be configured to store a computer program for performing the following steps:
[0085] S1, obtaining a global track image to be detected;
[0086] S2, inputting the global track image into the TGL Net model for track surface defect detection; the TGL Net model includes an image label segmentation module, an image feature interaction module, a feature fusion pyramid network and a classification regression module; the image label segmentation module segments the global track image into multiple non-overlapping local images; the image feature interaction module extracts feature information of the local images; the feature fusion pyramid network performs feature fusion on the feature information; the classification regression module classifies and regresses the feature maps output by the feature fusion pyramid network to obtain defect information of the local images;
[0087] S3, mapping the defect information in the local image to the global track image.
[0088] Optionally, in this embodiment, the above-mentioned computer storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.
[0089] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0090] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.
[0091] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:
[0092] S1, obtaining a global track image to be detected;
[0093] S2, inputting the global track image into the TGL Net model for track surface defect detection; the TGL Net model includes an image label segmentation module, an image feature interaction module, a feature fusion pyramid network and a classification regression module; the image label segmentation module segments the global track image into multiple non-overlapping local images; the image feature interaction module extracts feature information of the local images; the feature fusion pyramid network performs feature fusion on the feature information; the classification regression module classifies and regresses the feature maps output by the feature fusion pyramid network to obtain defect information of the local images;
[0094] S3, mapping the defect information in the local image to the global track image.
[0095] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.
[0096] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0097] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.
[0098] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
[0099] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0100] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0101] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a computer storage medium, including several instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned computer storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.
[0102] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A rail defect detection method based on deep learning, characterized in that: The method comprises: Acquire a global track image to be detected; The global track image is input into the TGL Net model for track surface defect detection; the TGL Net model includes an image label segmentation module, an image feature interaction module, a feature fusion pyramid network and a classification regression module; the image label segmentation module segments the global track image into multiple non-overlapping local images; the image feature interaction module extracts feature information of the local image; the feature fusion pyramid network is used to perform feature fusion on the feature information; the classification regression module classifies and regresses the feature map output by the feature fusion pyramid network to obtain defect information of the local image; Mapping defect information in the local image to the global track image; The image feature interaction module includes a first feature extraction layer and a second feature extraction layer. Extracting feature information of the local image through the image feature interaction module includes: dividing the local image into multiple windows in the first feature extraction layer, and performing self-attention calculation in each window; moving the window partition to generate a new window in the second feature extraction layer, so that the new window contains elements originally in different windows; Mapping the defect information in the local image to the global orbit image includes: determining a position index of the local image in the global orbit image, wherein the position index includes a row index and a column index; obtaining a defect position of the defect in the local image; and calculating a position of the defect in the local image in the global orbit image according to the defect position, the row index, and the column index; The classification and regression module classifies and regresses the feature map output by the feature fusion pyramid network, respectively, including: the classification and regression module predicts whether the feature map has defects; obtains the feature map with defects, performs subsequent defect detection on the feature map with defects, and maps the feature map without defects back to the global track image.
2. The method according to claim 1, characterized in that Before inputting the global track image into the TGL Net model for track surface defect detection, the method further includes: Acquire track surface defect dataset; Performing data enhancement on the track surface defect dataset to obtain a track surface defect sample set; The track surface defect sample set is used to train the initial model. After the training, the model is evaluated and parameters are adjusted to obtain a TGL Net model for track surface defect detection.
3. The method according to claim 2, characterized in that The track surface defect dataset is enhanced to obtain a track surface defect sample set including: The track surface defect images are scaled, horizontally flipped, rotated, Gaussian blurred, noisy, and brightness changed, and the specific background is fused to perform data expansion to obtain a sample set.
4. The method according to claim 1, characterized in that After training, the model is evaluated and parameters are adjusted to obtain the TGL Net model for track surface defect detection, including: Calculate bounding box regression loss, confidence loss and category loss. The bounding box regression loss is used to measure the difference between the predicted defect bounding box and the actual defect bounding box; the confidence loss is used to measure the model's prediction accuracy of whether a defect exists; and the category loss is used to measure the model's prediction accuracy of the defect category. Performing a weighted summation on the bounding box regression loss, the confidence loss, and the category loss to obtain a total loss; The total loss is used to evaluate the model and adjust parameters to obtain the TGL Net model for track surface defect detection.
5. A track defect detection device based on deep learning, characterized in that: include: An acquisition module, used for acquiring a global track image to be detected; A detection module is used to input the global track image into a TGL Net model for track surface defect detection; the TGL Net model includes an image marking segmentation module, an image feature interaction module, a feature fusion pyramid network and a classification regression module; the image marking segmentation module divides the global track image into multiple non-overlapping local images; the image feature interaction module extracts feature information of the local images; the feature fusion pyramid network performs feature fusion on the feature information; the classification regression module classifies and regresses the feature maps output by the feature fusion pyramid network to obtain defect information of the local images; A mapping module, used for mapping the defect information in the local image to the global track image; The image feature interaction module includes a first feature extraction layer and a second feature extraction layer. Extracting feature information of the local image through the image feature interaction module includes: dividing the local image into multiple windows in the first feature extraction layer, and performing self-attention calculation in each window; moving the window partition to generate a new window in the second feature extraction layer, so that the new window contains elements originally in different windows; Mapping the defect information in the local image to the global orbit image includes: determining a position index of the local image in the global orbit image, wherein the position index includes a row index and a column index; obtaining a defect position of the defect in the local image; and calculating a position of the defect in the local image in the global orbit image according to the defect position, the row index, and the column index; The classification and regression module classifies and regresses the feature map output by the feature fusion pyramid network, respectively, including: the classification and regression module predicts whether the feature map has defects; obtains the feature map with defects, performs subsequent defect detection on the feature map with defects, and maps the feature map without defects back to the global track image.
6. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; wherein: Memory, used to store computer programs; A processor, configured to execute the track defect detection method according to any one of claims 1 to 4 by running a program stored in a memory.
7. A computer storage medium, characterized in that: The computer storage medium includes a stored program, wherein the track defect detection method according to any one of claims 1 to 4 is executed when the program is executed.