A method, apparatus, device and medium for detecting rail damage

Abstract

The present application provides a rail damage detection method, device, equipment and medium, the method comprises: obtaining a to-be-detected picture; inputting the to-be-detected picture into a pre-trained rail damage detection model to obtain a damage category; wherein the rail damage detection model is used for preprocessing the to-be-detected picture to obtain a plurality of scale pooling feature maps, extracting the scale perception weight of each pooling feature map and mapping it to the pooling feature map to obtain a first fusion feature map, extracting the spatial perception weight of the first fusion feature map and mapping it to the first fusion feature map to obtain a second fusion feature map; identifying the second fusion feature map to obtain the damage category, which realizes scale perception first and then spatial perception for each pooling feature map, can obtain a larger and more flexible convolution kernel receptive field under the condition of the same size convolution kernel, improves the precision of rail damage detection type, and reduces the false detection rate.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of dynamic detection, and in particular to a rail damage detection method, device, equipment and medium. BACKGROUND

[0002] Rail damage refers to the breaking, cracking and other states that affect and limit the performance of the rail during use.

[0003] Target detection is a technical direction of computer vision and digital image processing, and is widely used in robot navigation, intelligent video monitoring, industrial detection, aerospace and many other fields. Through computer vision, the consumption of human capital is reduced, which has important practical significance. Target detection is an important branch of image processing and computer vision. Unlike image classification, target detection not only needs to determine the type of target in the image, but also needs to determine the coordinate position of the target in the image. Thanks to the rapid development of machine learning, target detection methods based on deep learning have developed rapidly.

[0004] In the prior art, in the process of predicting rail damage by using a small sample method based on the deep learning theory, the false detection rate of rail damage is too high, and the small damage target detection accuracy is low due to the small number of industrial samples. SUMMARY

[0005] In view of the above problems of the prior art, the purpose of the present application is to provide a rail damage detection method, device, equipment and medium to solve the problem of high false detection rate of rail damage and low small damage target detection accuracy in the process of predicting rail damage by using a small sample method in the prior art.

[0006] To solve the above technical problems, the specific technical solutions of the present application are as follows:

[0007] On the one hand, the present application provides a rail damage detection method, comprising:

[0008] obtaining a to-be-detected picture;

[0009] inputting the to-be-detected picture into a pre-trained rail damage detection model to obtain a damage category;

[0010] The rail damage detection model comprises a preprocessing module and a category detection module.

[0011] The preprocessing module is used for preprocessing the to-be-detected picture to obtain a plurality of scale pooling feature maps.

[0012] The category detection module is configured to extract scale-aware weights of each pooled feature map and map the scale-aware weights to the pooled feature map to obtain a first fused feature map, extract spatial-aware weights of the first fused feature map and map the spatial-aware weights to the first fused feature map to obtain a second fused feature map, and identify the second fused feature map to obtain the damage category.

[0013] extracting features of the to-be-detected picture to obtain an initial feature map of the to-be-detected picture;

[0014] performing down-sampling or up-sampling on the initial feature map according to a preset scale to obtain a multi-scale feature map;

[0015] performing region extraction and region pooling processing on each scale feature map to obtain a region feature map of each scale;

[0016] performing scale feature pooling processing on the region feature map of each scale to obtain a plurality of pooled feature maps.

[0017] As an embodiment of the present disclosure, the scale-aware weights of each pooled feature map are extracted and mapped to the pooled feature map to obtain a first fused feature map, which further includes:

[0018] performing average pooling on each pooled feature map to obtain an average pooled feature map;

[0019] performing linear convolution processing and nonlinear transformation on the average pooled feature map to obtain scale-aware weights of each pooled feature map;

[0020] performing product processing on the scale-aware weights of each pooled feature map and the pooled feature map to obtain the first fused feature map.

[0021] As an embodiment of the present disclosure, the average pooled feature map is subjected to linear convolution processing and nonlinear transformation to obtain scale features of each pooled feature map, which includes calculating the scale-aware weights of each pooled feature map by using the following formula:

[0022]

[0023] wherein, is an average value of the pooled feature map, F is the pooled feature map, S is determined according to the height and width of the pooled feature map, C is the number of channels of the feature map, f is a linear convolution function, and σ is an activation function.

[0024] As an embodiment of the present disclosure, the spatial-aware weights of the first fused feature map are extracted and mapped to the first fused feature map to obtain a second fused feature map, which further includes:

[0025] performing multi-kernel linear convolution and deformable convolution operation on the first fused feature map respectively to obtain spatial perception weight;

[0026] performing product operation on the first fused feature map and the spatial perception weight to obtain the second fused feature map.

[0027] As an embodiment of the present document,

[0028] The category detection module comprises a plurality of category detection branches, each category detection branch comprising a first lightweight dynamic detection module and a binary classification module, and each category detection branch is trained by using a historical picture of one category of rail damage;

[0029] The scale features of each pooled feature map are extracted by using the first lightweight dynamic detection module and mapped to the pooled feature map to obtain a first fused feature map, the spatial perception weight of the first fused feature map is extracted and mapped to the first fused feature map to obtain a second fused feature map; the second fused feature map is identified by using the binary classification module to obtain the damage category.

[0030] As an embodiment of the present document,

[0031] The category detection module further comprises a region detection branch, the region detection branch comprising a second lightweight dynamic detection module and a regression module, and being trained by using historical pictures of all categories of rail damage;

[0032] The scale features of each pooled feature map are extracted by using the second lightweight dynamic detection module and mapped to the pooled feature map to obtain a third fused feature map, the spatial perception weight of the third fused feature map is extracted and mapped to the third fused feature map to obtain a fourth fused feature map;

[0033] The fourth fused feature map is identified by using the regression module to determine the damage position in the picture to be detected.

[0034] In another aspect, the present document also provides a rail damage detection device, comprising:

[0035] An acquisition unit is configured to acquire a picture to be detected;

[0036] A damage category determination unit is configured to input the picture to be detected into a pre-trained rail damage detection model to obtain a damage category;

[0037] The rail damage detection model comprises a preprocessing module and a category detection module.

[0038] The preprocessing module is configured to preprocess the picture to be detected to obtain a plurality of scale pooled feature maps.

[0039] The category detection module is configured to extract scale-aware weights of each pooled feature map and map the scale-aware weights to the pooled feature map to obtain a first fused feature map, extract spatial-aware weights of the first fused feature map and map the spatial-aware weights to the first fused feature map to obtain a second fused feature map, and identify the second fused feature map to obtain the damage category. In another aspect, the present document also provides a computer device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the rail damage detection method of any one of the aspects when executing the computer program.

[0040] In another aspect, the present document also provides a computer readable storage medium, which stores a computer program, and the computer program is executable on a processor to implement the rail damage detection method of any one of the aspects.

[0041] The above technical solution realizes scale-awareness and spatial-awareness for each pooled feature map, can obtain a larger and more flexible convolution kernel receptive field under the condition of the same size of convolution kernel, improves the accuracy of rail damage detection types, and reduces the false detection rate.

[0042] In order to make the above and other objects, features and advantages of the present document more apparent and understandable, the following will describe preferred embodiments in detail, and the accompanying drawings will be described as follows. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the present document, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0044] Figure 1 The overall system diagram of the rail damage detection method of the embodiment of the present document is shown;

[0045] Figure 2 The step schematic diagram of the rail damage detection method of the embodiment of the present document is shown;

[0046] Figure 3 The training method schematic diagram of the rail damage detection model of the embodiment of the present document is shown

[0047] Figure 4 The rail damage detection device schematic diagram of the embodiment of the present document is shown;

[0048] Figure 5A topological diagram of a rail damage model of a rail damage detection device according to an embodiment of this invention is shown;

[0049] Figure 6 A data processing flow chart of a rail damage detection method according to an embodiment of the present invention is shown;

[0050] Figure 7 A data processing flow chart of a category-level detection module of a rail damage detection method according to an embodiment of this invention is shown;

[0051] Figure 8 A data processing flow chart of a lightweight dynamic detection module of a rail damage detection method according to an embodiment of this invention is shown;

[0052] Figure 9 A schematic diagram of a computer device according to an embodiment of the present invention is shown.

[0053] Description of the accompanying symbols:

[0054] 101. Camera;

[0055] 102. Database;

[0056] 103. Computing server;

[0057] 401. Get unit;

[0058] 402. Damage category determination unit;

[0059] 501, preprocessing module;

[0060] 502. Category detection module;

[0061] 5021, category detection branch;

[0062] 50211, the first lightweight dynamic detection module;

[0063] 50212, binary classification module;

[0064] 5022, regional detection branch;

[0065] 50221, the second lightweight dynamic detection module;

[0066] 50222, regression module;

[0067] 1. The image to be tested;

[0068] 2. Feature extraction module;

[0069] 3. Pooling module;

[0070] 4. Scale feature pooling module;

[0071] 511, Average Pooling Network;

[0072] 512, linear convolution network;

[0073] 513, learning factor network;

[0074] 514, first nonlinear transformation network;

[0075] 515, first product network;

[0076] 516, constant factor network;

[0077] 517, symmetric convolution network;

[0078] 518, second nonlinear transformation network;

[0079] 519, variability convolution network;

[0080] 5110, second product network;

[0081] 902, computer device;

[0082] 904, processor;

[0083] 906, memory;

[0084] 908, drive mechanism;

[0085] 910, input / output module;

[0086] 912, input device;

[0087] 914, output device;

[0088] 916, presentation device;

[0089] 918, graphical user interface;

[0090] 920, network interface;

[0091] 922, communication link;

[0092] 924, communication bus. DETAILED DESCRIPTION

[0093] The technical solutions in the embodiments herein will be described clearly and completely below with reference to the accompanying drawings in the embodiments herein. Obviously, the described embodiments are only part of the embodiments herein, rather than all the embodiments. Based on the embodiments herein, all the other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection.

[0094] It is to be understood that the terms "first", "second", and the like, used in the description and the claims of the present document as well as the above-described drawings, are used to distinguish between similar objects, and are not necessarily used to describe a particular sequential or chronological order. It is to be understood that the data thus used can be interchanged, where appropriate, so that the embodiments of the present document described herein can be carried out in other than the order shown or described herein. Furthermore, the terms "comprise" and "have", and any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, device, product, or apparatus that comprises a list of steps or units is not necessarily limited to those steps or units that are clearly listed, but can include other steps or units that are not clearly listed or inherent to such processes, methods, products, or apparatuses.

[0095] The present specification provides method operation steps as described in the embodiments or flowcharts, but can include more or less operation steps based on conventional or non-inventive labor. The order of steps listed in the embodiments is only one of the many execution orders of the steps, and does not represent the only execution order. In actual system or device product execution, the method order shown in the embodiments or the drawings can be executed in sequence or in parallel.

[0096] It should be noted that the data involved in the present application (including but not limited to data for detection, stored data, displayed data, etc.) are all information and data authorized by the user or authorized by all parties.

[0097] As Figure 1 A steel rail damage detection method includes a camera 101, a database 102, and an operation server 103.

[0098] It should be noted that the camera 101 herein can be set and run on a train, or can be run on a quality detection vehicle dedicated to a railway track. The camera 101 is used to take real-time pictures of the steel rail. The shooting frequency can be set as needed, for example, it can be set to four sampling points per meter, which means that four pictures need to be taken at equal intervals within a distance of one meter. That is, the corresponding mileage of each picture is 0.25 meters. After the camera 101 finishes shooting, the picture name can be named with the corresponding position coordinates and saved to the database 102.

[0099] The database 102 can save the corresponding to-be-detected pictures of the steel rails of a plurality of road sections, wherein the to-be-detected pictures are saved in time sequence. For example, in the first quarter of 2022, the to-be-detected pictures are uniformly saved, that is, the quality of the steel rails in the first quarter is detected, and whether there is damage in the to-be-detected pictures is checked. In this way, factors affecting train operation are excluded to reduce the risk of train operation and improve the safety and comfort of the train.

[0100] The operation server 103 runs a rail damage detection model. The rail damage detection model can preprocess a to-be-detected picture, and according to a result of the preprocessing, determine whether the to-be-detected picture is a damaged picture. If the to-be-detected picture is determined to be a damaged picture, the rail damage detection model can label a damage type and a damage position. The damage type corresponds to a damage type specified in an industry standard. There are up to nine different types of damage. In this way, the rail maintenance personnel can be informed in advance, so that the rail maintenance personnel can prepare tools and maintenance plans accordingly, and treat various types of damage with corresponding treatment, and quickly eliminate the damage.

[0101] In the prior art, in the method for detecting rail damage based on deep learning theory, due to the small number of industrial samples, when a small sample method is used for rail damage prediction, the false detection rate of rail damage is too high, and the detection accuracy of small damage targets is low.

[0102] To solve the above technical problems, a rail damage detection model is constructed and trained. The model can be used to receive a to-be-detected picture and determine which type of damage the to-be-detected picture belongs to among nine damage types. Compared with the prior art of determining whether a damage occurs among 32 damage types, the rail damage detection model determines which type of damage the damage belongs to among nine damage types, which greatly improves the damage detection accuracy and reduces the false detection rate.

[0103] It should be noted that the rail damage in this paper includes the breaking, cracking and other states that affect and limit the use performance of the rail during use. For example, it can refer to the breaking, cracking and other states that affect and limit the use performance of the rail during use. Rail cracking refers to the separation of part of the metal on the surface or inside the rail. It can also be other damage, such as wear, crushing, sinking (or depression), wave wear, bending deformation, surface defects, external damage, corrosion and other damage that affect the use performance of the rail in addition to cracking and breaking. The damage state of the rail refers to the macroscopic appearance (or shape) characteristics of the rail damage. It directly records and reflects the development process and result of the damage.

[0104] To solve the above problems, the embodiment of the present application provides a rail damage detection method, which can solve the problem of high false detection rate of rail damage and low detection accuracy of small damage targets when a small sample method is used for rail damage prediction, Figure 2is a schematic diagram of the steps of a rail damage detection method provided by the embodiments of the present application, the specification provides the method operation steps as described in the embodiments or flowcharts, but more or fewer operation steps can be included based on conventional or non-inventive labor. The order of steps listed in the embodiments is only one of the many step execution orders, and does not represent the only execution order. In actual system or device product execution, the method order shown in the embodiments or the drawings can be executed in sequence or in parallel. Specifically as shown in Figure 2 The method can include:

[0105] Step 201, obtaining a picture to be detected.

[0106] Step 202, inputting the picture to be detected into a pre-trained rail damage detection model to obtain a damage category.

[0107] The rail damage detection model includes a preprocessing module and a category detection module.

[0108] The preprocessing module is used to preprocess the picture to be detected to obtain a plurality of scale pooling feature maps.

[0109] The category detection module is used to extract scale perception weights of each pooling feature map and map them to the pooling feature map to obtain a first fusion feature map, extract spatial perception weights of the first fusion feature map and map them to the first fusion feature map to obtain a second fusion feature map, and identify the second fusion feature map to obtain the damage category. Through the above method, each pooling feature map is first scaled and then spatially perceived, which can achieve a larger and more flexible convolution kernel receptive field under the same size convolution kernel, improve the accuracy of rail damage detection type, and reduce the false detection rate.

[0110] As Figure 3 A schematic diagram of a training method of a rail damage detection model is shown, as an embodiment of the present application, the training process of the rail damage detection model of the present application includes:

[0111] Step 301, importing a training set into the rail damage detection model and performing training, and obtaining corresponding model parameters after training once.

[0112] In this step, after the training set is imported into the rail damage detection model, the predicted damage category and damage position can be obtained. Each image in the training set has a pre-labeled standard damage category and standard damage position. The loss function of the rail damage detection model is calculated by the predicted damage category, the predicted damage position, the standard damage category and the standard damage position.

[0113] Specifically,

[0114] loss = l box + l cls + l obj ;

[0115]

[0116] wherein, loss is a loss function, l box is a damage location box regression error, l cls is a classification judgment error, l obj is a target detection error, λ coord , λ class , λ noobj , λ obj is a weight, S 2 is the number of times of detection traversal by dividing each image in the batch data into SxS grids, is whether there is a target of interest (i.e. damage) in the (i, j)th damage location candidate box, ω i , h i , x i , y i are the width, height, horizontal coordinate and vertical coordinate of the predicted damage location candidate box, respectively, are the width, height, horizontal coordinate and vertical coordinate of the labeled damage location box, respectively, B is the number of damage location candidate boxes; classes is the number of categories of rail damage.

[0117] p i (c) is the probability that the object in the grid i belongs to category c, is the true probability that the object belongs to category c;

[0118] is whether there is no target of interest in the (i, j)th damage location candidate box, c i is the predicted damage category, is the labeled damage category.

[0119] After the loss function is calculated, the derivative of the loss function can be back-propagated to each model parameter of the rail damage detection model, so as to complete the update of the model parameters.

[0120] For example, after completing an update, the updated model parameters can be recorded and the model parameters can be calculated one by one. For example, after 10 training, 10 kinds of model parameters can be obtained, and each kind of model parameter corresponds to a different rail damage detection model. When the same image is sent to the rail damage detection models with different model parameters, different results can be obtained.

[0121] Further, in order to accelerate the training speed of the rail damage detection model, a processed data set such as the Microsoft COCO data set can be used for the rail detection model to prevent the rail damage detection model from Z-shaped convergence.

[0122] In step 302, the verification set is respectively introduced into the rail damage detection model with different model parameters, and the precision and recall of the rail damage detection model are determined, wherein the precision and the recall are obtained according to the intersection over union threshold.

[0123] In this step, since the step 301 has completed the phase training, it is necessary to verify the trained rail damage detection model, introduce the verification set into the rail damage detection model, and calculate the precision and recall. The precision can represent the accuracy of the rail damage detection model prediction, that is, the overlap area between the predicted detection frame and the labeled detection can be calculated. The recall can represent the probability of prediction error, that is, the probability of correctly detecting damage.

[0124] In this paper, the intersection over union threshold IOU can be set to any decimal number between 0-1.0, such as 0.1, 0.2, 0.3, etc.

[0125] Let TP be the number of positive samples predicted as positive (True Positive), that is, the detection frame detects the object and the IOU is greater than the threshold, indicating successful detection.

[0126] Let FP be the number of positive samples predicted as negative (False Positive), that is, the detection frame detects the object but the IOU is less than the threshold, indicating detection error.

[0127] Let FN be the number of negative samples predicted as negative (False Negative), that is, the detection frame fails to detect the object, but actually contains the object, indicating missed detection.

[0128] Let TN be the number of negative samples predicted as positive (True Negative), that is, the detection frame fails to detect the object, and actually does not contain the object. It is actually impossible to calculate and unnecessary to calculate.

[0129] Precision

[0130] Wherein, TP is the number of samples whose detection frame detection damage position coordinate interval and real damage position coordinate interval proportion is greater than the intersection over union threshold;

[0131] FP is the number of samples whose detection frame detection damage position coordinate interval and real damage position coordinate interval proportion is less than the intersection over union threshold;

[0132] Recall

[0133] TP is the number of samples whose detection box coordinate interval of the injury position and the real injury position coordinate interval ratio is greater than the intersection and union ratio threshold value;

[0134] FN is the number of samples whose detection box does not detect the injury sample, but actually contains the injury sample.

[0135] The verification stage can be performed multiple times, and in the embodiments of the present application, the training period of the verification stage can be 500.

[0136] Step 303, incrementing the intersection and union ratio threshold value to obtain the corresponding PR curve, wherein the PR curve is depicted according to the precision and recall.

[0137] In this step, because the precision and recall are obtained according to the intersection and union ratio threshold value, adjusting the intersection and union ratio threshold value will obtain a set of precision and recall.

[0138] After adjusting the intersection and union ratio threshold value to obtain a set of precision and recall, the precision can be taken as the ordinate, and the recall can be taken as the abscissa to draw a two-dimensional curve, i.e. the PR curve.

[0139] Step 304, calculating the mAP of the rail damage detection model according to the area surrounded by the PR curve and the coordinate axes.

[0140] The area surrounded by the PR curve and the coordinate axes is taken as the AP (Average Precision).

[0141] Step 305, selecting the rail damage detection model according to the mAP.

[0142] Because in step 301, a number of model parameters are obtained, by adjusting the intersection and union ratio threshold value, the AP of all rail damage detection models is calculated, and the average value of all APs is calculated, i.e. the mAP (mean Average Precision) is obtained, and the rail damage detection model with the maximum mAP is taken as the model used in the embodiments of the present application.

[0143] As an embodiment of the present application, the step of preprocessing the to-be-detected picture to obtain a plurality of scale pooling feature maps further comprises:

[0144] Extracting the features of the to-be-detected picture to obtain the initial feature map of the to-be-detected picture.

[0145] According to the preset scale, down-sampling or up-sampling the initial feature map to obtain a multi-scale feature map.

[0146] The region extraction and region pooling processing are performed on each scale feature map to obtain a region feature map of each scale.

[0147] The scale feature pooling processing is performed on the region feature map of each scale to obtain a plurality of pooled feature maps.

[0148] In this step, the features of the to-be-detected picture can be extracted by a feature extraction module, for example, the initial feature map of the to-be-detected picture is extracted by a feature pyramid network, and in this paper, at least three initial feature maps of the to-be-detected picture can be extracted according to different sizes. The extraction method includes convolution and sampling.

[0149] An intermediate scale, for example, 50x50, can be preset, and the initial feature map larger than the preset scale is down-sampled, for example, reduced; and the initial feature map smaller than the preset scale is up-sampled, for example, enlarged and spliced.

[0150] After the scale transformation is completed, the feature maps of each scale are subjected to region extraction and region pooling, and the region feature maps in various regions can be obtained.

[0151] The region feature maps subjected to region pooling are subjected to pooling of various scales, and a plurality of pooled feature maps can be obtained.

[0152] In the above manner, the receptive field can be increased, and the difficulty of subsequent feature extraction and the dimension of the parameters can also be reduced.

[0153] As an embodiment of the present disclosure, the category detection module includes a plurality of category detection branches, each category detection branch includes a first lightweight dynamic detection module and a binary classification module, each category detection branch is trained by using a historical picture of one category of rail damage;

[0154] The scale features of each pooled feature map are extracted by using the first lightweight dynamic detection module and mapped to the pooled feature map to obtain a first fused feature map, the spatial perception weight of the first fused feature map is extracted and mapped to the first fused feature map to obtain a second fused feature map; the binary classification module is used to identify the second fused feature map to obtain the damage category.

[0155] In the embodiments of the present application, the rail damage detection model can include several category detection branches, wherein each category detection branch has its corresponding lightweight dynamic detection module. The function of the lightweight dynamic detection module is to input the corresponding 0-1 number according to the pooling feature map. For example, the category detection branch in the present application is 9, and the module structure and parameters between the lightweight dynamic detection modules can be different. That is, when the same feature map is input into different lightweight dynamic detection modules, different results can be obtained. For example, the first lightweight dynamic detection module corresponds to the first damage type, the second lightweight dynamic detection module corresponds to the second damage type, and so on. When the first lightweight dynamic detection module recognizes the pooling feature map and outputs 1.0, it can be judged that the picture to be detected belongs to the first damage type.

[0156] The training process of different module structures and parameters can be obtained by importing the pictures of the corresponding damage type into the category detection branch of the damage type. The specific training process can also refer to the training process of the feature extraction module shown in the Figure 2 The specific training process of the feature extraction module will not be described here.

[0157] As an embodiment of the present application, the extraction of the scale perception weight of each pooling feature map and the mapping of the scale perception weight to the pooling feature map to obtain the first fusion feature map further comprises:

[0158] The pooling feature maps are averaged to obtain an average pooling feature map.

[0159] The average pooling feature map is subjected to linear convolution processing and nonlinear transformation to obtain the scale perception weight of each pooling feature map.

[0160] The scale perception weight of each pooling feature map is multiplied with the pooling feature map to obtain the first fusion feature map.

[0161] In this step, the dimension of the pooling feature map is determined according to the scale division feature of the feature extraction module, the height and width of each initial feature map in the feature extraction module, and the channel number of the initial feature map. Specifically, the dimension can be determined by the following formula: dimension = L x S x C, wherein L is the number of initial feature maps F output by the feature pyramid; S = HW, H is the height of the initial feature map, W is the width of the initial feature map, and C is the channel number of the initial feature map F.

[0162] In order to better preserve the features of the overall data of the pooling feature map and highlight the information between the damage and the background, the pooling feature map is processed by the average pooling method in the present application.

[0163] For example, according to the formula An average pooling feature map can be obtained, i.e., performing an average pooling operation on the pooled features according to the number of channels of the initial feature map, the height of the initial feature map, and the width of the initial feature map.

[0164] In order to ensure that the data of the average pooling feature map is linear, or in order to reduce or increase the number of channels to reduce or increase the dimension, a method of performing convolution with a 1x1 convolution kernel is used in this paper to reduce the dimension, so as to not change the width and height of the average pooling feature map, so as to ensure the features of the pooled feature map. Specifically, it can be represented by the formula , where f is a 1x1 convolution kernel.

[0165] After reducing the number of channels, in order to ensure that the pooled feature map does not appear matrix multiplication through the fully connected layer, and to avoid useless convolution, the embodiments of this paper can also perform nonlinear transformation after the 1x1 convolution kernel, which specifically includes: calculating the scale perception weight of each pooled feature map using the following formula:

[0166]

[0167] , where is the average pooling feature map, F is the feature map, S is the pixel of the feature map, S is determined according to the height and width of the feature map, C is the number of channels of the feature map, f is a linear convolution function, and σ is an activation function.

[0168] In this paper, the activation function

[0169] where the scale perception weight represents the pooled feature map of different sizes, and the weight of the scale of all features in the to-be-detected picture.

[0170] After obtaining the scale perception weight, the scale perception weight needs to be multiplied with the pooled feature map, specifically the formula , where L (F) is the first fusion feature map.

[0171] As an embodiment of this paper, the step of extracting the spatial perception weight of the first fusion feature map and mapping it into the first fusion feature map to obtain a second fusion feature map further includes:

[0172] Performing multi-kernel linear convolution and deformable convolution operations on the first fusion feature map to obtain spatial perception weights.

[0173] Performing a product operation on the first fusion feature map and the spatial perception weight to obtain the second fusion feature map.

[0174] In this step, the process of multi-kernel linear convolution can be represented by the following formula wherein ω l,k is a model parameter, p k is a feature point in the first fused feature map, and the model parameter is taken as a convolution kernel to implement linear convolution.

[0175] While completing the linear convolution, a bias amount of the feature point can be added to implement variable convolution operation, so as to improve the receptive field, which can be expressed by the following formula: wherein Δp k and Δm k is a bias amount, which can be preset or temporarily changed according to the distribution of the feature points.

[0176] The multi-kernel linear convolution and the variable convolution are fused, and the spatial perception weight can be obtained according to the formula

[0177] The spatial perception weight is multiplied with the first fused feature map to obtain a second fused feature map, and the specific formula is It should be noted that F in the formula is the first fused feature map.

[0178] In summary, the second fused feature map W(F) can be obtained by the formula W(F) = π S (π L (F)), wherein F is the pooled feature map, and W(F) is the second fused feature map.

[0179] As an embodiment of the present application, the category detection module further comprises a region detection branch, the region detection branch comprising a second lightweight dynamic detection module and a regression module, which are trained by using historical pictures of all categories of rail damage;

[0180] The scale features of each pooled feature map are extracted by the second lightweight dynamic detection module and mapped to the pooled feature map to obtain a third fused feature map, and the spatial perception weight of the third fused feature map is extracted and mapped to the third fused feature map to obtain a fourth fused feature map.

[0181] The fourth fused feature map is identified by the region module to determine the damage position in the picture to be detected.

[0182] In this step, the process of obtaining the fourth fused feature map from the pooled feature map is the same as the process of obtaining the second fused feature map from the pooled feature map described above, and will not be repeated here.

[0183] ​When the fourth fusion feature map is obtained, the regression module can regress a detection frame based on the fourth fusion feature map. For example, the fourth fusion feature map is the initial picture. Then, according to the size of the detection frame and the position of the detection frame, a detection frame with a determined position can be added to the fourth fusion feature map. The regression module determines the width and height of the detection frame, the horizontal coordinate of the center point of the detection frame, and the vertical coordinate of the center point of the detection frame.

[0184] The detection frame corresponds to the damage position. According to the parameters of the detection frame of the fourth fusion feature map and the scale of the fourth fusion feature map corresponding to the to-be-detected picture, the damage position in the to-be-detected picture can be obtained.

[0185] As shown in Figure 4 A schematic diagram of a steel rail damage detection device is shown, which includes:

[0186] The acquisition unit 401 is configured to acquire a to-be-detected picture.

[0187] The damage category determination unit 402 is configured to input the to-be-detected picture into a pre-trained steel rail damage detection model to obtain a damage category.

[0188] The steel rail damage detection model includes a preprocessing module and a category detection module.

[0189] The preprocessing module is configured to pre-process the to-be-detected picture to obtain a plurality of scale pooling feature maps.

[0190] The category detection module is configured to extract scale perception weights of each pooling feature map and map them to the pooling feature map to obtain a first fusion feature map, extract spatial perception weights of the first fusion feature map and map them to the first fusion feature map to obtain a second fusion feature map, and identify the second fusion feature map to obtain the damage category. Through the above device, each pooling feature map is first subjected to scale perception and then subjected to spatial perception, so that a larger and more flexible convolution kernel receptive field can be obtained under the condition of the same size convolution kernel, the accuracy of detecting the type of steel rail damage is improved, and the false detection rate is reduced.

[0191] As shown in Figure 5 A steel rail damage model topology of a steel rail damage detection device is shown, which includes, as an embodiment herein:

[0192] The preprocessing module 501 is configured to pre-process the to-be-detected picture to obtain a plurality of scale pooling feature maps.

[0193] The category detection module 502 includes a plurality of category detection branches 5021 and a region detection branch 5022.

[0194] The category detection branch 5021 is configured to extract scale-aware weights of each pooled feature map and map the scale-aware weights to the pooled feature map to obtain a first fused feature map, extract spatial-aware weights of the first fused feature map and map the spatial-aware weights to the first fused feature map to obtain a second fused feature map, and identify the second fused feature map to obtain the damage category.

[0195] The region detection branch 5022 is configured to extract scale features of each pooled feature map and map the scale features to the pooled feature map to obtain a third fused feature map, extract spatial-aware weights of the third fused feature map and map the spatial-aware weights to the third fused feature map to obtain a fourth fused feature map.

[0196] The fourth fused feature map is identified to determine the damage position in the to-be-detected picture.

[0197] The pre-processing module 501 includes a feature extraction module 2, a pooling module 3, and a scale feature pooling module 4.

[0198] In the category detection branch 5021, a first lightweight dynamic detection module 50211 and a binary classification module 50212 are included.

[0199] The first lightweight dynamic detection module 50211 is configured to extract scale-aware weights of each pooled feature map and map the scale-aware weights to the pooled feature map to obtain a first fused feature map, extract spatial-aware weights of the first fused feature map and map the spatial-aware weights to the first fused feature map to obtain a second fused feature map.

[0200] The binary classification module 50212 is configured to identify the second fused feature map to obtain the damage category.

[0201] In the region detection branch 5022, a second lightweight dynamic detection module 50221 and a regression module 50222 are included.

[0202] The second lightweight dynamic detection module 50221 is configured to extract scale features of each pooled feature map and map the scale features to the pooled feature map to obtain a third fused feature map, extract spatial-aware weights of the third fused feature map and map the spatial-aware weights to the third fused feature map to obtain a fourth fused feature map.

[0203] The regression module 50222 is configured to identify the fourth fused feature map to determine the damage position in the to-be-detected picture.

[0204] As shown in FIG. 1, a steel rail damage detection method includes the following steps. Figure 6 The data processing flowchart of the steel rail damage detection method includes the following steps.

[0205] In step 61, a to-be-detected picture 1 is obtained.

[0206] Step 62: Import the image to be detected into the feature extraction module 2 in the rail damage detection model. The feature extraction module 2 extracts feature maps of several scales according to the scale.

[0207] Step 63: Import the feature maps of each scale into the pooling module 3 in the rail damage detection model. The pooling module 3 performs region extraction and region pooling processing to obtain the regional feature maps of each scale.

[0208] Step 64: import the regional feature maps of each scale into the scale feature pooling module 4 in the rail damage detection model. The scale feature pooling module 4 performs scale feature pooling processing to obtain multiple pooling feature maps.

[0209] Step 65: Import the pooled feature map into the category detection module 5 in the rail damage detection model to determine the damage category and damage location of the image to be detected.

[0210] Among them, the network architecture of the feature extraction module 2, the pooling module 3 and the scale feature pooling module 4 can refer to the existing technology, and this article does not limit this.

[0211] Among them, the network architecture of the feature extraction module 2, the pooling module 3 and the scale feature pooling module 4 can refer to the existing technology, and this article does not limit this.

[0212] The category detection module 5 includes multiple category detection branches and one region detection branch. Each category detection branch includes a lightweight dynamic detection module and a binary classification module, and each category detection branch is trained using historical images of a specific category of rail damage. The region detection branch includes a lightweight dynamic detection module and a region module, and is trained using historical images of all categories of rail damage.

[0213] Specifically, the lightweight dynamic detection module is used to extract scale features from each pooled feature map and map them to a pooled feature map to obtain a first fused feature map. The spatial perception weights of the first fused feature map are extracted and mapped to the first fused feature map to obtain a second fused feature map. The binary classification module includes a fully connected layer and a softmax layer to output a yes or no result for the corresponding category. The regional module includes a fully connected layer and a sigmoid layer to output the coordinates of the damage location.

[0214] like Figure 7 The data processing flow chart of the category detection module of a rail damage detection method shown in FIG. 6 includes:

[0215] Step 651: Import the pooled feature map into the first lightweight dynamic detection module 50211 and the second lightweight dynamic detection module 50221 of each branch to obtain the second fused feature map and the fourth fused feature map respectively.

[0216] Step 652, the second fusion feature map is introduced into the binary classification module 50212 respectively, and the damage category of the to-be-detected picture is obtained.

[0217] Step 653, the fourth fusion feature map is introduced into the regression module 50222, and the damage position of the to-be-detected picture is obtained.

[0218] The lightweight dynamic detection modules in each branch described herein are the same in structure but different in parameters, such as Figure 8 A data processing flowchart of a lightweight dynamic detection module of a steel rail damage detection method is shown in FIG. 6. Step 651 includes:

[0219] Step 6511, the pooling feature map is introduced into the average pooling network 511 for average pooling to obtain an average pooling feature map.

[0220] Step 6512, the average pooling feature map is introduced into the linear convolution network 512 for 1x1 convolution.

[0221] Step 6513, the average pooling feature map is introduced into the learning factor network 513 to be multiplied by a learning factor.

[0222] Step 6514, the average pooling feature map is introduced into the first nonlinear transformation network 514 for nonlinear transformation to obtain scale perception weight.

[0223] Step 6515, the scale perception weight and the pooling feature map are introduced into the first product network 515 for multiplication to obtain a first fusion feature map.

[0224] Step 6516, the first fusion feature map is introduced into the constant factor network 516 to be multiplied by a constant factor.

[0225] Step 6517, the first fusion feature map is introduced into the symmetric convolution network 517 for 3x3 convolution.

[0226] Step 6518, the first fusion feature map is introduced into the second nonlinear transformation network 518 for linear convolution.

[0227] Step 6519, the first fusion feature map is introduced into the deformable convolution network 519 for deformable convolution.

[0228] Step 6520, the linear convolution result and the spatial perception weight obtained by the deformable convolution are introduced into the second product network 5110 with the first fusion feature map for multiplication operation to obtain a second fusion feature map.

[0229] As shown in FIG. 6, the lightweight dynamic detection module of the steel rail damage detection method includes: Figure 9As shown, a computer device 902 that provides an embodiment herein can include one or more processors 904, such as one or more central processing units (CPUs), each of which can implement one or more hardware threads. The computer device 902 can also include any memory 906 for storing any kind of information, such as code, settings, data, etc. Without limitation, for example, the memory 906 can include any one or combination of: any type of RAM, any type of ROM, a flash memory device, a hard disk, an optical disk, etc. More generally, any memory can use any technology for storing information. Further, any memory can provide volatile or non-volatile retention of information. Further, any memory can represent a fixed or removable component of the computer device 902. In one case, the computer device 902 can perform any operation of the associated instructions when executed by the processor 904, which are stored in any memory or combination of memories. The computer device 902 also includes one or more drive mechanisms 908, such as a hard disk drive mechanism, an optical disk drive mechanism, etc., for interacting with any memory.

[0230] The computer device 902 can also include an input / output module 910 (I / O) for receiving various inputs (via input devices 912) and for providing various outputs (via output devices 914). One particular output mechanism can include a presentation device 916 and an associated graphical user interface (GUI) 918. In other embodiments, the input / output module 910 (I / O), the input devices 912, and the output devices 914 can also not be included, just as a computer device in a network. The computer device 902 can also include one or more network interfaces 920 for exchanging data with other devices via one or more communication links 922. One or more communication buses 924 couple the above-described components together.

[0231] The communication links 922 can be implemented in any manner, such as through a local area network, a wide area network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. The communication links 922 can include any combination of hardwired links, wireless links, routers, gateway functionality, name servers, etc., governed by any protocol or combination of protocols.

[0232] Corresponding to the method in Figures 2-3 The embodiments herein also provide a computer readable storage medium, having stored thereon a computer program, which, when executed by a processor, performs the steps of the above method.

[0233] The embodiments herein also provide a computer readable instruction, wherein when a processor executes the instruction, the program therein causes the processor to perform the steps of the above method. Figures 2-3The method shown.

[0234] It should be understood that the size of the sequence number of each process described above in various embodiments herein does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments herein.

[0235] It should also be understood that in the embodiments herein, the term "and / or" is only a description of the association relationship of the associated objects, which means that there can be three relationships. For example, A and / or B can represent three cases of A alone, A and B together, and B alone. In addition, the character " / " in this paper generally represents that the front and rear associated objects are a "or" relationship.

[0236] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been described in the above description in general terms. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this paper.

[0237] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0238] In several embodiments provided herein, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displays or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can also be electrical, mechanical or other forms of connection.

[0239] The units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. According to actual needs, some or all of the units can be selected to achieve the purpose of the embodiments herein.

[0240] In addition, each functional unit in each embodiment herein can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0241] When the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions herein, essentially or in the form of a contribution to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment herein. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, and various other media that can store program codes.

[0242] The principles and implementation manners of the present application are described herein by using specific embodiments, and the above description of the embodiments is only used to help understand the methods and core ideas thereof; meanwhile, for those skilled in the art, according to the ideas herein, the specific implementation manners and application ranges can be changed, and the above description of the specification should not be understood as a limitation of the present application.

Claims

1. A method of rail damage detection, characterized by, The method comprises the following steps: acquiring a to-be-detected picture; inputting the to-be-detected picture into a pre-trained rail damage detection model to obtain a damage category; wherein the rail damage detection model comprises a preprocessing module and a category detection module; the preprocessing module is used for preprocessing the to-be-detected picture to obtain a plurality of scale pooling feature maps; the category detection module is used for extracting scale perception weights of each pooling feature map and mapping them to the pooling feature map to obtain a first fusion feature map, extracting spatial perception weights of the first fusion feature map and mapping them to the first fusion feature map to obtain a second fusion feature map, and identifying the second fusion feature map to obtain the damage category; wherein the extraction of the scale perception weights of each pooling feature map and the mapping of the scale perception weights to the pooling feature map to obtain the first fusion feature map further comprises: averaging pooling each pooling feature map to obtain an average pooling feature map; performing linear convolution processing and nonlinear transformation on the average pooling feature map to obtain the scale perception weights of each pooling feature map; performing product processing on the scale perception weights of each pooling feature map and the pooling feature map to obtain the first fusion feature map; the extraction of the spatial perception weights of the first fusion feature map and the mapping of the spatial perception weights into the first fusion feature map to obtain the second fusion feature map further comprises: performing multi-kernel linear convolution and deformable convolution operations on the first fusion feature map to obtain spatial perception weights; performing product operation on the first fusion feature map and the spatial perception weights to obtain the second fusion feature map.

2. The method of rail flaw detection according to claim 1, characterized in that, The preprocessing of the to-be-detected picture to obtain a plurality of scale pooling feature maps further comprises: extracting features of the to-be-detected picture to obtain an initial feature map of the to-be-detected picture; performing down-sampling or up-sampling on the initial feature map according to a preset scale to obtain multi-scale feature maps; performing region extraction and region pooling processing on each scale feature map to obtain region feature maps of each scale; performing scale feature pooling processing on the region feature maps of each scale to obtain a plurality of pooling feature maps.

3. The method of rail flaw detection according to claim 1, wherein The linear convolution processing and nonlinear transformation on the average pooling feature map to obtain the scale perception weights of each pooling feature map comprise calculating the scale perception weights of each pooling feature map by using the following formula: wherein, is the average value of a pooling feature map, F is the pooling feature map, S is determined according to the height and width of the pooling feature map, C is the number of channels of the feature map, f is a linear convolution function, and σ is an activation function.

4. The method of rail damage detection according to claim 1, wherein, The category detection module comprises a plurality of category detection branches, each category detection branch comprises a first lightweight dynamic detection module and a binary classification module, and each category detection branch is trained by using a historical picture of a kind of category rail damage; the first lightweight dynamic detection module is used for extracting scale features of each pooling feature map and mapping them to the pooling feature map to obtain a first fusion feature map, extracting spatial perception weights of the first fusion feature map and mapping them to the first fusion feature map to obtain a second fusion feature map, and the binary classification module is used for identifying the second fusion feature map to obtain the damage category.

5. The method of rail damage detection according to claim 1, wherein, The category detection module further comprises a region detection branch, the region detection branch comprises a second lightweight dynamic detection module and a regression module, and is trained by using historical pictures of all category rail damages; The scale features of each pooling feature map are extracted by using a second lightweight dynamic detection module, and the scale features are mapped to the pooling feature map to obtain a third fusion feature map. Spatial perception weights of the third fusion feature map are extracted and mapped to the third fusion feature map to obtain a fourth fusion feature map. The fourth fusion feature map is identified by using a regression module to determine the damage position in the picture to be detected.

6. A rail flaw detection apparatus characterized by comprising: The method comprises the following steps: An acquisition unit is configured to acquire a picture to be detected. An injury category determination unit is configured to input the picture to be detected into a pre-trained steel rail damage detection model to obtain an injury category. The steel rail damage detection model comprises a preprocessing module and a category detection module. The preprocessing module is configured to preprocess the picture to be detected to obtain a plurality of scale pooling feature maps. The category detection module is configured to extract scale perception weights of each pooling feature map and map the scale perception weights to the pooling feature map to obtain a first fusion feature map. Spatial perception weights of the first fusion feature map are extracted and mapped to the first fusion feature map to obtain a second fusion feature map. The second fusion feature map is identified to obtain the injury category. The extraction of the scale perception weights of each pooling feature map and the mapping of the scale perception weights to the pooling feature map to obtain the first fusion feature map further comprise the following steps: Each pooling feature map is averaged to obtain an average pooling feature map. The average pooling feature map is subjected to linear convolution processing and nonlinear transformation to obtain the scale perception weights of each pooling feature map. The scale perception weights of each pooling feature map are multiplied with the pooling feature map to obtain the first fusion feature map. The extraction of the spatial perception weights of the first fusion feature map and the mapping of the spatial perception weights to the first fusion feature map to obtain the second fusion feature map further comprise the following steps: The first fusion feature map is subjected to multi-kernel linear convolution and deformable convolution operations to obtain spatial perception weights. The first fusion feature map and the spatial perception weights are subjected to multiplication operation to obtain the second fusion feature map.

7. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steel rail damage detection method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steel rail damage detection method according to any one of claims 1-5.

Images