Brake equipment wear detection method and device, electronic equipment and storage medium

Abstract

The invention provides a brake equipment wear detection method and device, electronic equipment and a storage medium, and relates to the technical field of image processing, and the method comprises the steps: obtaining a to-be-corrected image of brake equipment; inputting the to-be-corrected image into a geometric correction module in a wear detection model to obtain a corrected image output by the geometric correction module; the wear detection model is obtained by training based on a sample image to be corrected and label data; and inputting the corrected image into a wear detection module in the wear detection model to obtain the scores of the wear area and the wear degree of the brake equipment output by the wear detection module. The to-be-corrected image is corrected through the geometric correction module in the wear detection model, the quality of the corrected image can be improved, then the corrected image is detected through the wear detection module, accurate detection of the score of the wear area and the wear degree of the brake equipment can be achieved, and therefore the wear detection precision of the brake equipment is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a brake equipment wear detection method, device, electronic equipment and storage medium. Background Art

[0002] Braking equipment is an important equipment in rail transit. Wear is one of the main reasons for its aging. Rapid detection of wear is crucial for the safe operation of trains. The pantograph-catenary relationship is one of the important basic relationships in rail transit. Therefore, rapid and effective detection of brake equipment wear has important theoretical research significance and engineering application value for the development of urban rail transit.

[0003] In the prior art, wear detection can only be performed on specific types of brake equipment, such as disc brake equipment, and is judged by the distance between a non-contact detection sensor and the brake disc. The sensor measurement is not accurate enough and the judgment standard is relatively simple, resulting in relatively low wear detection accuracy for the brake equipment. Summary of the invention

[0004] The present invention provides a brake equipment wear detection method, device, electronic equipment and storage medium, which are used to solve the problem of low wear detection accuracy of brake equipment.

[0005] The present invention provides a brake equipment wear detection method, comprising: Acquire an image of the braking device to be corrected; Inputting the image to be corrected into a geometric correction module in a wear detection model to obtain a corrected image output by the geometric correction module; the wear detection model is trained based on sample images to be corrected and label data; The corrected image is input into the wear detection module in the wear detection model to obtain the wear area and the wear degree score of the brake device output by the wear detection module.

[0006] According to a brake equipment wear detection method provided by the present invention, the geometric correction module includes a preprocessing unit, a self-attention encoder, a self-attention decoder and a conversion unit; The step of inputting the image to be corrected into a geometric correction module in a wear detection model to obtain a corrected image output by the geometric correction module comprises: Inputting the downsampled image obtained by downsampling the image to be corrected into the preprocessing unit to obtain the position perception feature output by the preprocessing unit; Inputting the position-aware feature into the self-attention encoder to obtain an encoded feature output by the self-attention encoder; Inputting the encoded feature into the self-attention decoder to obtain a first decoded feature output by the self-attention decoder; Inputting the second decoding feature obtained after converting the first decoding feature into the conversion unit to obtain a coordinate mapping matrix output by the conversion unit; The rectified image is determined based on the coordinate mapping matrix and the downsampled image.

[0007] According to a brake equipment wear detection method provided by the present invention, the preprocessing unit includes a background elimination module and at least one residual network; The step of inputting the downsampled image obtained by downsampling the image to be corrected into the preprocessing unit to obtain the position-aware features output by the preprocessing unit includes: Inputting the down-sampled image obtained by down-sampling the image to be corrected into the background removal module to obtain a foreground image output by the background removal module; Inputting the foreground image into the first residual network of the consecutive residual networks to obtain the residual features output by the last residual network; The position perception feature is determined based on the two-dimensional matrix feature obtained after the residual feature is converted and the two-dimensional position coding feature corresponding to the image to be corrected.

[0008] According to a brake equipment wear detection method provided by the present invention, the conversion unit includes a bilinear interpolation module and at least one convolution layer; The step of inputting the second decoding feature obtained after converting the first decoding feature into the conversion unit to obtain a coordinate mapping matrix output by the conversion unit includes: Inputting the second decoded feature obtained after converting the first decoded feature into the bilinear interpolation module to obtain the up-sampled feature output by the bilinear interpolation module; The up-sampled features are input to the first convolution layer in each of the consecutive convolution layers to obtain the coordinate mapping matrix output by the last convolution layer.

[0009] According to a brake equipment wear detection method provided by the present invention, the wear detection module includes a backbone network and a convolution block; The step of inputting the corrected image to a wear detection module in the wear detection model to obtain the wear area and the wear degree score of the brake device output by the wear detection module comprises: Inputting the rectified image into the backbone network to obtain a plurality of phantom feature maps output by the backbone network; The multiple ghost feature maps are input into the convolution block, and the wear area and the wear degree score of the brake device are obtained as outputs of the convolution block.

[0010] The present invention also provides a brake equipment wear detection device, comprising: An acquisition module, used for acquiring an image to be corrected of the braking device; An image correction module, used for inputting the image to be corrected into a geometric correction module in a wear detection model to obtain a corrected image output by the geometric correction module; the wear detection model is obtained by training based on sample images to be corrected and label data; The detection module inputs the corrected image into the wear detection module in the wear detection model to obtain the wear area and wear degree score of the brake device output by the wear detection module.

[0011] According to a brake equipment wear detection device provided by the present invention, the geometric correction module includes a preprocessing unit, a self-attention encoder, a self-attention decoder and a conversion unit; The image correction module is specifically used for: Inputting the downsampled image obtained by downsampling the image to be corrected into the preprocessing unit to obtain the position perception feature output by the preprocessing unit; Inputting the position-aware feature into the self-attention encoder to obtain an encoded feature output by the self-attention encoder; Inputting the encoded feature into the self-attention decoder to obtain a first decoded feature output by the self-attention decoder; Inputting the second decoding feature obtained after converting the first decoding feature into the conversion unit to obtain a coordinate mapping matrix output by the conversion unit; The rectified image is determined based on the coordinate mapping matrix and the downsampled image.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a brake device wear detection method as described above is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the brake device wear detection method as described in any one of the above is implemented.

[0014] The brake equipment wear detection method, device, electronic device and storage medium provided by the present invention obtain the image to be corrected of the brake equipment; input the image to be corrected into the geometric correction module in the wear detection model to obtain the corrected image output by the geometric correction module; the wear detection model is trained based on the sample image to be corrected and the label data; input the corrected image into the wear detection module in the wear detection model to obtain the wear area and the wear degree score of the brake equipment output by the wear detection module. The image to be corrected is corrected by the geometric correction module in the wear detection model, so that the quality of the corrected image can be improved, and then the corrected image is detected by the wear detection module, so that the wear area and the wear degree score of the brake equipment can be accurately detected, thereby improving the accuracy of wear detection of the brake equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 It is one of the flow charts of the brake equipment wear detection method provided by the present invention.

[0017] Figure 2 This is the second flow chart of the brake equipment wear detection method provided by the present invention.

[0018] Figure 3 It is a structural schematic diagram of the wear detection model provided by the present invention.

[0019] Figure 4 It is a structural schematic diagram of the embedded device for brake equipment wear detection provided by the present invention.

[0020] Figure 5 It is a structural schematic diagram of the brake equipment wear detection device provided by the present invention.

[0021] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] In order to facilitate a clearer understanding of the embodiments of the present application, the relevant knowledge is first introduced as follows.

[0024] 1. Existing technical solutions

[0025] Prior art solution 1: a disc brake device including a brake lining wear detection device.

[0026] The scheme relates to a disc brake device of a vehicle, which includes a brake disc and at least two brake linings that are in frictional contact with the brake disc in a brake application position, the brake linings are respectively supported by a brake lining bracket, and the brake linings are supported on a brake caliper or on a caliper rod of the brake caliper; and a device for determining the brake lining thickness of at least one of the brake linings, the device including an electronic evaluation device and a sensor device, the sensor device detects the corresponding brake lining thickness of at least one brake lining and generates an output signal representing the corresponding size of the brake lining thickness by cooperating with the electronic evaluation device, the sensor device including at least one non-contact sensor, so that the sensor detects the distance between the non-contact sensor and the side of the brake disc pointing toward the non-contact sensor in a non-contact manner at least in the brake application position.

[0027] However, this technical solution mainly targets the wear of disc brake equipment and judges the distance between the non-contact detection sensor and the brake disc. The judgment standard is relatively simple and the sensor measurement is not accurate enough, resulting in low wear detection accuracy.

[0028] Prior art solution 2: Combined detection of the wear of brake system components, especially brake system components of motor vehicles.

[0029] The solution relates to a sensor assembly for detecting the wear of brake system components, in particular for a brake system of a motor vehicle. The brake system has at least one brake disc and a brake lining. The sensor assembly detects an image of the brake disc, wherein the wear state of the brake disc is detected by the sensor assembly and can be displayed without the need to view the brake disc.

[0030] However, this technical solution is mainly aimed at the wear detection of brake discs in brake equipment, and has a single function. At the same time, due to the fast running time of the train, sensor collection is more difficult, resulting in low wear detection accuracy.

[0031] 2. Related key terms.

[0032] 1. Braking equipment.

[0033] Braking equipment is an important equipment in the railway system. Its main function is to slow down or stop the train during operation. Braking equipment usually includes air brakes, electrical brakes and mechanical brakes. Among them, air brakes achieve braking by changing the friction between the train wheels and the track, electrical brakes use the reverse electromagnetic force of the motor to achieve braking, and mechanical brakes mainly achieve braking through mechanical structures, such as brake drums or brake discs.

[0034] 2. Wear and tear.

[0035] Wear refers to the phenomenon that the surface or interior of an object is lost or reduced due to friction and impact when it is in motion or in use. Wear is one of the main causes of brake equipment aging and requires regular inspection and maintenance.

[0036] 3. Embedded devices.

[0037] An embedded device is an electronic device designed to perform certain specific tasks. It is usually embedded in a larger system and is responsible for running operating systems and applications. Embedded devices are characterized by small size, low power consumption and stable performance, and are widely used in industrial control and consumer electronics.

[0038] Combine the following Figure 1-Figure 4 The brake equipment wear detection method of the present invention is described.

[0039] Figure 1 FIG. 1 is one of the flow charts of the brake equipment wear detection method provided by the present invention, such as Figure 1 As shown, the method includes the following steps 101-102.

[0040] Step 101: Acquire an image to be corrected of a braking device.

[0041] It should be noted that the brake equipment wear detection method provided by the present invention can be applied to the scenario of brake equipment wear detection in the railway system, and the executor of the method can be a brake equipment wear detection device, such as an embedded device, an electronic device, or a control module in the brake equipment wear detection device for executing the brake equipment wear detection method.

[0042] Specifically, the image to be corrected of the braking equipment can be collected by an external visual device on the rail train.

[0043] Step 102: input the image to be corrected into a geometric correction module in a wear detection model to obtain a corrected image output by the geometric correction module; the wear detection model is trained based on sample images to be corrected and label data.

[0044] It should be noted that the images of the brake equipment can be collected by external visual equipment on rail trains, which is of great significance for detecting the wear of the brake equipment. However, the images of the brake equipment obtained by the visual acquisition equipment during the operation of the rail train will be distorted. In order to solve the distortion problem, many intelligent methods try to design complex models, but these complex models have high requirements for hardware configuration, high cost and difficult maintenance. For embedded devices that need to respond quickly, the accuracy of brake equipment wear detection is very low.

[0045] Based on the above problems, the wear detection model provided by the present invention includes a geometric correction module and a wear detection module, wherein the geometric correction module is used to correct the distortion of the image to be corrected to obtain a corrected image; the wear detection module is used to detect the wear area and the wear degree score of the brake equipment in the corrected image. The wear detection model is trained based on the sample image to be corrected and the label data to obtain a trained wear detection model.

[0046] After the wear detection model is trained, the image to be corrected Input to the geometric correction module to get the corrected image output by the geometric correction module ; Wherein, H represents the height of the image to be corrected, and W represents the width of the image to be corrected.

[0047] Step 103: input the corrected image to the wear detection module in the wear detection model to obtain the wear area and wear degree score of the brake device output by the wear detection module.

[0048] Specifically, the corrected image Input to the wear detection module, the wear area and wear degree score P of the brake equipment output by the wear detection module can be obtained, wherein the wear area is detected by using the detection frame express.

[0049] The brake equipment wear detection method provided by the present invention obtains the image to be corrected of the brake equipment; inputs the image to be corrected into the geometric correction module in the wear detection model to obtain the corrected image output by the geometric correction module; the wear detection model is trained based on the sample image to be corrected and the label data; inputs the corrected image into the wear detection module in the wear detection model to obtain the wear area and the wear degree score of the brake equipment output by the wear detection module. The image to be corrected is corrected by the geometric correction module in the wear detection model, so that the quality of the corrected image can be improved, and then the corrected image is detected by the wear detection module, so that the wear area and the wear degree score of the brake equipment can be accurately detected, thereby improving the accuracy of wear detection of the brake equipment.

[0050] Optionally, the geometric correction module includes a preprocessing unit, a self-attention encoder, a self-attention decoder and a conversion unit; the step of inputting the image to be corrected into the geometric correction module in the wear detection model to obtain a corrected image output by the geometric correction module includes: The downsampled image obtained by downsampling the image to be corrected is input into the preprocessing unit to obtain the position-aware features output by the preprocessing unit; the position-aware features are input into the self-attention encoder to obtain the encoded features output by the self-attention encoder; the encoded features are input into the self-attention decoder to obtain the first decoded features output by the self-attention decoder; the second decoded features obtained after converting the first decoded features are input into the conversion unit to obtain the coordinate mapping matrix output by the conversion unit; the corrected image is determined based on the coordinate mapping matrix and the downsampled image.

[0051] Specifically, the image to be corrected Downsampled image obtained by downsampling Input to the preprocessing unit, and you can get the position-aware features output by the preprocessing unit. ; Position-aware features Input to the self-attention encoder to obtain the encoded features output by the self-attention encoder .

[0052] It should be noted that the self-attention encoder has a total of k=6 self-attention encoding layers, each of which includes a multi-head attention (MHA) and a feed forward neural network (FFN). The encoding process is expressed by the following formulas (1)-(4).

[0053] (1)

[0054] (2)

[0055] (3)

[0056] (4)

[0057] in, , i represents the i-th self-attention encoding layer, and the value range of i is 1 to 6. , and Both represent the parameter matrices that can be learned, , represents the number of multi-head attention mechanism heads, Indicates the jth The corresponding location-aware features, represents the sequence corresponding to the position-aware features, represents the output of the i-th self-attention encoding layer. When i=6, Represents the encoded features of the self-attention encoder output .

[0058] The calculation of multi-head attention is expressed as shown in formulas (5)-(6).

[0059] (5)

[0060] (6)

[0061] Among them, Q represents the query vector (Query), K represents the key vector (Key), and V represents the value vector (Value). represents the dimension size of K, and h represents the h-th attention head.

[0062] The attention weight is obtained by calculating the similarity between the query vector Q and the key vector K, and then the final attention value is obtained on the value vector V by the attention weight. Then, the value returned by each attention head is After the concatenation operation (Concat), the attention of each attention encoding layer is obtained.

[0063] Encoding features Input to the self-attention decoder, and you can get the first decoding feature output by the self-attention decoder .

[0064] It should be noted that the self-attention decoder has a total of k = 6 self-attention decoding layers, each of which includes multi-head attention and feedforward neural networks. The encoding process is expressed by the following formulas (7)-(12).

[0065] (7)

[0066] (8)

[0067] (9)

[0068] (10)

[0069] (11)

[0070] (12)

[0071] in, Indicates the jth The corresponding two-dimensional position encoding feature, represents the sequence corresponding to the two-dimensional position encoding feature, represents the output of the i-th self-attention decoding layer. When i=6, Represents the first decoded feature output by the self-attention decoder .

[0072] The first decoding feature The second decoding feature obtained after linear transformation Input to the conversion unit, and you can get the coordinate mapping matrix output by the conversion unit. ; Based on the coordinate mapping matrix and downsampled image ,use The corrected image can be determined .

[0073] Optionally, the preprocessing unit includes a background removal module and at least one residual network; the down-sampled image obtained by downsampling the image to be corrected is input into the preprocessing unit to obtain the position-aware features output by the preprocessing unit, including: The downsampled image obtained by downsampling the image to be corrected is input into the background removal module to obtain a foreground image output by the background removal module; the foreground image is input into the first residual network in each consecutive residual network to obtain the residual features output by the last residual network; the position perception features are determined based on the two-dimensional matrix features obtained after the residual features are converted and the two-dimensional position coding features corresponding to the image to be corrected.

[0074] Specifically, the downsampled image obtained by downsampling the image to be corrected Input to the background removal module, and you can get the foreground image output by the background removal module. ; The foreground image By inputting the first residual network in each consecutive residual network, the residual features output by the last residual network can be obtained. ; Among them, the number of residual networks can be 3. After each residual network, the feature map is downsampled once. After three consecutive residual networks, the residual feature is finally obtained. Based on the residual feature The two-dimensional matrix features obtained after conversion The two-dimensional position encoding feature corresponding to the image to be corrected , the location-aware features can be determined .

[0075] Optionally, determining the position perception feature based on the two-dimensional matrix feature obtained after the residual feature conversion and the two-dimensional position coding feature corresponding to the image to be corrected includes: The two-dimensional matrix feature obtained after the residual feature is converted and the two-dimensional position coding feature corresponding to the image to be corrected are summed to obtain the position perception feature.

[0076] Specifically, the two-dimensional matrix feature obtained after converting the residual feature The two-dimensional position coding feature corresponding to the image to be corrected is adopted You can get location-aware features .

[0077] Optionally, the conversion unit includes a bilinear interpolation module and at least one convolution layer; the second decoding feature obtained by converting the first decoding feature is input into the conversion unit to obtain a coordinate mapping matrix output by the conversion unit, including: The second decoding feature obtained after converting the first decoding feature is input into the bilinear interpolation module to obtain the up-sampled feature output by the bilinear interpolation module; the up-sampled feature is input into the first convolution layer in each consecutive convolution layer to obtain the coordinate mapping matrix output by the last convolution layer.

[0078] Specifically, the first decoding feature The second decoded feature obtained after conversion Input to the bilinear interpolation module, the bilinear interpolation module is used to decode the second feature By upsampling, we can get the upsampling features output by the bilinear interpolation module. ; Input the upsampled features to the first convolution layer in each consecutive convolution layer, and you can get the coordinate mapping matrix output by the last convolution layer ; Among them, the number of convolutional layers can be 2.

[0079] Optionally, the wear detection module includes a backbone network and a convolution block; the step of inputting the rectified image to the wear detection module in the wear detection model to obtain the wear area and the wear degree score of the brake device output by the wear detection module includes: The rectified image is input into the backbone network to obtain a plurality of phantom feature maps output by the backbone network; the plurality of phantom feature maps are input into the convolution block to obtain the wear area and wear degree score of the brake device output by the convolution block.

[0080] Specifically, the wear detection module does not need to detect multiple types of targets, but only needs to detect the wear area, but the wear detection module also needs to quantify the degree of wear. The backbone network included in the wear detection module can be a Ghost network or another backbone network.

[0081] The corrected image By inputting into the backbone network, multiple phantom feature maps output by the backbone network can be obtained; wherein, the backbone network can obtain multiple phantom feature maps through several linear operations, and the number of parameters of the linear operation is much smaller than the number of parameters of the convolution operation.

[0082] By inputting multiple ghost feature maps into the convolution block, the wear area of ​​the brake equipment output by the convolution block can be obtained. and the wear degree score ; Among them, the convolution block includes 4 The convolutional layer.

[0083] It should be noted that when training the wear detection module, the wear degree scores output by the wear detection module are divided into 500 categories, namely , there is an ordered relationship between classes, that is, The degree of wear is less than The degree of wear, , n is the wear degree of the nth category greater than m. The real wear degree data in the training data set is , the real detection box is , then the loss function L in the wear detection module training process includes and The two parts are expressed by the following formulas (13)-(15) respectively.

[0084] (13)

[0085] (14)

[0086] (15)

[0087] in, Indicates balance and Parameters, The value is 0.5; EMD is expressed by formula (16), IOU represents the intersection-over-union ratio of two wear detection frames, and IOU is expressed by formula (17).

[0088] (16)

[0089] (17)

[0090] in, Represents the cumulative probability value of the predicted wear degree, The probability accumulation value that represents the actual wear degree. CDF refers to the Cumulative Distribution Function. CDF(m) is accumulated from 1 to m. The value of is 2.

[0091] The brake equipment wear detection method provided by the present invention realizes functions such as correction of collected images, detection of wear positions and estimation of wear extent.

[0092] Figure 2 FIG. 2 is a flow chart of the brake equipment wear detection method provided by the present invention. Figure 2 As shown, the method includes steps 201 to 211.

[0093] Step 201: Acquire the image to be corrected of the braking device.

[0094] Step 202: input the down-sampled image obtained by down-sampling the image to be corrected into a background removal module to obtain a foreground image output by the background removal module.

[0095] Step 203: input the foreground image into the first residual network in the consecutive residual networks to obtain the residual features output by the last residual network.

[0096] Step 204: Determine the position perception feature based on the two-dimensional matrix feature obtained after the residual feature conversion and the two-dimensional position coding feature corresponding to the image to be corrected.

[0097] Step 205: Input the position-aware features into the self-attention encoder to obtain the encoded features output by the self-attention encoder.

[0098] Step 206: Input the encoded feature into the self-attention decoder to obtain the first decoded feature output by the self-attention decoder.

[0099] Step 207: input the second decoded feature obtained after converting the first decoded feature into a bilinear interpolation module to obtain an up-sampled feature output by the bilinear interpolation module.

[0100] Step 208: Input the upsampled features to the first convolutional layer in each consecutive convolutional layer to obtain the coordinate mapping matrix output by the last convolutional layer.

[0101] Step 209: Determine a rectified image based on the coordinate mapping matrix and the down-sampled image.

[0102] Step 210: Input the rectified image into the backbone network to obtain a plurality of phantom feature maps output by the backbone network.

[0103] Step 211: Input multiple ghost feature maps into the convolution block to obtain the wear area and wear degree score of the brake equipment output by the convolution block.

[0104] Figure 3 is a schematic diagram of the structure of the wear detection model provided by the present invention, such as Figure 3 As shown, the wear detection model includes a geometric correction module and a wear detection module, wherein the geometric correction module includes a preprocessing unit, a self-attention encoder, a self-attention decoder and a conversion unit, the preprocessing unit includes a background elimination module and at least one residual network, and the conversion unit includes a bilinear interpolation module and at least one convolutional layer; the wear detection module includes a backbone network and a convolutional block.

[0105] The downsampled image obtained by downsampling the image to be corrected is input into the background removal module in the preprocessing unit in the geometric correction module in the wear detection model to obtain the foreground image output by the background removal module; the foreground image is input into the first residual network in each consecutive residual network in the preprocessing unit to obtain the residual features output by the last residual network; based on the two-dimensional matrix features obtained after the residual feature conversion and the two-dimensional position coding features corresponding to the image to be corrected, the position-aware features are determined; the position-aware features are then input into the self-attention encoder to obtain the coding features output by the self-attention encoder; the coding features are input into the self-attention decoder to obtain the first decoding features output by the self-attention decoder; the second decoding features obtained after the first decoding features are converted are then input into the bilinear interpolation module in the conversion unit in the geometric correction module to obtain the upsampled features output by the bilinear interpolation module; the upsampled features are input into the first convolutional layer in each consecutive convolutional layer in the conversion unit to obtain the coordinate mapping matrix output by the last convolutional layer; finally, the corrected image is determined based on the coordinate mapping matrix and the downsampled image.

[0106] The corrected image is input into the backbone network in the wear detection module in the wear detection model to obtain multiple phantom feature maps output by the backbone network; the multiple phantom feature maps are input into the convolution block in the wear detection module to obtain the wear area and wear degree score of the braking equipment output by the convolution block.

[0107] The method provided by the present invention performs deep modeling on the image of the braking equipment and combines the field condition characteristics of the braking equipment image with the global feature extraction capabilities of the self-attention encoder and the self-attention decoder in the geometric correction module to perform geometric correction on the image of the braking equipment, thereby providing a good benchmark for wear detection of the braking equipment, and further realizing the customization of the self-attention encoder and the self-attention decoder for image detection of the braking equipment on embedded devices, reducing the number of parameters of the brake equipment wear detection model, improving the computational efficiency, and at the same time improving the accuracy of wear detection.

[0108] Figure 4 is a schematic diagram of the structure of the embedded device for detecting wear of the brake device provided by the present invention, such as Figure 4 As shown, the embedded device for deploying the rapid detection system includes a camera module deploying the OV7725 series and a wear detection module deployed by the ARM processor of Cortex-A76, wherein the ARM processor is connected to a storage module, and the storage module is connected to a storage read-write interface, which can write the position of the brake device during input and read the wear detection result of the brake device during output. The embedded device is deployed on the train, and the ARM processor starts the camera to continuously take pictures at a certain time interval, and transmits them back to the ARM processor to start the rapid detection system to complete the detection, and the results are stored in the storage module.

[0109] The wear detection model provided by the present invention can be implemented by other detection algorithms, such as a detection algorithm based on a large-scale visual text pre-training model, but the large-scale visual text pre-training model has two defects. First, a large amount of image and text matching data on the Internet is collected during the training process, the data quality is not high, and the coverage of professional field data is very limited. Therefore, the perception and cognitive ability of the large-scale visual text pre-training model for the braking equipment image is seriously insufficient. Even if the pre-training model is fine-tuned by a small number of annotated images of the braking equipment, the number of braking equipment samples is too small, and the number of pre-training model parameters is too large, and the gradient change is not obvious when the pre-training model is fine-tuned. Second, the number of parameters of the large-scale pre-training model is too large, and the interpretability is weak, making it difficult to optimize in an embedded device in a targeted manner, and even more difficult to customize the braking equipment wear detection function in an embedded device.

[0110] The brake equipment wear detection device provided by the present invention is described below. The brake equipment wear detection device described below and the brake equipment wear detection method described above can be referenced to each other.

[0111] Figure 5 Schematic diagram of the structure of the brake equipment wear detection device provided by the present invention. Figure 5 As shown, the brake equipment wear detection device 500 includes: an acquisition module 501, an image correction module 502 and a detection module 503; wherein,

[0112] An acquisition module 501 is used to acquire an image to be corrected of a braking device; An image correction module 502, used for inputting the image to be corrected into a geometric correction module in a wear detection model, and outputting a corrected image by the geometric correction module; the wear detection model is obtained by training based on sample images to be corrected and label data; The detection module 503 is used to input the corrected image into the wear detection module in the wear detection model to obtain the wear area and the wear degree score of the brake device output by the wear detection module.

[0113] The brake equipment wear detection device provided by the present invention obtains the image to be corrected of the brake equipment; inputs the image to be corrected into the geometric correction module in the wear detection model; the geometric correction module outputs the corrected image; the wear detection model is trained based on the sample image to be corrected and the label data; inputs the corrected image into the wear detection module in the wear detection model, and obtains the wear area and the wear degree score of the brake equipment output by the wear detection module. The image to be corrected is corrected by the geometric correction module in the wear detection model, so that the quality of the corrected image can be improved, and then the corrected image is detected by the wear detection module, so that the wear area and the wear degree score of the brake equipment can be accurately detected, thereby improving the accuracy of wear detection of the brake equipment.

[0114] Optionally, the geometric correction module includes a preprocessing unit, a self-attention encoder, a self-attention decoder and a conversion unit; the image correction module 502 is specifically used to: Inputting the downsampled image obtained by downsampling the image to be corrected into the preprocessing unit to obtain the position perception feature output by the preprocessing unit; Inputting the position-aware feature into the self-attention encoder to obtain an encoded feature output by the self-attention encoder; Inputting the encoded feature into the self-attention decoder to obtain a first decoded feature output by the self-attention decoder; Inputting the second decoding feature obtained after converting the first decoding feature into the conversion unit to obtain a coordinate mapping matrix output by the conversion unit; The rectified image is determined based on the coordinate mapping matrix and the downsampled image.

[0115] Optionally, the preprocessing unit includes a background removal module and at least one residual network; the image correction module 502 is further used to: Inputting the down-sampled image obtained by down-sampling the image to be corrected into the background removal module to obtain a foreground image output by the background removal module; Inputting the foreground image into the first residual network of the consecutive residual networks to obtain the residual features output by the last residual network; The position perception feature is determined based on the two-dimensional matrix feature obtained after the residual feature is converted and the two-dimensional position coding feature corresponding to the image to be corrected.

[0116] Optionally, the conversion unit includes a bilinear interpolation module and at least one convolution layer; the image correction module 502 is further used for: Inputting the second decoded feature obtained after converting the first decoded feature into the bilinear interpolation module to obtain the up-sampled feature output by the bilinear interpolation module; The up-sampled features are input to the first convolution layer in each of the consecutive convolution layers to obtain the coordinate mapping matrix output by the last convolution layer.

[0117] Optionally, the wear detection module includes a backbone network and a convolution block; the detection module 503 is specifically used to: Inputting the rectified image into the backbone network to obtain a plurality of phantom feature maps output by the backbone network; The multiple ghost feature maps are input into the convolution block, and the wear area and the wear degree score of the brake device are obtained as outputs of the convolution block.

[0118] Figure 6 is a schematic diagram of the physical structure of an electronic device provided by the present invention, such as Figure 6As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the brake equipment wear detection method, which includes: obtaining the image to be corrected of the brake equipment; inputting the image to be corrected into the geometric correction module in the wear detection model to obtain the corrected image output by the geometric correction module; the wear detection model is trained based on the sample image to be corrected and the label data; inputting the corrected image into the wear detection module in the wear detection model to obtain the wear area and the wear degree score of the brake equipment output by the wear detection module.

[0119] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0120] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the brake equipment wear detection method provided by the above-mentioned methods, the method comprising: obtaining an image to be corrected of the brake equipment; inputting the image to be corrected into a geometric correction module in a wear detection model to obtain a corrected image output by the geometric correction module; the wear detection model is trained based on sample images to be corrected and label data; inputting the corrected image into a wear detection module in the wear detection model to obtain the wear area and wear degree score of the brake equipment output by the wear detection module.

[0121] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0122] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A brake equipment wear detection method, characterized in that: include: Acquire an image of the braking device to be corrected; Inputting the image to be corrected into a geometric correction module in a wear detection model to obtain a corrected image output by the geometric correction module; the wear detection model is trained based on sample images to be corrected and label data; The corrected image is input into the wear detection module in the wear detection model to obtain the wear area and the wear degree score of the brake device output by the wear detection module.

2. The brake equipment wear detection method according to claim 1, characterized in that: The geometric correction module includes a preprocessing unit, a self-attention encoder, a self-attention decoder and a conversion unit; The step of inputting the image to be corrected into a geometric correction module in a wear detection model to obtain a corrected image output by the geometric correction module comprises: Inputting a downsampled image obtained by downsampling the image to be corrected into the preprocessing unit to obtain a position-aware feature output by the preprocessing unit; Inputting the position-aware feature into the self-attention encoder to obtain an encoded feature output by the self-attention encoder; Inputting the encoded feature into the self-attention decoder to obtain a first decoded feature output by the self-attention decoder; Inputting the second decoding feature obtained after converting the first decoding feature into the conversion unit to obtain a coordinate mapping matrix output by the conversion unit; The rectified image is determined based on the coordinate mapping matrix and the downsampled image.

3. The brake equipment wear detection method according to claim 2, characterized in that: The preprocessing unit includes a background removal module and at least one residual network; The step of inputting the downsampled image obtained by downsampling the image to be corrected into the preprocessing unit to obtain the position-aware features output by the preprocessing unit includes: Inputting the down-sampled image obtained by down-sampling the image to be corrected into the background removal module to obtain a foreground image output by the background removal module; Inputting the foreground image into the first residual network of the consecutive residual networks to obtain the residual features output by the last residual network; The position perception feature is determined based on the two-dimensional matrix feature obtained after the residual feature is converted and the two-dimensional position coding feature corresponding to the image to be corrected.

4. The brake equipment wear detection method according to claim 3, characterized in that: The method of determining the position perception feature based on the two-dimensional matrix feature obtained after the residual feature conversion and the two-dimensional position coding feature corresponding to the image to be corrected includes: The two-dimensional matrix feature obtained after the residual feature is converted and the two-dimensional position coding feature corresponding to the image to be corrected are summed to obtain the position perception feature.

5. The brake equipment wear detection method according to claim 2, characterized in that: The conversion unit includes a bilinear interpolation module and at least one convolutional layer; The step of inputting the second decoding feature obtained after converting the first decoding feature into the conversion unit to obtain a coordinate mapping matrix output by the conversion unit includes: Inputting the second decoded feature obtained after converting the first decoded feature into the bilinear interpolation module to obtain the up-sampled feature output by the bilinear interpolation module; The up-sampled features are input to the first convolution layer in each of the consecutive convolution layers to obtain the coordinate mapping matrix output by the last convolution layer.

6. The brake equipment wear detection method according to claim 1, characterized in that: The wear detection module includes a backbone network and a convolutional block; The step of inputting the corrected image to a wear detection module in the wear detection model to obtain the wear area and the wear degree score of the brake device output by the wear detection module comprises: Inputting the rectified image into the backbone network to obtain a plurality of phantom feature maps output by the backbone network; The multiple ghost feature maps are input into the convolution block, and the wear area and the wear degree score of the brake device are obtained as outputs of the convolution block.

7. A brake equipment wear detection device, characterized in that: include: An acquisition module, used for acquiring an image to be corrected of the braking device; An image correction module, used for inputting the image to be corrected into a geometric correction module in a wear detection model, and outputting a corrected image by the geometric correction module; the wear detection model is obtained by training based on sample images to be corrected and label data; The detection module is used to input the corrected image into the wear detection module in the wear detection model to obtain the wear area and wear degree score of the brake device output by the wear detection module.

8. The brake equipment wear detection device according to claim 7, characterized in that: The geometric correction module includes a preprocessing unit, a self-attention encoder, a self-attention decoder and a conversion unit; The image correction module is specifically used for: Inputting a downsampled image obtained by downsampling the image to be corrected into the preprocessing unit to obtain a position-aware feature output by the preprocessing unit; Inputting the position-aware feature into the self-attention encoder to obtain an encoded feature output by the self-attention encoder; Inputting the encoded feature into the self-attention decoder to obtain a first decoded feature output by the self-attention decoder; Inputting the second decoding feature obtained after converting the first decoding feature into the conversion unit to obtain a coordinate mapping matrix output by the conversion unit; The rectified image is determined based on the coordinate mapping matrix and the downsampled image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the brake equipment wear detection method according to any one of claims 1 to 6 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the brake device wear detection method according to any one of claims 1 to 6 is implemented.

Images