Fault detection method and fault detection system for car lifting jack

Abstract

The invention discloses a fault detection method and a fault detection system for a car lifting jack. The method comprises the following steps: acquiring an image; inputting the collected image into the trained single-time multi-bounding-frame detector network model of the fusion residual network, and identifying a to-be-detected part by the single-time multi-bounding-frame detector network model of the fusion residual network: if the identification is normal or the detection precision is higher than a set value, displaying that no fault exists; and if the condition that identification cannot be carried out or the detection precision is lower than a set value occurs, a fault warning is sent out. According to the invention, fault missing detection of the car lifting jack can be reduced, and the automation level is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of railway train detection, and in particular relates to a fault detection method and a fault detection system for a train crane. Background Art

[0002] In the operation and management of rail trains, fixed driving machines are relatively important maintenance equipment, mainly used for lifting train vehicles and maintenance of underbody components. Due to the large size and weight of trains, lifting operations are somewhat dangerous. Therefore, it is crucial to ensure the installation quality of the equipment and reduce the failure rate of the equipment.

[0003] During the process of lifting the train, due to the more or less certain distribution errors of the same lifting machine group, there are load differences and certain vibrations in different lifting machine equipment, which easily leads to the loosening and falling off of the lifting machine parts (especially the lifting arm bolts). At present, the main maintenance method for the lifting machine is manual inspection. Due to the large number of lifting machine parts and their small size, this traditional method has problems such as high labor costs and missed faults. In addition, during the process of lifting the carriage, since personnel cannot monitor the equipment operation status in real time, there are difficulties in driving the machine maintenance, and most of the current inspections are for larger component failures.

[0004] In view of this, how to design a technology that reduces missed fault detection, has low labor costs, and achieves automation to a certain extent is a technical problem to be solved by the present invention. Summary of the invention

[0005] The present invention provides a fault detection method and a fault detection system for a vehicle lifting machine, which can reduce missed fault detection of the vehicle lifting machine and improve the automation level.

[0006] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0007] In one aspect, the present invention provides a method for detecting a fault of a vehicle lifting machine, comprising the following steps:

[0008] Collect images;

[0009] The collected image is input into the trained single-shot multi-bounding box detector network model of the fused residual network, and the single-shot multi-bounding box detector network model of the fused residual network identifies the parts to be detected of the crane:

[0010] If the recognition is normal or the detection accuracy is higher than the set value, it will be displayed as no fault;

[0011] If the recognition fails or the detection accuracy is lower than the set value, a fault warning will be issued.

[0012] In the above-mentioned method for detecting faults of a crane, the single-shot multi-boundary box detector network model of the fused residual network includes:

[0013] The backbone residual network Resnet101 is used to preliminarily extract target features and output feature maps;

[0014] A multi-scale feature detection module is used to perform further feature extraction and target prediction on the feature map.

[0015] In the above-mentioned method for detecting the fault of the crane, the formula for initially extracting the target features by the backbone residual network Resnet101 is as follows:

[0016] H(X)=F(X,W i )+X

[0017] Among them, H(X) represents the output of the backbone residual network Resnet101; F(X, W i ) represents the residual; X represents the input image.

[0018] In the above-mentioned method for detecting faults of a crane, two pre-selection boxes with aspect ratios of 3 and 1 / 3 are added to the feature map output by the ResNet101 network.

[0019] In the above-mentioned method for detecting a fault of a crane, the steps of training the single-shot multi-bounding box detector network model of the fused residual network include:

[0020] S21, establish a data set;

[0021] S22, initialize the single-shot multi-bounding box detector network model fused with the residual network;

[0022] S23, inputting the data set into the initialized single-shot multi-bounding box detector network model of the fusion residual network for training and learning;

[0023] S24, optimizes the single multi-bounding box detector network model of the fused residual network through the loss function and the optimizer to obtain the optimized weight coefficient and complete the training.

[0024] In the above-mentioned method for detecting the fault of the vehicle lifting machine, the calculation formula of the loss function is as follows:

[0025]

[0026] Among them, L represents the loss function; N represents the number of matched prediction boxes; L conf represents the position error loss; α represents the weight coefficient; L loc Represents the confidence error loss.

[0027] In the above-mentioned method for detecting the fault of the vehicle crane, the formula of the position error loss is as follows:

[0028]

[0029] Among them, L loc (x, l, g) represents the position error loss; POS represents the positive sample set; i represents the i-th sample in the positive sample set; {cx, cy, w, h} represents the position parameters of the anchor point; m represents the position parameters of the m-th anchor point; Indicates a unique identifier; L smoothL1 represents the smooth norm loss function; Represents the predicted value, that is, the offset value between the bounding box and the default box; Represents the true value, that is, the offset value between the true box and the default box.

[0030] The formula for the confidence error loss is as follows:

[0031]

[0032] Among them, L conf (x, c) represents the confidence error loss; POS represents the positive sample set; i represents the i-th sample in the positive sample set; Indicates a unique identifier; Represents the probability of negative sample classification; Indicates the probability that the i-th anchor point classification is p; It means that the category of anchor point anchori is most likely to be p.

[0033] In another aspect, the present invention provides a vehicle crane fault detection system, comprising:

[0034] An image acquisition module, which is used to acquire images of the vehicle crane;

[0035] A fault detection module connected to the image acquisition module, wherein the fault detection module is provided with a method for detecting a fault of a vehicle crane as described in any one of the above items, and the fault detection module processes the image of the vehicle crane and determines whether the vehicle crane has a fault;

[0036] The alarm module is connected to the fault detection module and generates an alarm prompt when the fault detection module detects that the vehicle lifting machine has a fault.

[0037] In another aspect, the present invention further provides an electronic device, comprising: a processor, and a memory and a transceiver communicatively connected to the processor;

[0038] The memory stores computer-executable instructions; the transceiver is used to send and receive data;

[0039] The processor executes the computer-executable instructions stored in the memory to implement any of the above methods.

[0040] In another aspect, the present invention also provides a computer-readable storage medium,

[0041] The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement any of the above methods.

[0042] Compared with the prior art, the advantages and positive effects of the present invention are: through the single multi-boundary box detector network model designed by integrating the residual network concept, the collected car lifting machine image is recognized to detect whether the car lifting machine has a fault, which can improve the detection accuracy of the car lifting arm bolts for small-area detection targets, realize the automation of fault detection, reduce the output of manual labor, and improve the fault detection rate; the trained single multi-boundary box detector network model integrated with the residual network has good real-time performance, which can provide important technical support for the image-based inspection of easily detached parts of the car lifting machine, and greatly improve the safety of train maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0044] Figure 1 A flowchart of an embodiment of a method for detecting a fault of a vehicle lifting machine according to the present invention;

[0045] Figure 2 It is a structural diagram of the RN SSD network model in an embodiment of the vehicle crane fault detection method of the present invention;

[0046] Figure 3 A flowchart of training the RN SSD network model in an embodiment of the vehicle crane fault detection method of the present invention;

[0047] Figure 4 It is a structural schematic diagram of the residual block of the RN SSD network model in an embodiment of the fault detection method of the vehicle crane of the present invention;

[0048] Figure 5 It is a multi-scale prediction schematic diagram of the RN SSD network model in an embodiment of the vehicle crane fault detection method of the present invention;

[0049] Figure 6It is a structural schematic diagram of a pre-selection box of an RN SSD network model in an embodiment of a method for detecting a fault of a vehicle crane of the present invention;

[0050] Figure 7 This is a schematic diagram of grayscale processing of a data set when training an RN SSD network model in an embodiment of a method for fault detection of a vehicle crane of the present invention;

[0051] Figure 8 This is a diagram of the detection results of inputting the image of the vehicle-lifting machine into the RN SSD network model in one embodiment of the vehicle-lifting machine fault detection method of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] It should be noted that in the description of the present invention, the terms "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings, which are only for the convenience of description, and do not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0054] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0055] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may include that the first and second features are in direct contact, or may include that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, a first feature being "above", "above" and "above" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply indicates that the first feature is higher in level than the second feature. A first feature being "below", "below" and "below" a second feature includes that the first feature is directly below and obliquely below the second feature, or simply indicates that the first feature is lower in level than the second feature.

[0056] The disclosure below provides many different embodiments or examples to realize different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or reference letters in different examples, and this repetition is for the purpose of simplicity and clarity, which itself does not indicate the relationship between the various embodiments and / or settings discussed. In addition, the present invention provides various specific examples of processes and materials, but those of ordinary skill in the art can be aware of the application of other processes and / or the use of other materials.

[0057] like Figures 1 to 8 As shown, the present application provides a method for detecting a fault of a vehicle lifting machine, comprising the following steps:

[0058] Collect images, specifically:

[0059] Use high-resolution line array cameras to capture clear images of the crane.

[0060] The collected image is input into the trained Resnet-single shot multibox detector (RN SSD) network model, hereinafter referred to as the RN SSD network model, and the Resnet-single shot multibox detector network model identifies the parts to be detected.

[0061] If the normal recognition or detection accuracy is higher than the set value, the display screen of the terminal, such as a computer, industrial computer, etc., will show that there is no fault;

[0062] If the identification fails or the detection accuracy is lower than the set value, a fault warning will be issued, such as a fault warning prompt on the display screen of the terminal and a buzzer will sound an alarm at the same time.

[0063] According to production experience, the detection accuracy is set to 0.9. When recognition fails or the detection accuracy is lower than 0.9, a fault warning is issued, thereby improving the safety of crane maintenance.

[0064] The RN SSD network model consists of two parts:

[0065] One part is the backbone residual network Resnet101 located at the front end of the network, which is used to preliminarily extract target features and form feature maps.

[0066] The main residual network Resnet101 is composed of several residual blocks, which allows the gradient to propagate by skipping a certain number of layers, which makes it easier to train Resnet101 with deeper layers and avoids the "degradation problem" of gradient disappearance and explosion. Figure 4 As shown in the figure, part of the input image X is learned through the weight layer, and the other part is directly used as the initial value of the output through identity mapping. At the same time, convolution is performed on the residual branch to obtain the residual F(X, W i ), residual F(X, W i ) will continue to approach the value of 0 during the back propagation training process. Finally, the two parts of the results are added to obtain the output value H(X) that approximates the input value X. Therefore, the formula for the preliminary extraction of target features by the backbone residual network Resnet101 is as follows:

[0067] H(X)=F(X,W i )+X

[0068] Among them, H(X) represents the output of the backbone residual network Resnet101; F(X, W i ) represents the residual; X represents the input image.

[0069] The other part is a multi-scale feature detection module located at the back end of the network and composed of multiple convolutional layers. In this application, the multi-scale feature detection module includes 6 convolutional layers, which are used to perform feature extraction and target prediction on each feature map.

[0070] The formula for further feature extraction and target prediction of the multi-scale feature detection module on the feature map is as follows:

[0071] The multi-scale feature detection module performs further feature extraction and target detection on the feature map, and outputs the feature map for prediction in the convolutional layers conv4_3, conv7_2, conv8_2, conv9_2, and conv10_2, such as Figure 2 and Figure 5 As shown, the feature information of the detected target can be obtained, and the features of objects of different scales in the image can be extracted.

[0072] In order to further improve the detection capability of the RN SSD network model for small target installation stations, this application uses the feature map output by the Res5c_relu layer of the Resnet101 network, such as Figure 2 As shown in the figure, two pre-selection boxes with aspect ratios of 3 and 1 / 3 are added by modifying the network code of the network model. In order to improve the defect that the RN SSD network model is insensitive to small targets, combined with the characteristics of the shape and size of the target to be detected by the crane, and after a large number of experiments and verifications, the detection effect of adding pre-selection boxes with aspect ratios of 3 and 1 / 3 is the best.

[0073] For example, when adding two pre-selection boxes with aspect ratios of 4 and 1 / 4, when using the RN SSD network model for fault detection, it was found that the accuracy decreased, and the average detection accuracy decreased by about 5 percentage points.

[0074] For another example, when four pre-selection boxes with aspect ratios of 3 and 1 / 3 were added, when the RN SSD network model was used for fault detection, it was found that the computational burden increased, and the average detection time for each image increased by about 30%.

[0075] Therefore, adding two pre-selection boxes can not only ensure the feature extraction capability, but also will not greatly increase the network redundancy and increase the computational burden, thus taking into account both the accuracy and real-time performance of detection.

[0076] At this time, the number of pre-selected boxes generated at each pixel of the feature map is 6, such as Figure 6 As shown, the green pre-selection box indicates the pre-selection box with an aspect ratio of 2 and 1 / 2; the red pre-selection box indicates the pre-selection box with an aspect ratio of 1 and 1; and the blue pre-selection box indicates the newly added pre-selection box with an aspect ratio of 3 and 1 / 3.

[0077] This application uses the method of deepening the network to improve the model's ability to extract small target features, and uses the residual network Resnet101 as the backbone network. While solving the problem of the model's insensitivity to small target installation positions, it avoids the problem of network degradation and ensures that the overall detection accuracy does not decrease.

[0078] Input the image of the vehicle crane into the trained RN SSD model and obtain the detection result without faults as shown below Figure 8 As shown, the value above the green box is the detection accuracy. The green box indicates that the detection result is that the part can be identified normally and no failure occurs. If the part cannot be identified normally or the detection accuracy is lower than the set value, the box outside the part is a red box.

[0079] The steps for training the RN SSD network model include:

[0080] S1, establish a data set, specifically:

[0081] First, a high-resolution linear array camera is used to collect clear images of the crane. Due to the images of different locations, different equipment, different lighting and other factors, the images of the crane's arm are quite different. Therefore, it is necessary to collect enough target images to improve the detection accuracy and adaptability of the RN SSD network model. To this end, this application takes 8,000 images of the crane at different times and from different angles as data set samples. At the same time, the collected images are translated, rotated, mirrored, cropped, and scaled to expand the data set samples to 15,000.

[0082] Then, the images of the dataset samples are grayscale converted and enhanced: the color RGB image is converted to a grayscale image, the grayscale threshold is set, the pixels less than the threshold 175 are replaced with black pixels, and the pixels greater than the threshold 175 are set as white pixels, and the threshold segmentation is performed to highlight the target. The image after grayscale conversion and enhancement is shown in the figure below. Figure 7 shown.

[0083] Finally, all the sample images of the dataset are converted to the dataset VOC2007 format, and the category and location information of the detection targets in the dataset are annotated using the labelImg marking tool to generate an xml file; then the division work is completed, 12,000 images are used as the training set, and 3,000 images are used as the test set. In the training set, the RN SSD network model is cross-validated in units of 2,000 images. At this point, the dataset is established.

[0084] S2, initialize the RN SSD network model, specifically:

[0085] Initialize the weight coefficients using the Gaussian distribution initialization method, that is, use a Gaussian distribution with a mean of 0 and a standard deviation of 0.001 to initialize the weight coefficients. Initialization can prevent gradients from disappearing or exploding, ensuring the stability of the RN SSD network model.

[0086] S3, input the data set into the initialized RN SSD network model for training and learning.

[0087] S4, optimize the RN SSD network model through the loss function and optimizer to obtain the optimal weight coefficient and complete the training. Specifically:

[0088] The loss function is the weighted sum of the position error loss and the confidence error loss, and its formula is as follows:

[0089]

[0090] Among them, L represents the loss function; N represents the number of matched prediction boxes; L confrepresents the position error loss; α represents the weight coefficient; L loc Represents the confidence error loss.

[0091] The formula for position error loss is as follows:

[0092]

[0093] Among them, L loc (x, l, g) represents the position error loss; POS represents the positive sample set; i represents the i-th sample in the positive sample set; {cx, cy, w, h} represents the position parameter of the anchor point; m represents the position parameter of the m-th anchor point; Indicates the unique identification flag; L smoothL1 represents the smooth norm loss function; Represents the predicted value, that is, the offset value between the bounding box and the default box; Represents the true value, that is, the offset value between the ground truth box and the default box.

[0094] The formula for confidence error loss is as follows:

[0095]

[0096] Among them, L conf (x, c) represents the confidence error loss; POS represents the positive sample set; i represents the i-th sample in the positive sample set; Indicates the unique identification flag; Represents the probability of negative sample classification; Indicates the probability that the i-th anchor classification is p; It means that the category of anchor point anchori is most likely to be p;

[0097] The loss function L is calculated repeatedly and iteratively, and the weights are optimized by the optimizer Adam to obtain the optimal weight coefficient. At this time, the network model training RN SSD is completed.

[0098] In another aspect, the present application provides a vehicle crane fault detection system, comprising:

[0099] The image acquisition module is used to acquire images of the vehicle crane. The image acquisition module adopts a high-resolution linear array camera.

[0100] A fault detection module is connected to the image acquisition module. The fault detection module is provided with the above-mentioned vehicle crane fault detection method. The fault detection module processes the image of the vehicle crane and determines whether the vehicle crane has a fault.

[0101] The alarm module is connected to the fault detection module and generates an alarm prompt when the fault detection module detects that the vehicle lifting machine has a fault.

[0102] In another aspect, the present invention further provides an electronic device, comprising: a processor, and a memory and a transceiver communicatively connected to the processor;

[0103] The memory stores computer executable instructions; the transceiver is used to send and receive data;

[0104] The processor executes the computer-executable instructions stored in the memory to implement the method described above.

[0105] In another aspect, the present application also provides a computer-readable storage medium,

[0106] The computer-readable storage medium stores computer-executable instructions, which are used to implement the above-mentioned method when executed by a processor.

[0107] The present application can realize the automation of fault detection, reduce the output of manual labor, and improve the fault detection rate; the present application also has good real-time performance, and can provide important technical support for the image-based detection of easily detached parts of the crane, greatly improving the safety of the crane maintenance.

[0108] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0109] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for a person skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to replace some of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions claimed to be protected by the present invention.

[0110] Wherever possible, the various aspects and features described and shown in the specification may be applied separately and these separate aspects may be made the subject of divisional applications.

Claims

1. A method for detecting a fault of a vehicle lifting machine, characterized in that: The following steps are involved: Collect images; The collected image is input into the trained single-shot multi-bounding box detector network model of the fused residual network, and the single-shot multi-bounding box detector network model of the fused residual network identifies the parts to be detected of the crane: If the recognition is normal or the detection accuracy is higher than the set value, it will be displayed as no fault; If the recognition fails or the detection accuracy is lower than the set value, a fault warning will be issued.

2. The method for detecting a fault of a vehicle lifting machine according to claim 1, characterized in that: The single-shot multi-bounding box detector network model of the fused residual network includes: The backbone residual network Resnet101 is used to preliminarily extract target features and output feature maps; A multi-scale feature detection module is used to perform further feature extraction and target prediction on the feature map.

3. The method for detecting a fault of a vehicle lifting machine according to claim 2, characterized in that: The formula for initially extracting target features by the backbone residual network Resnet101 is as follows: H(X)=F(X,W i )+X Among them, H(X) represents the output of the backbone residual network Resnet101; F(X, W i ) represents the residual; X represents the input image.

4. The method for detecting a fault of a vehicle lifting machine according to claim 3, characterized in that: Two pre-selected boxes with aspect ratios of 3 and 1 / 3 are added to the feature map output by the ResNet101 network.

5. The method for detecting a fault of a vehicle lifting machine according to claim 1, characterized in that: The steps of training the single-shot multi-bounding box detector network model of the fused residual network include: S21, establish a data set; S22, initialize the single-shot multi-bounding box detector network model fused with the residual network; S23, inputting the data set into the initialized single-shot multi-bounding box detector network model of the fusion residual network for training and learning; S24, optimizes the single multi-bounding box detector network model of the fused residual network through the loss function and the optimizer to obtain the optimized weight coefficient and complete the training.

6. The method for detecting a fault of a vehicle lifting machine according to claim 5, characterized in that: The calculation formula of the loss function is as follows: Among them, L represents the loss function; N represents the number of matched prediction boxes; L conf represents the position error loss; α represents the weight coefficient; L loc Represents the confidence error loss.

7. The method for detecting a fault of a vehicle lifting machine according to claim 6, characterized in that: The formula for the position error loss is as follows: Among them, L loc (x, l, g) represents the position error loss; POS represents the positive sample set; i represents the i-th sample in the positive sample set; {cx, cy, w, h} represents the position parameters of the anchor point; m represents the position parameters of the m-th anchor point; Indicates a unique identifier; L smoothL1 represents the smooth norm loss function; Represents the predicted value, that is, the offset value between the bounding box and the default box; Represents the true value, that is, the offset value between the true box and the default box. The formula for the confidence error loss is as follows: Among them, L conf (x, c) represents the confidence error loss; POS represents the positive sample set; i represents the i-th sample in the positive sample set; Indicates a unique identifier; Represents the probability of negative sample classification; Indicates the probability that the i-th anchor point classification is p; It indicates that the category of the anchor point is most likely to be p.

8. A vehicle crane fault detection system, characterized in that: include: An image acquisition module, which is used to acquire images of the vehicle crane; a fault detection module connected to the image acquisition module, wherein the fault detection module is provided with the method for detecting a fault of a vehicle crane as claimed in any one of claims 1 to 7, and wherein the fault detection module processes the image of the vehicle crane and determines whether the vehicle crane has a fault; The alarm module is connected to the fault detection module and generates an alarm prompt when the fault detection module detects that the vehicle lifting machine has a fault.

9. An electronic device, characterized in that: include: A processor, and a memory and a transceiver communicatively connected to the processor; The memory stores computer-executable instructions; the transceiver is used to send and receive data; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images