Winding rope disorder detection method and system

The grayscale and thermal distribution images of the winched steel rope are obtained through infrared cameras and thermal imagers. Combined with the deep learning network model, the problem of misjudgment of traditional image detection in insufficient light and dust environments is solved, and the winched rope is realized Accurate detection and alarm.

CN118918097APending Publication Date: 2024-11-08JIANGSU ADVANCED CONSTR MASCH INNOVATION CENT LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411170607.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-23
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

In the field of engineering machinery, the identification and detection of sweeping ropes. In the case of insufficient light and high dust, traditional image detection methods are prone to misjudgment. The amount of color image processing data collected by ordinary cameras is large, which increases the probability of misjudgment .

Method used

An infrared camera is used to obtain the grayscale image of the winched steel rope and a thermal imager to obtain the heat distribution image, extract the thermal abnormality area, and detect it through the deep learning network model (YOLOv5) combined with the grayscale image, reducing the data processing volume and improving accuracy.

Benefits of technology

Thermal imager locates the temperature abnormality area, reduces the amount of data processing, improves judgment accuracy, reduces the probability of misjudgment, and realizes accurate detection and alarm of the winch rope.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918097A_ABST
    Figure CN118918097A_ABST
Patent Text Reader

Abstract

The invention discloses a winch disordered rope detection method and system, and the method comprises the steps: obtaining a winch steel rope thermal imaging image through a thermal imager, obtaining a winch steel rope gray level image through an infrared camera, and carrying out the image processing algorithm of the gray level image corresponding to a heat distribution image temperature abnormal region according to the region, therefore, whether the winding rope disorder problem really exists in the area or not is determined. According to the method, the data processing amount of the processor is reduced, the data processing efficiency of the processor is indirectly improved, and the correctness of a judgment result can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a winch tangled rope detection method and system, belonging to the field of engineering machinery technical control. Background Art

[0002] In the field of engineering machinery, the recognition and detection of tangled winch ropes are greatly affected by traditional image detection in conditions of insufficient light and high dust content. Traditional image processing methods mainly determine whether there is any extrusion of the winch rope in the entire image captured by the camera. There is no processing step to first extract the image of the thermal abnormality area. The problem caused by this is that since the entire image is detected, the amount of image processing data will be large, which can easily lead to the problem of incorrect judgment of tangled winch ropes.

[0003] The existing technology mainly collects the image information of the winch rope through a camera, and then judges whether the wire rope is released out of order by training the recognition results of the network model based on image processing technology. In addition, the cameras used to collect images are all ordinary cameras.

[0004] Since the entire image is processed, the amount of data processed is relatively large, and the probability of misjudgment caused in the image processing process will increase. Since ordinary cameras collect color images, color information has no practical significance in judging the disorder of wire ropes, which increases the processing burden in the image processing process. Summary of the invention

[0005] The present invention provides a method and system for detecting a tangled winch rope, which solves the problems disclosed in the background technology.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is: a method for detecting a disordered rope of a winch, comprising:

[0007] Obtain grayscale image and thermal distribution image of the winch rope;

[0008] Extracting thermal anomaly areas in thermal distribution images;

[0009] Mapping the thermal anomaly area to the corresponding area in the grayscale image to obtain the grayscale image corresponding to the thermal anomaly area;

[0010] Detect the grayscale image corresponding to the thermal anomaly area to determine whether there is a disordered wire rope phenomenon.

[0011] Furthermore, the heat distribution image of the hoisting rope is expressed as: ; Filtered thermal distribution image It is expressed as:

[0012]

[0013] Where 𝑥 and y are the coordinates of the image, is a Gaussian function, is the filter window size.

[0014] Furthermore, the method for extracting the thermal anomaly area in the thermal distribution image is:

[0015] The filtered thermal distribution image Perform temperature threshold segmentation and set temperature threshold ,The pixels above the threshold are marked as thermal anomaly areas, and the thermal anomaly areas generate binary images :

[0016] ;

[0017] For binary images Conduct connected region analysis, mark and count all independent thermal anomaly regions, and obtain multiple candidate thermal anomaly regions ;

[0018] ,in, It is a thermal anomaly area The coordinates of the upper left corner in the image are the thermal anomaly area The starting position of It is a thermal anomaly area The width of It is a thermal anomaly area Height;

[0019] Furthermore, the grayscale image of the hoisting rope is represented as: , the grayscale image corresponding to the thermal anomaly area is expressed as: .

[0020] Furthermore, the method of detecting the grayscale image corresponding to the thermal anomaly area and determining whether there is a disordered steel wire phenomenon is as follows:

[0021] The input is input into the pre-trained deep learning network model, which uses the YOLOv5 model. The model structure includes an input layer, multiple convolutional layers, a pooling layer, and a fully connected layer. The input layer receives , after the convolution operation and activation function processing, the convolution operation is expressed as:

[0022]

[0023] in, is the convolution kernel, is the bias, is the activation function, * represents the convolution operation;

[0024] Multiple convolutional layers and pooling layers perform feature extraction and dimensionality reduction operations alternately, and finally perform classification and location prediction in the fully connected layer; the model outputs the results ;

[0025] Based on the output of the deep learning model , when the detection probability Exceeding the set threshold It was determined that there was a disorderly steel rope phenomenon in this area.

[0026] Furthermore, histogram equalization is used to improve the contrast of the grayscale image of the winch rope. It is expressed as: .

[0027] Furthermore, when it is determined that there is a disordered phenomenon of the steel ropes, an alarm message is issued, and the alarm message includes the location of the abnormal area and the detection result.

[0028] Accordingly, the present invention also provides a winch tangled rope detection system, comprising:

[0029] Infrared camera, used to obtain grayscale image of winch rope;

[0030] Thermal imager, used to obtain thermal distribution images of winch ropes;

[0031] The controller is used to extract the thermal anomaly area in the thermal distribution image; map the thermal anomaly area to the corresponding area in the grayscale image to obtain the grayscale image corresponding to the thermal anomaly area; detect the grayscale image corresponding to the thermal anomaly area to determine whether there is a disordered steel rope phenomenon.

[0032] Furthermore, it also includes: an alarm; when it is determined that there is a disorder phenomenon of the steel ropes, the controller sends an alarm message through the alarm.

[0033] Furthermore, it also includes: a display; when it is determined that there is a disorder phenomenon of the steel ropes, the controller displays the position of the abnormal area and the detection result through the display.

[0034] The beneficial effects achieved by the present invention are:

[0035] (1) The present invention locates the abnormal temperature area of ​​the hoisting rope through a thermal imager, then extracts the abnormal area and processes only the abnormal area, thereby reducing the amount of data processing, improving processing efficiency, and improving the accuracy of judgment.

[0036] (2) The present invention uses an infrared camera to collect grayscale images of the winch rope. Such grayscale images contain a small amount of data and do not lose feature information during the image processing process, thereby reducing the amount of data processing and improving reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a structural schematic diagram of the winch disordered rope detection system of the present invention;

[0038] Figure 2 This is a schematic diagram of the flow of the method for detecting a disordered rope of a winch according to the present invention;

[0039] Figure 3 It is a schematic diagram of the network model training process in the winch tangled rope detection method of the present invention. DETAILED DESCRIPTION

[0040] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0041] Embodiment 1:

[0042] like Figure 1 As shown, this embodiment provides a hardware technical solution for the winch tangled rope detection system: it includes a thermal imager, an infrared camera, a controller, a display, and an alarm. The thermal imager and the infrared camera are placed near the winch steel rope, the thermal imager is used to collect the heat distribution image of the winch steel rope in real time, and the infrared camera is used to collect the grayscale image of the winch steel rope in real time. The display, alarm and controller are placed in the cab, and the controller uses an MCU, which is used for image processing, image display and output alarm information. The heat distribution image and grayscale image of the steel rope are transmitted to the controller in the cab in real time through the data line, the controller processes the image, the display and the controller transmit data through the HDMI bus, the controller will send the steel rope image information to the display through the HDMI bus in real time, and the controller will first process the heat distribution image. If the rope is tangled, the friction and extrusion between the steel ropes will cause the temperature of the tangled rope part to be higher than the normal temperature of the surrounding area, so that the abnormal area can be pre-judged by temperature information. When a thermal abnormal area appears, the abnormal area is synchronously mapped to the grayscale image collected by the infrared camera, and then the controller processes and judges the grayscale image of the mapped part, and finally it is concluded whether the rope tangled phenomenon actually occurs. The function of the alarm is to provide voice alarm information to the people in the cab when the rope is tangled, reminding the driver to stop the operation of the winch in time, and display the relevant information on the monitor to remind the operator to take necessary measures in time.

[0043] The entire system realizes real-time monitoring and abnormality detection of the steel rope status, thereby ensuring the safe operation of the winch. By combining infrared images and thermal imaging technology, the system can more comprehensively and accurately identify abnormal conditions of the steel rope, provide timely and effective safety warnings to operators, and ensure the safe operation and maintenance of construction machinery. This intelligent monitoring system brings more efficient and reliable solutions to the engineering field, helping to improve the safety and reliability of engineering equipment.

[0044] Embodiment 2:

[0045] like Figure 2 As shown, this embodiment provides a method for detecting a disordered rope in a winch. The process controller obtains the thermal distribution image and grayscale image of the winch steel rope. The process first processes the thermal distribution image and uses image processing to identify possible thermal anomaly areas. A thermal imager is used to record the temperature data of the steel rope surface to generate a thermal distribution image. , where 𝑥 and y are the coordinates of the image. The thermal distribution image is subjected to noise filtering, and a Gaussian filter is used to eliminate image noise and improve image quality. Thermal distribution image after filtering It is expressed as:

[0046]

[0047] in, is a Gaussian function, where i and j are the coordinates within the filter window, is the filter window size.

[0048] The filtered thermal distribution image Perform temperature threshold segmentation to extract abnormal temperature areas. Set temperature threshold , mark the pixels above the threshold as abnormal areas. Generate binary image :

[0049]

[0050] For binary images Conduct connected region analysis, mark and count all independent thermal anomaly regions, and obtain multiple candidate thermal anomaly regions If no thermal anomaly areas are detected , the system will not trigger the subsequent grayscale image processing process and directly end the current processing flow. , the controller will record the location and size information of these abnormal areas , and trigger the processing flow of the corresponding grayscale image. It is a thermal anomaly area The coordinates of the upper left corner in the image. This point is the thermal anomaly area. The starting position of It is a thermal anomaly area The width of It is a thermal anomaly area The grayscale image is collected by the infrared camera outside the cab, and the controller obtains the grayscale image of the winch rope from the infrared camera. , record the grayscale image corresponding to the thermal anomaly area. Perform image enhancement on the grayscale image and use histogram equalization to improve the image contrast. The processed grayscale image It is expressed as:

[0051] The controller only processes the grayscale image of the part related to the detected thermal anomaly area to reduce the amount of data processing. Corresponding to thermal anomaly area . Input to the pre-trained deep learning network model for further analysis. The YOLOv5 model is used. The model structure includes an input layer, multiple convolutional layers, a pooling layer, and a fully connected layer. The input layer receives , after convolution operation and activation function processing, image features are extracted. The convolution operation is expressed as: ;

[0052] in, is the convolution kernel, is the bias, is the activation function, and * represents the convolution operation.

[0053] Multiple convolutional layers and pooling layers perform feature extraction and dimensionality reduction operations alternately, and finally perform classification and positioning prediction in the fully connected layer. The model output includes the output results of the detection box and classification probability. .

[0054] Based on the output of the deep learning model , set the judgment rules. When the detection probability Exceeding the set threshold It was determined that there was a problem of out-of-sequence steel ropes in this area.

[0055] The judgment formula is: ;

[0056] If the system confirms that there is a disorder, the controller will display the abnormal information on the display and trigger the speaker to send an alarm signal to remind the driver to stop the rope winding operation. The alarm information includes the location of the abnormal area and the detection result. If no disorder is found in the steel rope, the system will continue to process the next frame of the image and repeat the above process.

[0057] The entire processing flow combines the information of thermal distribution maps and grayscale images, and the trained network model realizes real-time monitoring of the steel rope status and abnormality identification.

[0058] Figure 3The training process of the network model. In the winch tangled rope detection method based on infrared image and sensor data fusion, the image processing part is a crucial link in the whole system, which directly affects the accuracy and reliability of the detection. Image processing involves multiple key steps, including image annotation, data set production, sensor data recording, image processing and data fusion, as well as model training and optimization deployment. First, a large number of infrared images containing disordered and normal states of steel ropes are collected. These images are manually annotated using annotation tools, and the annotation content includes the specific location of the steel rope and the abnormal area. The annotation format usually adopts the YOLO format, recording the coordinates, width, height and category label of the annotation box. According to the annotation results, a training set, a validation set and a test set are produced. The data set is divided into 70% training set, 20% validation set and 10% test set. Data enhancement operations are performed on the data set, including rotation, scaling, cropping, flipping and color transformation, to increase data diversity and prevent model overfitting. While collecting images, sensor data related to the steel rope, including temperature, is recorded. The sensor data is synchronized to the corresponding image frame. The sensor data is recorded in a CSV file, with each line recording data at a time point. At the same time, model training, deployment, and application are also crucial links.

[0059] In the training and deployment phase of the model, first select a deep learning model suitable for processing single-channel images, such as a convolutional neural network (CNN). CNN can effectively learn image features and perform classification. The prepared dataset is used to train the selected CNN model. When training the model, the cross entropy loss function is used for classification training, and the smooth L1 loss function is used for bounding box regression training. The optimization algorithm can be Adam or SGD. During the model training process, monitor the training loss and validation loss to avoid overfitting. The early stopping strategy and learning rate decay strategy can be used to optimize the training process. Through the back-propagation algorithm and optimizer, the model gradually learns and improves the ability to identify abnormal thermal areas of the winch steel rope. Subsequently, the trained model is pruned and quantized to reduce the complexity of the model and the consumption of computing resources, and finally deployed to the winch system of the construction machinery.

[0060] On the other hand, in the application stage of the model, the collected infrared images are first preprocessed. This includes operations such as denoising, enhancement, and histogram equalization to reduce interference and highlight target features. At the same time, the sensor data related to the winch system is recorded so that the abnormal area can be mapped to the infrared image. In the process of real-time monitoring of infrared images and sensor data, the winch rope disorder detection is carried out to detect abnormal situations in time and take corresponding measures to ensure the safe operation of construction machinery.

[0061] Through the above detailed steps, combined with image processing, fusion application of sensor data and infrared images, model training and deployment, and model application, the detection accuracy and efficiency of winch tangled rope problems can be effectively improved, providing important support for the safe operation of construction machinery. This comprehensive method combines image processing technology, deep learning models and sensor data fusion to bring more advanced and reliable solutions to the engineering field. The application of this technology will provide important technical support for safety management and equipment maintenance in the engineering field, and is expected to play an important role in actual engineering applications.

[0062] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

[0063] Embodiment 3:

[0064] A computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions, and when the instructions are executed by a computing device, the computing device executes a method for detecting a tangled rope in a winch.

[0065] A computing device includes one or more controllers, one or more memories, and one or more programs, wherein the one or more programs are stored in the one or more memories and are configured to be executed by the one or more controllers, and the one or more programs include instructions for executing a winch tangled rope detection method.

[0066] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0067] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a controller of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the controller of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0068] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0069] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0070] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are included in the scope of the claims of the present invention to be approved.

Claims

1. A method for detecting a tangled winch rope. It is characterized in that include: Obtain grayscale image and thermal distribution image of the winch rope; Extracting thermal anomaly areas in thermal distribution images; Mapping the thermal anomaly region to a corresponding region in the grayscale image to obtain a grayscale image corresponding to the thermal anomaly region; Detect the grayscale image corresponding to the thermal anomaly area to determine whether there is a disordered wire rope phenomenon.

2. The method for detecting a tangled winch rope according to claim 1, Features: The heat distribution image of the winch rope is expressed as: ; Filtered thermal distribution image It is expressed as: ; Where 𝑥 and y are the coordinates of the image, is a Gaussian function, is the filter window size.

3. The method for detecting a tangled winch rope according to claim 2, Features: The method for extracting thermal anomaly areas in thermal distribution images is: The filtered thermal distribution image Perform temperature threshold segmentation and set temperature threshold ,The pixels above the threshold are marked as thermal anomaly areas, and the thermal anomaly areas generate binary images : ; For binary images Conduct connected region analysis, mark and count all independent thermal anomaly regions, and obtain multiple candidate thermal anomaly regions ; ,in, It is a thermal anomaly area The coordinates of the upper left corner in the image are the thermal anomaly area The starting position of It is a thermal anomaly area The width of It is a thermal anomaly area height.

4. The method for detecting a tangled winch rope according to claim 3, Features: The grayscale image of the hoisting rope is represented as: , the grayscale image corresponding to the thermal anomaly area is expressed as: .

5. The method for detecting a tangled winch rope according to claim 4, Features: The method of detecting the grayscale image corresponding to the thermal anomaly area and determining whether there is a disordered steel wire phenomenon is as follows: The input is input into the pre-trained deep learning network model, which uses the YOLOv5 model. The model structure includes an input layer, multiple convolutional layers, a pooling layer, and a fully connected layer. The input layer receives , after the convolution operation and activation function processing, the convolution operation is expressed as: ; in, is the convolution kernel, is the bias, f is the activation function, and * represents the convolution operation; Multiple convolutional layers and pooling layers perform feature extraction and dimensionality reduction operations alternately, and finally perform classification and location prediction in the fully connected layer; the model outputs the results ; Based on the output of the deep learning model , when the detection probability Exceeding the set threshold It was determined that there was a disorderly steel rope phenomenon in this area.

6. The method for detecting a tangled winch rope according to claim 4, It is characterized in that Also includes: Histogram equalization is used to improve the contrast of the grayscale image of the winch rope. It is expressed as: .

7. The method for detecting a tangled winch rope according to claim 1, It is characterized in that Also includes: When it is determined that there is a disordered phenomenon of the steel ropes, an alarm message is issued, and the alarm message includes the location of the abnormal area and the detection result.

8. A winch tangled rope detection system, It is characterized in that include: Infrared camera, used to obtain grayscale image of winch rope; Thermal imager, used to obtain thermal distribution images of winch ropes; A controller, used for extracting thermal anomaly areas in the thermal distribution image; Mapping the thermal anomaly area to the corresponding area in the grayscale image to obtain the grayscale image corresponding to the thermal anomaly area; Detect the grayscale image corresponding to the thermal anomaly area to determine whether there is a disordered wire rope phenomenon.

9. The winch tangled rope detection system according to claim 8, It is characterized in that Also includes: Alarm; When it is determined that the steel ropes are out of order, the controller sends out an alarm message through the alarm.

10. The winch tangled rope detection system according to claim 8, It is characterized in that Also includes: monitor; When it is determined that there is a disordered wire rope phenomenon, the controller displays the location of the abnormal area and the detection results through the display.

Citation Information

Patent Citations

  • Monitoring method and device based on infrared thermal imaging technology

    CN108419045A

  • Elevator steel wire rope detection method based on machine vision

    CN112270658A

  • Steel wire rope eddy current thermal imaging defect detection device

    CN212255191U

  • Facial recognition living body detection method based on facial iris recognition and thermal imaging technology

    WO2021227124A1