Method and system for calculating torsion angle of four-rope grab bucket of crane based on YOL0v5

Through the YOLOv5 algorithm, the twist angle of the four-rope grab is monitored and calculated in real time, and the crane movement is automatically adjusted, which solves the problem of shaking caused by dynamic disturbances in the crane operation of the four-rope grab, and improves the safety and efficiency of the operation.

CN118929450APending Publication Date: 2024-11-12HENAN MINE CRANE

Patent Information

Application Number
CN202411258374.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-09
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

During crane operation, the four-rope grab is susceptible to dynamic disturbances during operation, resulting in shaking, increasing the difficulty of load control and may cause safety accidents. The existing technology relies on operator experience and intuition to adjust, and labor intensity Large and difficult to ensure long-term, high-precision stable control.

Method used

Using the four-rope grab twist angle calculation method based on YOLOv5, through image acquisition, preprocessing, feature extraction, position positioning and coordinate conversion, the four-rope grab grab posture is monitored in real time and the twist angle is calculated, and the crane action is automatically adjusted to reduce the burden on the operator. And improve job safety.

Benefits of technology

Real-time stable control of four-rope grabs is achieved, the safety and efficiency of crane operations are improved, the labor intensity of the operator is reduced, and the stability of long-term and high-precision is ensured.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118929450A_ABST
    Figure CN118929450A_ABST
Patent Text Reader

Abstract

The embodiment of the invention discloses a crane four-rope grab bucket torsion angle calculation method and system based on YOL0v5, and the method comprises the following steps: firstly, triggering an image collection unit through a crane PLC control program, and capturing an image of a four-rope grab bucket in real time; then, preprocessing the image by using a lightweight YOLOv5 algorithm, and extracting feature information; then, recognizing grab bucket features in the image by using a YOLOv5 algorithm, and determining the position of the grab bucket features; then, the rotating coordinate value of the lifting appliance is calculated through coordinate transformation; and finally, calculating and outputting the torsional angle of the grab bucket according to the rotating coordinate value. According to the method, by optimizing the YOLOv5 algorithm and lightweight design, the grab bucket recognition precision and the system operation efficiency are improved, the torsional angle of the four-rope grab bucket can be output in real time through accurate coordinate transformation and rotation angle calculation, and accurate feedback and control basis is provided for crane operation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent lifting of a four-rope grab bucket of a crane, and in particular to a method for calculating the torsion angle of a four-rope grab bucket of a crane based on YOL0v5. Background Art

[0002] During crane operation, especially in material handling tasks involving four-rope grabs, ensuring load stability and safety is crucial. However, due to the complexity of the crane system and the various influencing factors of the external environment, the four-rope grab is often subject to the combined effects of multiple dynamic disturbances during operation, causing it to produce varying degrees of shaking and twisting in the horizontal (truck and trolley running direction) and vertical (lifting action) directions.

[0003] First of all, as the key component connecting the four-rope grab and the crane body, the elastic characteristics and dynamic response of the wire rope directly affect the stability of the grab. When the crane accelerates or decelerates the trolley or car, the four-rope grab will swing accordingly due to the inertia force. This swing not only increases the difficulty of load control, but may also cause safety accidents due to excessive swing amplitude.

[0004] Secondly, the lifting action of the four-rope grab is itself a complex dynamic process. During the lifting process, factors such as the weight, shape, and center of gravity of the load will affect the posture and stability of the grab. Especially in the early stage of lifting and when approaching the target height, the grab is more likely to experience unstable shaking and twisting due to speed changes or external interference.

[0005] In addition, external environmental factors such as wind, temperature changes, and friction between mechanical parts can also affect the stability of the four-rope grab. Wind force on the grab will generate additional torque, causing the grab to deviate from the predetermined trajectory; temperature changes may cause the wire rope to expand and contract, thereby affecting the hanging state of the grab; and friction between mechanical parts may produce irregular vibration and noise, further exacerbating the instability of the grab.

[0006] Faced with these challenges, operators rely mainly on personal experience and intuition to control the twisting of the four-rope grab. They try to keep the load stable by constantly adjusting the speed and direction of the trolley and car, as well as the lifting height and speed of the grab. However, this method is not only labor-intensive and skill-intensive, but also difficult to ensure long-term, high-precision stable control. More importantly, it limits the operating efficiency of the crane because the operator needs to spend a lot of time observing, judging and adjusting.

[0007] Therefore, the present invention provides a new solution to solve this problem. Summary of the invention

[0008] The main purpose of this application is to provide a method and system for calculating the torsion angle of a four-rope grab of a crane based on YOL0v5, aiming to be able to monitor the posture of the four-rope grab in real time, automatically calculate the torsion angle and adjust the crane action accordingly, thereby reducing the burden on the operator and improving operation safety.

[0009] To achieve the above purpose, the present application provides a method for calculating the torsion angle of a crane four-rope grab bucket based on YOL0v5, comprising the following steps:

[0010] S1, image acquisition: When the crane starts to operate, the crane PLC control program sends a trigger pulse to the image acquisition unit. After receiving the trigger pulse, the image acquisition unit starts the camera to collect the real-time posture image of the four-rope grab;

[0011] S2, image preprocessing: Use the lightweight YOLOv5 algorithm to preprocess the collected images to extract feature information in the images;

[0012] S3, grab bucket identification: the pre-processed image is subjected to feature extraction through the YOLOv5 algorithm to identify the four-rope grab bucket features in the image;

[0013] S4. Display the position of the grab bucket: The YOLOv5 algorithm analyzes and classifies the image and then locates it to determine the position of the four-rope grab bucket;

[0014] S5, coordinate transformation: according to the position of the spreader detected by YOLOv5, the coordinate value after rotation is calculated through the coordinate transformation between the pixel coordinate system and the world coordinate system;

[0015] S6. Output torsion angle: calculate the rotation angle of the rotated coordinate value and output the torsion angle.

[0016] Optionally, in step S2, the preprocessing needs to convert the image data from floating point numbers into quantized values, and the specific numerical formula after quantization is:

[0017]

[0018]

[0019] In the formula: Q is the specific value after quantization; R is the weight parameter value; Z is the zero point value; xf is the input data; n is the number of bits of the value after quantization processing.

[0020] Optionally, the feature extraction in step S3 specifically includes the following steps:

[0021] S3.1. Depthwise Convolution:

[0022] Applying a convolution kernel to each input channel results in the same number of output channels, but the feature map size of each output channel is the same as the input;

[0023] S3.2, point-by-point convolution:

[0024] Perform 1x1 point-by-point convolution on the result of the depthwise separable convolution to obtain a feature map of the same dimension as the input feature map;

[0025] S3.3. Add the result of point-by-point convolution to the original feature map to enhance feature representation:

[0026]

[0027] Where: W C , H C are the width and height of the input feature map respectively; N is the number of channels of the convolution kernel; M is the number of convolution kernels; H O is the height of the convolution module; W O is the width of the convolution module.

[0028] Optionally, in step S4, a pruning design judgment function is used to optimize the network structure, and the specific algorithm is:

[0029]

[0030] Where: N is the number of channels after network pruning; x is the number of channels of the original network convolution kernel; f(x) is the maximum value that is less than x and a multiple of 9 after pruning.

[0031] Optionally, the specific operation method of coordinate conversion in step S5 is as follows:

[0032] S5.1, obtain the detection result of the four-rope grab bucket from the YOLOv5 algorithm;

[0033] S5.2. Convert the coordinates in the camera pixel coordinate system to the coordinates in the world coordinate system. The formula is as follows:

[0034]

[0035] Where: Zc is the distance between the camera and the object, specifically the length of the four-rope grab rope;

[0036] is the pixel coordinate; is the internal parameter matrix;

[0037] is the external parameter matrix; is the world coordinate;

[0038] S5.3. Coordinate calculation after rotation

[0039] Determine the angle θ that the image needs to be rotated, usually with the counterclockwise direction as the positive direction. For any point (x 0 ,y 0 ), which rotates around the center (x c ,y c )The new coordinates (x′, y′) after rotation by angle θ can be calculated by the following formula:

[0040] Clockwise rotation:

[0041] x′=(x 0 -x c )*cosθ+(y 0 -y c )*sinθ+x c

[0042] y′=(x 0 -x c )*sinθ+(y 0 -y c )*cosθ+y c

[0043] Counterclockwise rotation:

[0044] x′=(x 0 -x c )*cosθ-(y 0 -y c )*sinθ+x c

[0045] y′=(x 0 -x c )*sinθ+(y 0 -y c )*cosθ+y c

[0046] In the above, the rotation angle θ is in radians.

[0047] Optionally, the specific calculation process of the rotation angle θ in step S6 is as follows:

[0048] S6.1, Feature point pair recognition:

[0049] Determine the two characteristic points A(x 1 ,y 1 ) and B(x 2 ,y 2 );

[0050] S6.2. Calculation of feature point coordinates after rotation:

[0051] Apply the coordinate rotation transformation in step S5 to feature points A and B to obtain the rotated coordinates A′(x′ 1 , y′ 1 ) and B′(x′ 2 , y′ 2 );

[0052] S6.3. By vector and The angle between them is used to calculate the rotation angle θ:

[0053]

[0054] S6.4. Output the calculated torsion angle θ as the rotation angle of the four-rope grab.

[0055] Optionally, the feature extraction network of the lightweight YOLOv5 algorithm uses a Conv convolution module and MobileNetv2 as a Backbone.

[0056] Optionally, the neck network of the lightweight YOLOv5 algorithm adopts an FPN+PAN feature pyramid structure to transmit feature information of targets of different sizes.

[0057] A crane four-rope grab torsion angle calculation system based on YOL0v5, the system comprising:

[0058] A CCD industrial camera, mounted on the trolley platform of the crane, for capturing real-time images of the four-rope grab;

[0059] An image acquisition card is used to transmit the image data captured by the CCD industrial camera to an industrial computer;

[0060] A rotary encoder for measuring the length of the lifting rope of the four-rope grab;

[0061] A PLC for controlling the operation of the crane and the triggering of the image acquisition card; and

[0062] A marker is disposed on the top of the four-rope grab and is located directly below the CCD industrial camera, and is used as a reference for assisting image recognition and coordinate positioning.

[0063] Optionally, the marker is an infrared structured light source or a circular reflector.

[0064] The method and system for calculating the torsion angle of a four-rope grab of a crane based on YOLOv5 proposed in the embodiment of the present application, on the one hand, adopts a lightweight YOLOv5 algorithm, optimizes the feature extraction network, uses MobileNetv2 as the backbone, and uses the FPN+PAN feature pyramid structure, effectively improving the recognition accuracy of the four-rope grab in the image, especially performing well in small target detection tasks, and significantly improving the overall detection performance of the system; on the other hand, through lightweight design, including quantized model processing, using Ghost convolution to reduce the amount of calculation, and network pruning technology, the model complexity and computing resource requirements are greatly reduced, so that the system can run efficiently on resource-constrained embedded devices, improving real-time performance and system throughput. Based on the position of the spreader detected by YOLOv5, the torsion angle of the four-rope grab can be output in real time through precise coordinate transformation and rotation angle calculation, providing accurate feedback and control basis for crane operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and the embodiments in the drawings do not constitute any limitation to the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0066] Figure 1 A schematic flow chart of a method for calculating the torsion angle of a four-rope grab bucket of a crane based on YOL0v5 provided in one embodiment of the present application;

[0067] Figure 2 A schematic diagram of a lightweight YOLOv5 algorithm network structure provided in an embodiment of the present application;

[0068] Figure 3 is a schematic diagram of the transformation between a pixel coordinate system and a world coordinate system provided in an embodiment of the present application;

[0069] Figure 4 It is a structural schematic diagram of a crane four-rope grab torsion angle calculation system based on YOL0v5 provided in another embodiment of the present application;

[0070] Figure 5 This is a schematic diagram of marker image acquisition provided in another embodiment of the present application.

[0071] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0072] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0073] The above and other technical contents, features and effects of the present invention are described in detail below with reference to the attached Figure 1-5 The detailed description of the embodiments will clearly show that the structural contents mentioned in the following embodiments are all based on the drawings in the specification.

[0074] Exemplary embodiments of the present invention will be described below with reference to the accompanying drawings.

[0075] The main solutions of the embodiments of this application are:

[0076] like Figure 1 As shown in FIG. 1 , the torsion angle calculation method of the crane four-rope grab bucket based on YOL0v5 includes the following steps:

[0077] S1, image acquisition: When the crane starts to operate, the crane PLC control program sends a trigger pulse to the image acquisition unit. After receiving the trigger pulse, the image acquisition unit starts the camera to collect the real-time posture image of the four-rope grab;

[0078] S2, image preprocessing: Use the lightweight YOLOv5 algorithm to preprocess the collected images to extract feature information in the images;

[0079] S3, grab bucket identification: the pre-processed image is subjected to feature extraction through the YOLOv5 algorithm to identify the four-rope grab bucket features in the image;

[0080] S4. Display the position of the grab bucket: The YOLOv5 algorithm analyzes and classifies the image and then locates it to determine the position of the four-rope grab bucket;

[0081] S5, coordinate transformation: according to the position of the spreader detected by YOLOv5, the coordinate value after rotation is calculated through the coordinate transformation between the pixel coordinate system and the world coordinate system;

[0082] S6. Output torsion angle: calculate the rotation angle of the rotated coordinate value and output the torsion angle.

[0083] In the above, the feature extraction network of the lightweight YOLOv5 algorithm uses the Conv convolution module to replace the original Focus module to reduce the amount of calculation; MobileNetv2 is used as the backbone, and MobileNetv2 uses depthwise separable convolution. This structure decomposes the standard convolution into depth convolution and point-by-point convolution, which significantly reduces the amount of calculation and model size while maintaining high accuracy.

[0084] The neck network of the lightweight YOLOv5 algorithm uses the FPN+PAN feature pyramid structure to transmit the feature information of objects of different sizes. Among them, FPN (Feature Pyramid Network) fuses features of different levels through bottom-up, top-down and lateral connections to construct a feature pyramid with rich semantic information, which helps the model detect objects of different sizes. Figure 2 As shown in the figure, PAN (Path Aggregation Network) further enhances the lateral connection of features based on FPN. Through bottom-up path enhancement, low-level features can be better transmitted to high-level levels, promoting the flow of information between different levels, especially helping to improve the detection performance of small targets.

[0085] The lightweight YOLOv5 algorithm achieves lightweight model while maintaining or improving detection accuracy as much as possible by optimizing the feature extraction network, adopting a more efficient Backbone, and improving the neck network structure. It performs well in handling small target detection tasks, especially.

[0086] In the above, in step S2, the image data needs to be converted from floating point numbers to quantized values ​​for preprocessing. The specific numerical formula after quantization is:

[0087]

[0088]

[0089] In the formula: Q is the specific value after quantization; R is the weight parameter value; Z is the zero point value; xf is the input data; n is the number of bits of the value after quantization processing.

[0090] In the actual process, by applying different quantization models, the 8-bit fixed-point quantization model becomes significantly smaller. Converting floating-point tensors into unit8 tensors can reduce storage space and memory bandwidth, further improve system throughput, and alleviate possible delays caused by the system.

[0091] To further improve the efficiency of the network, the lightweight YOLOv5 algorithm uses Ghost Conv. Ghost Conv generates additional "ghost" feature maps (Ghost Feature Maps) at a lower computational cost by adding a series of linear transformations after a small number of convolutions, thereby enriching the number of features that the network can learn.

[0092] During the feature extraction process, the lightweight YOLOv5 algorithm uses the Conv convolution module to replace the original YOLOv5 Focus module, and uses MobileNetv2 to replace the original CSP module in Backbone.

[0093] The feature extraction in step S3 specifically includes the following steps:

[0094] S3.1. Depthwise Convolution:

[0095] Applying a convolution kernel to each input channel results in the same number of output channels, but the feature map size of each output channel is the same as the input;

[0096] S3.2, point-by-point convolution:

[0097] Perform 1x1 point-by-point convolution on the result of the depthwise separable convolution to obtain a feature map of the same dimension as the input feature map;

[0098] S3.3. Add the result of point-by-point convolution to the original feature map to enhance feature representation:

[0099]

[0100] Where: W C , H C are the width and height of the input feature map respectively; N is the number of channels of the convolution kernel; M is the number of convolution kernels; H O is the height of the convolution module; W O is the width of the convolution module.

[0101] After calculation, it can be seen that the size of the output feature map is Wc*Hc*N.

[0102] Based on feature extraction, the YOLOv5 algorithm uses a convolutional neural network (CNN) to classify objects in an image. In the field of convolutional neural network compression, network pruning is a technical means to effectively reduce network complexity, alleviate overfitting, and improve the inference speed of quantization algorithms. The structured pruning strategy of convolution kernels focuses on the convolution kernel level. It trains the network normally in advance, then identifies and removes those convolution kernels or channels whose weight links are lower than the preset threshold, and supplements them with sparsity adjustment measures to achieve lightweight models. Specifically, first, a sparsity regularization term (such as L1 regularization) is introduced during the training process to encourage the model weights to become sparse. Then, after the training is completed, unimportant connections or parameters are pruned according to the sparsity of the weights.

[0103] In the specific implementation process, the pruning design judgment function is used in step S4 to optimize the network structure. The specific algorithm is:

[0104]

[0105] Where: N is the number of channels after network pruning; x is the number of channels of the original network convolution kernel; f(x) is the maximum value that is less than x and a multiple of 9 after pruning.

[0106] According to the pruning judgment function in formula (4), the number of channels is adjusted to meet the optimization requirements of the embedded platform while being as close as possible to the number of channels of the original network convolution kernel to reduce information loss. This method can gradually guide the model to develop a sparse structure during the training process, so that the performance of the model can be better maintained after pruning.

[0107] In order to improve the prediction speed and efficiency of the network, in the DenseNet network of the YOLOv5 algorithm, in addition to adjusting the number of channels to optimize performance, a one-time aggregation module can be used to remove dense connections and aggregate shallow features in the last layer. This improvement can reduce the amount of calculation, improve prediction speed and efficiency, while retaining the powerful feature extraction capabilities of DenseNet. This improvement may be particularly effective for small target detection tasks on four-rope grabs because it allows the network to still capture important feature information with fewer computing resources.

[0108] System test: In the development process of the crane four-rope grab torsion angle calculation system based on YOLOv5, the system test used the PyTorch framework for model training and verification. Considering that the model will eventually need to be deployed on resource-constrained embedded devices, the hardware environment was specially configured through the RKNN framework. In order to achieve cross-platform compilation management, the development team installed the cross-compilation environment on the PC terminal and remotely logged in to the mobile device through the SSH protocol for operation. The CMakeLists.txt configuration file written using the CMake tool ensures the efficiency and consistency of the compilation process. During the development process, CLion, as the main integrated development environment, is not only used for writing and training Python code, but also for converting the trained model into a C++ program to run on the target hardware.

[0109] The performance of the system is quantified by the mAP evaluation index. During the model training process, the precision and recall rates are strictly controlled within the range of 0 to 1, and the final evaluation results are obtained by summing the integrals. In order to ensure the reliability of the results, the entire experiment was repeated 5 times to obtain the average value for comparative analysis.

[0110] Preprocessing of the dataset is another key step in system development. The PASCAL VOC dataset, which is widely used in object detection tasks, was selected. This dataset contains rich and diverse samples and background features, which is very suitable for training and testing deep learning models. The label format of the dataset has been converted to adapt to the training requirements of YOLOv5. Anchors are generated by clustering methods, and image enhancement techniques such as brightness, saturation, contrast adjustment and geometric transformation are used to improve the model's adaptability to different environments. The application of Mosaic data enhancement technology further improves the generalization of the model and effectively prevents overfitting.

[0111] Detection effect: The performance of the YOLOv5 algorithm in the small target detection task has been verified. The experimental results show that the system's small target detection ability has been significantly improved in terms of mAP@0.5 and mAP@0.50.96 indicators, among which the WCAL-PAN method has improved by 3.3% in mAP@0.5 and 5.2% in mAP@0.5:0.95. Overall, the designed system has improved the small target detection effect by 2.7%, which confirms the effectiveness and reliability of the system in practical applications.

[0112] According to the position of the sling detected by YOLOv5, the actual twisting amount of the sling can be calculated through the coordinate transformation between the pixel coordinate system and the world coordinate system, and then the twisting angle can be calculated according to the length of the sling rope. Figure 3It is a schematic diagram of the transformation between the pixel coordinate system and the world coordinate system. In the figure, Ow-XwYwZw is the world coordinate system, O-XcYcZc is the camera coordinate system, Od-uv is the pixel coordinate system, and O1-XY is the image coordinate system.

[0113] In the specific implementation process, the specific operation method of coordinate conversion in step S5 is as follows:

[0114] S5.1, obtain the detection result of the four-rope grab bucket from the YOLOv5 algorithm;

[0115] S5.2. Convert the coordinates in the camera pixel coordinate system to the coordinates in the world coordinate system. The formula is as follows:

[0116]

[0117] Where: Zc is the distance between the camera and the object, specifically the length of the four-rope grab rope;

[0118] is the pixel coordinate; is the internal parameter matrix;

[0119] is the external parameter matrix; is the world coordinate;

[0120] S5.3. Coordinate calculation after rotation

[0121] Determine the angle θ that the image needs to be rotated, usually with the counterclockwise direction as the positive direction. For any point (x 0 ,y 0 ), which rotates around the center (x c ,y c )The new coordinates (x′, y′) after rotation by angle θ can be calculated by the following formula:

[0122] Clockwise rotation:

[0123] x′=(x 0 -x c )*cosθ+(y 0 -y c )*sinθ+x c

[0124] y′=(x 0 -x c )*sinθ+(y 0 -y c )*cosθ+y c

[0125] Counterclockwise rotation:

[0126] x′=(x 0-x c )*cosθ-(y 0 -y c )*sinθ+x c

[0127] y′=(x 0 -x c )*sinθ+(y 0 -y c )*cosθ+y c

[0128] In the above, the rotation angle θ is in radians.

[0129] The specific calculation process of the rotation angle θ in step S6 is as follows:

[0130] S6.1, Feature point pair recognition:

[0131] Determine the two characteristic points A(x 1 ,y 1 ) and B(x 2 ,y 2 );

[0132] S6.2. Calculation of feature point coordinates after rotation:

[0133] Apply the coordinate rotation transformation in step S5 to feature points A and B to obtain the rotated coordinates A′(x′ 1 , y′ 1 ) and B′(x′ 2 , y′ 2 );

[0134] S6.3. By vector and The angle between them is used to calculate the rotation angle θ:

[0135]

[0136] S6.4. Output the calculated torsion angle θ as the rotation angle of the four-rope grab.

[0137] In implementing the above steps S1 to S6, on the one hand, the lightweight YOLOv5 algorithm is used to effectively improve the recognition accuracy of the four-rope grab in the image by optimizing the feature extraction network, using MobileNetv2 as the Backbone and the FPN+PAN feature pyramid structure, especially in the small target detection task, which significantly improves the overall detection performance of the system; on the other hand, through lightweight design, including quantized model processing, using Ghost convolution to reduce the amount of calculation, and network pruning technology, the model complexity and computing resource requirements are greatly reduced, so that the system can run efficiently on resource-constrained embedded devices, improving real-time performance and system throughput. Based on the position of the spreader detected by YOLOv5, the torsion angle of the four-rope grab can be output in real time through precise coordinate transformation and rotation angle calculation, providing accurate feedback and control basis for crane operations.

[0138] After introducing the method of the exemplary embodiment of the present application, the following is a reference to Figure 4 , Figure 4 This is a schematic diagram of the structure of a crane four-rope grab torsion angle calculation system based on YOL0v5 provided in one embodiment of the present application. Figure 4 As shown, the crane four-rope grab torsion angle calculation system based on YOL0v5 includes:

[0139] A CCD industrial camera, mounted on the trolley platform of the crane, for capturing real-time images of the four-rope grab;

[0140] Image acquisition card, used to transmit image data captured by CCD industrial camera to industrial computer;

[0141] A rotary encoder for measuring the length of the lifting rope of the four-rope grab;

[0142] A PLC for controlling the operation of the crane and the triggering of the image acquisition card; and

[0143] The marker is set on the top of the four-rope grab and is located directly below the CCD industrial camera. It is used as a reference to assist image recognition and coordinate positioning.

[0144] In the specific working process, the CCD industrial camera collects image data from the top of the four-rope grab, and then the image acquisition card quickly and accurately transmits the image data captured by the CCD industrial camera to the industrial computer. The YOLOv5 algorithm is run on the industrial computer to process and identify the transmitted images, so as to quickly and accurately identify the markers in the image and locate the markers. The existence of markers is crucial for image recognition and coordinate positioning. They serve as reference objects to help the system identify and locate the grab more accurately. In this system, the markers can be infrared structured light sources or circular reflectors, such as Figure 5When the infrared structured light source is used as a marker, it can avoid interference with the operator because it emits invisible infrared light, while ensuring that the camera can clearly capture the posture of the grab, so it is suitable for application scenarios with complex lighting environments. The circular reflector forms a clear mark in the image by reflecting light, assisting the system to quickly and accurately locate, so it is suitable for simple lighting environments.

[0145] The torsion angle results calculated by the system will be output to the display screen or control system in real time for the operator's reference and decision-making. The system is also equipped with a human-computer interaction interface, through which the operator can view the torsion angle calculation results, monitor the crane's operating status and send control instructions. The YOLOv5-based crane four-rope grab torsion angle calculation system integrates high-precision CCD industrial cameras, fast image transmission technology, advanced YOLOv5 image recognition algorithms and flexible marker design to achieve real-time and accurate calculation of the crane's four-rope grab torsion angle, and achieve high-efficiency, high-precision and high-stability operation monitoring, significantly improving the safety and efficiency of crane operations.

[0146] Finally, it should be noted that the above embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

[0147] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

Claims

1. A method for calculating the torsion angle of a crane four-rope grab bucket based on YOL0v5, characterized in that: The steps include: S1, image acquisition: When the crane starts to operate, the crane PLC control program sends a trigger pulse to the image acquisition unit. After receiving the trigger pulse, the image acquisition unit starts the camera to collect the real-time posture image of the four-rope grab; S2, image preprocessing: Use the lightweight YOLOv5 algorithm to preprocess the collected images to extract feature information in the images; S3, grab bucket identification: the pre-processed image is subjected to feature extraction through the YOLOv5 algorithm to identify the four-rope grab bucket features in the image; S4. Display the position of the grab bucket: The YOLOv5 algorithm analyzes and classifies the image and then locates it to determine the position of the four-rope grab bucket; S5, coordinate transformation: according to the position of the spreader detected by YOLOv5, the coordinate value after rotation is calculated through the coordinate transformation between the pixel coordinate system and the world coordinate system; S6. Output torsion angle: calculate the rotation angle of the rotated coordinate value and output the torsion angle.

2. The method for calculating the torsion angle of a crane four-rope grab bucket based on YOL0v5 according to claim 1 is characterized in that: In step S2, the image data needs to be converted from floating point numbers to quantized values ​​for preprocessing. The specific numerical formula after quantization is: In the formula: Q is the specific value after quantization; R is the weight parameter value; Z is the zero point value; xf is the input data; n is the number of bits of the value after quantization processing.

3. The method for calculating the torsion angle of a crane four-rope grab bucket based on YOL0v5 according to claim 2 is characterized in that: The feature extraction in step S3 specifically includes the following steps: S3.

1. Depthwise Convolution: Applying a convolution kernel to each input channel results in the same number of output channels, but the feature map size of each output channel is the same as the input; S3.2, point-by-point convolution: Perform 1x1 point-by-point convolution on the result of the depthwise separable convolution to obtain a feature map of the same dimension as the input feature map; S3.

3. Add the result of point-by-point convolution to the original feature map to enhance feature representation: Where: W C , H C are the width and height of the input feature map respectively; N is the number of channels of the convolution kernel; M is the number of convolution kernels; H O is the height of the convolution module; W O is the width of the convolution module.

4. The method for calculating the torsion angle of a crane four-rope grab bucket based on YOL0v5 according to claim 3 is characterized in that: In step S4, the pruning design judgment function is used to optimize the network structure, and the specific algorithm is: Where: N is the number of channels after network pruning; x is the number of channels of the original network convolution kernel; f(x) is the maximum value that is less than x and a multiple of 9 after pruning.

5. The method for calculating the torsion angle of a crane four-rope grab bucket based on YOL0v5 according to claim 4 is characterized in that: The specific operation method of coordinate conversion in step S5 is as follows: S5.1, obtain the detection result of the four-rope grab bucket from the YOLOv5 algorithm; S5.

2. Convert the coordinates in the camera pixel coordinate system to the coordinates in the world coordinate system. The formula is as follows: Where: Zc is the distance between the camera and the object, specifically the length of the four-rope grab rope; is the pixel coordinate; is the internal parameter matrix; is the external parameter matrix; is the world coordinate; S5.

3. Coordinate calculation after rotation Determine the angle θ that the image needs to be rotated. Usually, the counterclockwise direction is the positive direction. For any point (x0, y0) on the image, the angle θ around the rotation center (x c ,y c )The new coordinates (x′, y′) after rotation by angle θ can be calculated by the following formula: Clockwise rotation: x′=(x0-x c )*cosθ+(y0-y c )*sinθ+x c y′=(x0-x c )*sinθ+(y0-y c )*cosθ+y c Counterclockwise rotation: x′=(x0-x c )*cosθ-(y0-y c )*sinθ+x c y′=(x0-x c )*sinθ+(y0-y c )*cosθ+y c In the above, the rotation angle θ is in radians.

6. The method for calculating the torsion angle of a crane four-rope grab bucket based on YOL0v5 according to claim 5 is characterized in that: The specific calculation process of the rotation angle θ in step S6 is as follows: S6.1, Feature point pair recognition: Determine two feature points A (x1, y1) and B (x2, y2) for calculating the torsion angle; S6.

2. Calculation of feature point coordinates after rotation: Apply the coordinate rotation transformation in step S5 to the feature points A and B to obtain the rotated coordinates A′(x′1, y′1) and B′(x′2, y′2); S6.

3. By vector and The angle between them is used to calculate the rotation angle θ: S6.

4. Output the calculated torsion angle θ as the rotation angle of the four-rope grab.

7. The method for calculating the torsion angle of a crane four-rope grab bucket based on YOL0v5 according to claim 1 is characterized in that: The feature extraction network of the lightweight YOLOv5 algorithm uses the Conv convolution module and MobileNetv2 as the backbone.

8. The method for calculating the torsion angle of a crane four-rope grab bucket based on YOL0v5 according to claim 7 is characterized in that: The neck network of the lightweight YOLOv5 algorithm adopts the FPN+PAN feature pyramid structure to transmit feature information of objects of different sizes.

9. The crane four-rope grab torsion angle calculation system based on YOL0v5 is characterized by: The system comprises: A CCD industrial camera, mounted on the trolley platform of the crane, for capturing real-time images of the four-rope grab; An image acquisition card is used to transmit the image data captured by the CCD industrial camera to an industrial computer; A rotary encoder for measuring the length of the lifting rope of the four-rope grab; A PLC for controlling the operation of the crane and the triggering of the image acquisition card; and A marker is disposed on the top of the four-rope grab and is located directly below the CCD industrial camera, and is used as a reference for assisting image recognition and coordinate positioning.

10. The crane four-rope grab torsion angle calculation system based on YOL0v5 according to claim 9 is characterized in that: The marker is an infrared structured light source or a circular reflector.

Citation Information

Patent Citations

  • Automatic anti-twist system and method for port crane spreader

    CN108892043A

  • Automatic container grabbing and releasing system based on machine vision and operation method

    CN113879984A

  • Method and device for determining inclination angle of lifting hook, hoisting machinery and readable storage medium

    CN117720009A

  • YOL0v5-based crane four-rope grab bucket swing angle calculation method and system

    CN118929451A

  • System for shape recognition based image processing

    KR102102471B1

Cited By

  • Crane grab bucket steel wire rope deviation rectification control method and system

    CN119929670A