YOL0v5-based crane four-rope grab bucket swing angle calculation method and system
Through the YOLOv5 algorithm, the swing angle of the four-rope grab is monitored and calculated in real time, and the crane movement is automatically adjusted, which solves the problem of swinging and unstable four-rope grab in crane operation, and improves the operation safety and efficiency.
Patent Information
- Application Number
- CN202411258375.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-09
- Publication Date
- 2024-11-12
AI Technical Summary
During crane operation, the four-rope grab is susceptible to dynamic disturbances during operation, resulting in swing and unstable, increasing the difficulty of load control and may cause safety accidents. The operator relies on experience and intuition to make adjustments, and the labor intensity is high And it is difficult to ensure long-term and high-precision stable control.
Using the four-rope grab swing angle calculation method based on YOLOv5, the swing angle of the four-rope grab is monitored in real time through image acquisition, preprocessing, feature extraction and posture analysis, and the swing angle is calculated based on the length and offset of the rope, and the swing angle is calculated automatically. Adjust the crane movements to reduce the burden on the operator and improve operational safety.
Real-time attitude monitoring and automatic adjustment of the four-rope grab is realized, which improves the safety and efficiency of crane operations, reduces the labor intensity and skill requirements of the operator, and ensures long-term and high-precision stable control.
Smart Images

Figure CN118929451A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of intelligent lifting of a four-rope grab bucket of a crane, and in particular to a method for calculating a swing angle of a four-rope grab bucket of a crane based on YOL0v5. Background Art
[0002] During crane operation, especially in material handling tasks involving four-rope grabs, ensuring load stability and safety is crucial. However, due to the complexity of the crane system and the various influencing factors of the external environment, the four-rope grab is often subject to the combined effects of multiple dynamic disturbances during operation, causing it to produce varying degrees of shaking and twisting in the horizontal (truck and trolley running direction) and vertical (lifting action) directions.
[0003] First of all, as the key component connecting the four-rope grab and the crane body, the elastic characteristics and dynamic response of the wire rope directly affect the stability of the grab. When the crane accelerates or decelerates the trolley or car, the four-rope grab will swing accordingly due to the inertia force. This swing not only increases the difficulty of load control, but may also cause safety accidents due to excessive swing amplitude.
[0004] Secondly, the lifting action of the four-rope grab is itself a complex dynamic process. During the lifting process, factors such as the weight, shape, and center of gravity of the load will affect the posture and stability of the grab. Especially in the early stage of lifting and when approaching the target height, the grab is more likely to experience unstable shaking and twisting due to speed changes or external interference.
[0005] In addition, external environmental factors such as wind, temperature changes, and friction between mechanical parts can also affect the stability of the four-rope grab. Wind force on the grab will generate additional torque, causing the grab to deviate from the predetermined trajectory; temperature changes may cause the wire rope to expand and contract, thereby affecting the hanging state of the grab; and friction between mechanical parts may produce irregular vibration and noise, further exacerbating the instability of the grab.
[0006] Faced with these challenges, operators rely mainly on personal experience and intuition to control the swing of the four-rope grab. They try to keep the load stable by constantly adjusting the speed and direction of the trolley and car, as well as the lifting height and speed of the grab. However, this method is not only labor-intensive and skill-intensive, but also difficult to ensure long-term, high-precision stable control. More importantly, it limits the operating efficiency of the crane because the operator needs to spend a lot of time observing, judging and adjusting.
[0007] Therefore, the present invention provides a new solution to solve this problem. Summary of the invention
[0008] The main purpose of this application is to provide a method and system for calculating the swing angle of a four-rope grab bucket of a crane based on YOL0v5, aiming to be able to monitor the posture of the four-rope grab bucket in real time, automatically calculate the swing angle and adjust the crane action accordingly, thereby reducing the burden on the operator and improving operation safety.
[0009] To achieve the above purpose, the present application provides a method for calculating the swing angle of a crane four-rope grab bucket based on YOL0v5, comprising the following steps:
[0010] S1, image acquisition: When the crane starts to operate, the crane PLC control program sends a trigger pulse to the image acquisition unit. After receiving the trigger pulse, the image acquisition unit starts the camera to collect the real-time posture image of the four-rope grab;
[0011] S2, image preprocessing: Use the lightweight YOLOv5 algorithm to preprocess the collected images to extract feature information in the images;
[0012] S3, grab bucket identification: the pre-processed image is subjected to feature extraction through the YOLOv5 algorithm to identify the four-rope grab bucket features in the image;
[0013] S4. Display the position of the grab bucket: The YOLOv5 algorithm analyzes and classifies the image and then locates it to determine the position of the four-rope grab bucket;
[0014] S5. Swing calculation: According to the position of the spreader detected by YOLOv5, the swing amounts Δx and Δy along the X-axis and Y-axis directions are calculated through coordinate conversion between the pixel coordinate system and the world coordinate system:
[0015] Δx=x w -x0, Δy=y w -y0
[0016] Among them, x0 and y0 represent the coordinates of the starting position on the X-axis and Y-axis; w ,y w Represents the coordinates of the current position on the X-axis and Y-axis;
[0017] S6. Output swing angle: Calculate the swing angle α according to the swing amount and the length of the hanging rope of the four-rope grab:
[0018]
[0019] Among them, l is the length of the suspension rope, and d is the horizontal displacement of the suspension rope.
[0020] Optionally, in step S2, the image data needs to be preprocessed from floating point numbers to quantized values, and the specific numerical formula after quantization is:
[0021]
[0022]
[0023] In the formula: Q is the specific value after quantization; R is the weight parameter value; Z is the zero point value; xf is the input data; n is the number of bits of the value after quantization processing.
[0024] Optionally, the feature extraction in step S3 specifically includes the following steps:
[0025] S3.1. Depthwise Convolution:
[0026] Applying a convolution kernel to each input channel results in the same number of output channels, but the feature map size of each output channel is the same as the input;
[0027] S3.2, point-by-point convolution:
[0028] Perform 1x1 point-by-point convolution on the result of the depthwise separable convolution to obtain a feature map of the same dimension as the input feature map;
[0029] S3.3. Add the result of point-by-point convolution to the original feature map to enhance feature representation:
[0030]
[0031] Where: W C , H C are the width and height of the input feature map respectively; N is the number of channels of the convolution kernel; M is the number of convolution kernels; H O is the height of the convolution module; W O is the width of the convolution module.
[0032] Optionally, in step S4, a pruning design judgment function is used to optimize the network structure, and the specific algorithm is:
[0033]
[0034] Where: N is the number of channels after network pruning; x is the number of channels of the original network convolution kernel; f(x) is the maximum value that is less than x and a multiple of 9 after pruning.
[0035] Optionally, the specific operation method of coordinate conversion in step S5 is as follows:
[0036] Get the detection results of the four-rope grab from the YOLOv5 algorithm;
[0037] The coordinates in the camera pixel coordinate system are converted to the coordinates in the world coordinate system. The formula is as follows:
[0038]
[0039] Where: Zc is the distance between the camera and the object, specifically the length of the four-rope grab rope;
[0040] is the pixel coordinate; is the internal parameter matrix;
[0041] is the external parameter matrix; is the world coordinate.
[0042] Optionally, the coordinate transformation process uses the distortion coefficient obtained by camera calibration to correct radial distortion and tangential distortion in the image. The specific operation steps are as follows:
[0043] Obtaining the distortion coefficient of the camera using a camera calibration method;
[0044] Convert the detected pixel coordinates (u, v) to the coordinates (x d ,y d );
[0045] Use the distortion correction formula to correct (x d ,y d ) to obtain the corrected coordinates (x′, y′).
[0046] Optionally, the distortion correction formula is:
[0047] Radial distortion:
[0048] x′=x(1+k1r 2 +k2r 4 +k3r 6 )
[0049] y′=y(1+k1r 2 +k2r 4 +k3r 6 )
[0050] Among them, r 2 =x 2 +y 2 , (x, y) is the coordinate of the distortion point, (x′, y′) is the coordinate after correction, k1, k2, k3 are the radial distortion coefficients;
[0051] Tangential distortion:
[0052] x′=x+[2p1y+p2(r 2 +2x 2 )
[0053] y′=y+[2p1x+p2(r 2 +2x 2 )
[0054] Among them, p1 and p2 are tangential distortion coefficients.
[0055] Optionally, the feature extraction network of the lightweight YOLOv5 algorithm uses a Conv convolution module and MobileNetv2 as a Backbone; the neck network of the lightweight YOLOv5 algorithm uses an FPN+PAN feature pyramid structure to transmit feature information of targets of different sizes.
[0056] A crane four-rope grab bucket swing angle calculation system based on YOL0v5, the system comprising:
[0057] A CCD industrial camera, mounted on the trolley platform of the crane, for capturing real-time images of the four-rope grab;
[0058] An image acquisition card is used to transmit the image data captured by the CCD industrial camera to an industrial computer;
[0059] A rotary encoder for measuring the length of the lifting rope of the four-rope grab;
[0060] A PLC for controlling the operation of the crane and the triggering of the image acquisition card; and
[0061] A marker is disposed on the top of the four-rope grab and is located directly below the CCD industrial camera, and is used as a reference for assisting image recognition and coordinate positioning.
[0062] Optionally, the marker is an infrared structured light source or a circular reflector.
[0063] The method and system for calculating the swing angle of a four-rope grab bucket of a crane based on YOLOv5 proposed in the embodiment of the present application, on the one hand, adopts a lightweight YOLOv5 algorithm, optimizes the feature extraction network, uses MobileNetv2 as the backbone, and uses the FPN+PAN feature pyramid structure, effectively improves the recognition accuracy of the four-rope grab bucket in the image, especially performs well in the small target detection task, and significantly improves the overall detection performance of the system; on the other hand, through lightweight design, including quantization model processing, using Ghost convolution to reduce the amount of calculation, and network pruning technology, the model complexity and computing resource requirements are greatly reduced, so that the system can run efficiently on resource-constrained embedded devices, improving real-time performance and system throughput. Based on the position of the spreader detected by YOLOv5, through precise coordinate transformation and distortion correction processing, the swing angle of the four-rope grab bucket can be output in real time, providing accurate feedback and control basis for crane operations. BRIEF DESCRIPTION OF THE DRAWINGS
[0064] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and the embodiments in the drawings do not constitute any limitation to the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0065] Figure 1 A schematic diagram of a flow chart of a method for calculating the swing angle of a four-rope grab bucket of a crane based on YOL0v5 provided in an embodiment of the present application;
[0066] Figure 2 A schematic diagram of the swing amount calculation principle provided in an embodiment of the present application;
[0067] Figure 3 A schematic diagram of a lightweight YOLOv5 algorithm network structure provided in an embodiment of the present application;
[0068] Figure 4 is a schematic diagram of the transformation between a pixel coordinate system and a world coordinate system provided in an embodiment of the present application;
[0069] Figure 5 It is a structural schematic diagram of a crane four-rope grab bucket swing angle calculation system based on YOL0v5 provided in another embodiment of the present application;
[0070] Figure 6 This is a schematic diagram of marker image acquisition provided in another embodiment of the present application.
[0071] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0072] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.
[0073] The above and other technical contents, features and effects of the present invention are described in detail below with reference to the attached Figure 1-6 The detailed description of the embodiments will clearly show that the structural contents mentioned in the following embodiments are all based on the drawings in the specification.
[0074] Exemplary embodiments of the present invention will be described below with reference to the accompanying drawings.
[0075] The main solutions of the embodiments of this application are:
[0076] like Figure 1As shown in FIG. 1 , the method for calculating the swing angle of a crane four-rope grab bucket based on YOL0v5 includes the following steps:
[0077] S1, image acquisition: When the crane starts to operate, the crane PLC control program sends a trigger pulse to the image acquisition unit. After receiving the trigger pulse, the image acquisition unit starts the camera to collect the real-time posture image of the four-rope grab;
[0078] S2, image preprocessing: Use the lightweight YOLOv5 algorithm to preprocess the collected images to extract feature information in the images;
[0079] S3, grab bucket identification: the pre-processed image is subjected to feature extraction through the YOLOv5 algorithm to identify the four-rope grab bucket features in the image;
[0080] S4. Display the position of the grab bucket: The YOLOv5 algorithm analyzes and classifies the image and then locates it to determine the position of the four-rope grab bucket;
[0081] S5. Swing calculation: According to the position of the spreader detected by YOLOv5, the swing amounts Δx and Δy along the X-axis and Y-axis directions are calculated through coordinate conversion between the pixel coordinate system and the world coordinate system:
[0082] Δx=x w -x0, Δy=y w -y0
[0083] Among them, x0 and y0 represent the coordinates of the starting position on the X-axis and Y-axis; w ,y w Represents the coordinates of the current position on the X-axis and Y-axis;
[0084] S6. Output swing angle: Calculate the swing angle α according to the swing amount and the length of the hanging rope of the four-rope grab:
[0085]
[0086] like Figure 2 As shown, l is the length of the suspension rope and d is the horizontal offset of the suspension rope.
[0087] In the above, the feature extraction network of the lightweight YOLOv5 algorithm uses the Conv convolution module to replace the original Focus module to reduce the amount of calculation; MobileNetv2 is used as the backbone, and MobileNetv2 uses depthwise separable convolution. This structure decomposes the standard convolution into depth convolution and point-by-point convolution, which significantly reduces the amount of calculation and model size while maintaining high accuracy.
[0088] The neck network of the lightweight YOLOv5 algorithm uses the FPN+PAN feature pyramid structure to transmit the feature information of objects of different sizes. Among them, FPN (Feature Pyramid Network) fuses features of different levels through bottom-up, top-down and lateral connections to construct a feature pyramid with rich semantic information, which helps the model detect objects of different sizes. Figure 3 As shown in the figure, PAN (Path Aggregation Network) further enhances the lateral connection of features based on FPN. Through bottom-up path enhancement, low-level features can be better transmitted to high-level levels, promoting the flow of information between different levels, especially helping to improve the detection performance of small targets.
[0089] The lightweight YOLOv5 algorithm achieves lightweight model while maintaining or improving detection accuracy as much as possible by optimizing the feature extraction network, adopting a more efficient Backbone, and improving the neck network structure. It performs well in handling small target detection tasks, especially.
[0090] In the above, in step S2, the image data needs to be converted from floating point numbers to quantized values for preprocessing. The specific numerical formula after quantization is:
[0091]
[0092]
[0093] In the formula: Q is the specific value after quantization; R is the weight parameter value; Z is the zero point value; xf is the input data; n is the number of bits of the value after quantization processing.
[0094] In the actual process, by applying different quantization models, the 8-bit fixed-point quantization model becomes significantly smaller. Converting floating-point tensors into unit8 tensors can reduce storage space and memory bandwidth, further improve system throughput, and alleviate possible delays caused by the system.
[0095] To further improve the efficiency of the network, the lightweight YOLOv5 algorithm uses Ghost Conv. Ghost Conv generates additional "ghost" feature maps (Ghost Feature Maps) at a lower computational cost by adding a series of linear transformations after a small number of convolutions, thereby enriching the number of features that the network can learn.
[0096] During the feature extraction process, the lightweight YOLOv5 algorithm uses the Conv convolution module to replace the original YOLOv5 Focus module, and uses MobileNetv2 to replace the original CSP module in Backbone.
[0097] The feature extraction in step S3 specifically includes the following steps:
[0098] S3.1. Depthwise Convolution:
[0099] Applying a convolution kernel to each input channel results in the same number of output channels, but the feature map size of each output channel is the same as the input;
[0100] S3.2, point-by-point convolution:
[0101] Perform 1x1 point-by-point convolution on the result of the depthwise separable convolution to obtain a feature map of the same dimension as the input feature map;
[0102] S3.3. Add the result of point-by-point convolution to the original feature map to enhance feature representation:
[0103]
[0104] Where: W C , H C are the width and height of the input feature map respectively; N is the number of channels of the convolution kernel; M is the number of convolution kernels; H O is the height of the convolution module; W O is the width of the convolution module.
[0105] After calculation, it can be seen that the size of the output feature map is Wc*Hc*N.
[0106] Based on feature extraction, the YOLOv5 algorithm uses a convolutional neural network (CNN) to classify objects in an image. In the field of convolutional neural network compression, network pruning is a technical means to effectively reduce network complexity, alleviate overfitting, and improve the inference speed of quantization algorithms. The structured pruning strategy of convolution kernels focuses on the convolution kernel level. It trains the network normally in advance, then identifies and removes those convolution kernels or channels whose weight links are lower than the preset threshold, and supplements them with sparsity adjustment measures to achieve lightweight models. Specifically, first, a sparsity regularization term (such as L1 regularization) is introduced during the training process to encourage the model weights to become sparse. Then, after the training is completed, unimportant connections or parameters are pruned according to the sparsity of the weights.
[0107] In the specific implementation process, the pruning design judgment function is used in step S4 to optimize the network structure. The specific algorithm is:
[0108]
[0109] Where: N is the number of channels after network pruning; x is the number of channels of the original network convolution kernel; f(x) is the maximum value that is less than x and a multiple of 9 after pruning.
[0110] According to the branch pruning judgment function in formula (5), the number of channels is adjusted to meet the optimization requirements of the embedded platform while being as close as possible to the number of channels of the original network convolution kernel to reduce information loss. This method can gradually guide the model to develop a sparse structure during the training process, so that the performance of the model can be better maintained after pruning.
[0111] In order to improve the prediction speed and efficiency of the network, in the DenseNet network of the YOLOv5 algorithm, in addition to adjusting the number of channels to optimize performance, a one-time aggregation module can be used to remove dense connections and aggregate shallow features in the last layer. This improvement can reduce the amount of calculation, improve prediction speed and efficiency, while retaining the powerful feature extraction capabilities of DenseNet. This improvement may be particularly effective for small target detection tasks on four-rope grabs because it allows the network to still capture important feature information with fewer computing resources.
[0112] System test: In the development process of the crane four-rope grab swing angle calculation system based on YOLOv5, the system test used the PyTorch framework for model training and verification. Considering that the model will eventually need to be deployed on resource-constrained embedded devices, the hardware environment was specially configured through the RKNN framework. In order to achieve cross-platform compilation management, the development team installed the cross-compilation environment on the PC terminal and remotely logged in to the mobile device through the SSH protocol for operation. The CMakeLists.txt configuration file written using the CMake tool ensures the efficiency and consistency of the compilation process. During the development process, CLion, as the main integrated development environment, is not only used for writing and training Python code, but also for converting the trained model into a C++ program to run on the target hardware.
[0113] The performance of the system is quantified by the mAP evaluation index. During the model training process, the precision and recall rates are strictly controlled within the range of 0 to 1, and the final evaluation results are obtained by summing the integrals. In order to ensure the reliability of the results, the entire experiment was repeated 5 times to obtain the average value for comparative analysis.
[0114] Preprocessing of the dataset is another key step in system development. The PASCAL VOC dataset, which is widely used in object detection tasks, was selected. This dataset contains rich and diverse samples and background features, which is very suitable for training and testing deep learning models. The label format of the dataset has been converted to adapt to the training requirements of YOLOv5. Anchors are generated by clustering methods, and image enhancement techniques such as brightness, saturation, contrast adjustment and geometric transformation are used to improve the model's adaptability to different environments. The application of Mosaic data enhancement technology further improves the generalization of the model and effectively prevents overfitting.
[0115] Detection effect: The performance of the YOLOv5 algorithm in the small target detection task has been verified. The experimental results show that the system's small target detection ability has been significantly improved in terms of mAP@0.5 and mAP@0.50.96 indicators, among which the WCAL-PAN method has improved by 3.3% in mAP@0.5 and 5.2% in mAP@0.5:0.95. Overall, the designed system has improved the small target detection effect by 2.7%, which confirms the effectiveness and reliability of the system in practical applications.
[0116] According to the position of the sling detected by YOLOv5, the actual twisting amount of the sling can be calculated through the coordinate transformation between the pixel coordinate system and the world coordinate system, and then the swing angle can be calculated according to the length of the sling rope. Figure 4 It is a schematic diagram of the transformation between the pixel coordinate system and the world coordinate system. In the figure, Ow-XwYwZw is the world coordinate system, O-XcYcZc is the camera coordinate system, Od-uv is the pixel coordinate system, and O1-XY is the image coordinate system.
[0117] In the specific implementation process, the specific operation method of coordinate conversion in step S5 is as follows:
[0118] Get the detection results of the four-rope grab from the YOLOv5 algorithm;
[0119] The coordinates in the camera pixel coordinate system are converted to the coordinates in the world coordinate system. The formula is as follows:
[0120]
[0121] Where: Zc is the distance between the camera and the object, specifically the length of the four-rope grab rope;
[0122] is the pixel coordinate; is the internal parameter matrix;
[0123] is the external parameter matrix; is the world coordinate.
[0124] In the above, the coordinate transformation process uses the distortion coefficients obtained by camera calibration to correct the radial distortion and tangential distortion in the image. The specific steps are as follows:
[0125] Obtaining the distortion coefficient of the camera using a camera calibration method;
[0126] Convert the detected pixel coordinates (u, v) to the coordinates (x d ,y d );
[0127] Use the distortion correction formula to correct (x d ,y d ) to obtain the corrected coordinates (x′, y′).
[0128] The distortion correction formula is:
[0129] Radial distortion:
[0130] x′=x(1+k1r 2 +k2r 4 +k3r 6 )
[0131] y′=y(1+k1r 2 +k2r 4 +k3r 6 )
[0132] Among them, r 2 =x 2 +y 2 , (x, y) is the coordinate of the distortion point, (x′, y′) is the coordinate after correction, k1, k2, k3 are the radial distortion coefficients;
[0133] Tangential distortion:
[0134] x′=x+[2p1y+p2(r 2 +2x 2 )
[0135] y′=y+[2p1x+p2(r 2 +2x 2 )]
[0136] Among them, p1 and p2 are tangential distortion coefficients.
[0137] Through the above distortion correction process, the radial distortion and tangential distortion generated in the camera imaging process can be eliminated, the accuracy of the position and shape information of the target object in the image can be improved, and the accuracy of the YOLOv5 algorithm detection results can be improved.
[0138] In implementing the above steps S1 to S6, on the one hand, the lightweight YOLOv5 algorithm is used to effectively improve the recognition accuracy of the four-rope grab in the image by optimizing the feature extraction network, using MobileNetv2 as the Backbone and the FPN+PAN feature pyramid structure, especially in the small target detection task, which significantly improves the overall detection performance of the system; on the other hand, through lightweight design, including quantized model processing, using Ghost convolution to reduce the amount of calculation, and network pruning technology, the model complexity and computing resource requirements are greatly reduced, so that the system can run efficiently on resource-constrained embedded devices, improving real-time performance and system throughput. Based on the position of the spreader detected by YOLOv5, through precise coordinate transformation and distortion correction processing, the swing angle of the four-rope grab can be output in real time, providing accurate feedback and control basis for crane operations.
[0139] After introducing the method of the exemplary embodiment of the present application, the following is a reference to Figure 5 , Figure 5 This is a schematic diagram of the structure of a crane four-rope grab bucket swing angle calculation system based on YOL0v5 provided in one embodiment of the present application. Figure 5 As shown, the crane four-rope grab bucket swing angle calculation system based on YOL0v5 includes:
[0140] A CCD industrial camera, mounted on the trolley platform of the crane, for capturing real-time images of the four-rope grab;
[0141] Image acquisition card, used to transmit image data captured by CCD industrial camera to industrial computer;
[0142] A rotary encoder for measuring the length of the lifting rope of the four-rope grab;
[0143] A PLC for controlling the operation of the crane and the triggering of the image acquisition card; and
[0144] The marker is set on the top of the four-rope grab and is located directly below the CCD industrial camera. It is used as a reference to assist image recognition and coordinate positioning.
[0145] In the specific working process, the CCD industrial camera collects image data from the top of the four-rope grab, and then the image acquisition card quickly and accurately transmits the image data captured by the CCD industrial camera to the industrial computer. The YOLOv5 algorithm is run on the industrial computer to process and identify the transmitted images, so as to quickly and accurately identify the markers in the image and locate the markers. The existence of markers is crucial for image recognition and coordinate positioning. They serve as reference objects to help the system identify and locate the grab more accurately. In this system, the markers can be infrared structured light sources or circular reflectors, such as Figure 6When the infrared structured light source is used as a marker, it can avoid interference with the operator because it emits invisible infrared light, while ensuring that the camera can clearly capture the posture of the grab, so it is suitable for application scenarios with complex lighting environments. The circular reflector forms a clear mark in the image by reflecting light, assisting the system to quickly and accurately locate, so it is suitable for simple lighting environments.
[0146] The swing angle results calculated by the system will be output to the display screen or control system in real time for the operator's reference and decision-making. The system is also equipped with a human-computer interaction interface, through which the operator can view the swing angle calculation results, monitor the crane's operating status and send control instructions. The YOLOv5-based crane four-rope grab swing angle calculation system integrates high-precision CCD industrial cameras, fast image transmission technology, advanced YOLOv5 image recognition algorithms and flexible marker design to achieve real-time and accurate calculation of the crane's four-rope grab swing angle, and achieve high-efficiency, high-precision and high-stability operation monitoring, significantly improving the safety and efficiency of crane operations.
[0147] Finally, it should be noted that the above embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.
[0148] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.
Claims
1. A method for calculating the swing angle of a four-rope grab bucket of a crane based on YOL0v5, characterized in that: The steps include: S1, image acquisition: When the crane starts to operate, the crane PLC control program sends a trigger pulse to the image acquisition unit. After receiving the trigger pulse, the image acquisition unit starts the camera to collect the real-time posture image of the four-rope grab; S2, image preprocessing: Use the lightweight YOLOv5 algorithm to preprocess the collected images to extract feature information in the images; S3, grab bucket identification: the pre-processed image is subjected to feature extraction through the YOLOv5 algorithm to identify the four-rope grab bucket features in the image; S4. Display the position of the grab bucket: The YOLOv5 algorithm analyzes and classifies the image and then locates it to determine the position of the four-rope grab bucket; S5. Swing calculation: According to the position of the spreader detected by YOLOv5, the swing amounts Δx and Δy along the X-axis and Y-axis directions are calculated through coordinate conversion between the pixel coordinate system and the world coordinate system: Δx=x w -x0,Δy=y w -y0 Among them, x0 and y0 represent the coordinates of the starting position on the X-axis and Y-axis; w ,y w Represents the coordinates of the current position on the X-axis and Y-axis; S6. Output swing angle: Calculate the swing angle α according to the swing amount and the length of the hanging rope of the four-rope grab: Among them, l is the length of the suspension rope, and d is the horizontal displacement of the suspension rope.
2. The method for calculating the swing angle of a crane four-rope grab bucket based on YOL0v5 according to claim 1 is characterized in that: In step S2, the image data needs to be converted from floating point numbers to quantized values for preprocessing. The specific numerical formula after quantization is: In the formula: Q is the specific value after quantization; R is the weight parameter value; Z is the zero point value; xf is the input data; n is the number of bits of the value after quantization processing.
3. The method for calculating the swing angle of a crane four-rope grab bucket based on YOL0v5 according to claim 2 is characterized in that: The feature extraction in step S3 specifically includes the following steps: S3.
1. Depthwise Convolution: Applying a convolution kernel to each input channel results in the same number of output channels, but the feature map size of each output channel is the same as the input; S3.2, point-by-point convolution: Perform 1x1 point-by-point convolution on the result of the depthwise separable convolution to obtain a feature map of the same dimension as the input feature map; S3.
3. Add the result of point-by-point convolution to the original feature map to enhance feature representation: Where: W C , H C are the width and height of the input feature map respectively; N is the number of channels of the convolution kernel; M is the number of convolution kernels; H O is the height of the convolution module; W O is the width of the convolution module.
4. The method for calculating the swing angle of a crane four-rope grab bucket based on YOL0v5 according to claim 3 is characterized in that: In step S4, the pruning design judgment function is used to optimize the network structure, and the specific algorithm is: Where: N is the number of channels after network pruning; x is the number of channels of the original network convolution kernel; f(x) is the maximum value that is less than x and a multiple of 9 after pruning.
5. The method for calculating the swing angle of a crane four-rope grab bucket based on YOL0v5 according to claim 4 is characterized in that: The specific operation method of coordinate conversion in step S5 is as follows: Get the detection results of the four-rope grab from the YOLOv5 algorithm; The coordinates in the camera pixel coordinate system are converted to the coordinates in the world coordinate system. The formula is as follows: Where: Zc is the distance between the camera and the object, specifically the length of the four-rope grab rope; is the pixel coordinate; is the internal parameter matrix; is the external parameter matrix; is the world coordinate.
6. The method for calculating the swing angle of a crane four-rope grab bucket based on YOL0v5 according to claim 5 is characterized in that: The coordinate transformation process uses the distortion coefficients obtained by camera calibration to correct the radial distortion and tangential distortion in the image. The specific operation steps are as follows: Obtaining the distortion coefficient of the camera using a camera calibration method; Convert the detected pixel coordinates (u, v) to the coordinates (x d ,y d ); Use the distortion correction formula to correct (x d ,y d ) to obtain the corrected coordinates (x′, y′).
7. The method for calculating the swing angle of a crane four-rope grab bucket based on YOL0v5 according to claim 6 is characterized in that: The distortion correction formula is: Radial distortion: x′=x(1+k1r 2 +k2r 4 +k3r 6 ) y′=y(1+k1r 2 +k2r 4 +k3r 6 ) Among them, r 2 =x 2 +y 2 , (x, y) is the coordinate of the distortion point, (x′, y′) is the coordinate after correction, k1, k2, k3 are the radial distortion coefficients; Tangential distortion: x′=x+[2p1y+p2(r 2 +2x 2 )] <h2 style=";text-align:left;direction:ltr">y′ = y+[2p1x+p2(r<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> +2x<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> )] Among them, p1 and p2 are tangential distortion coefficients.
8. The method for calculating the swing angle of a crane four-rope grab bucket based on YOL0v5 according to claim 1, characterized in that: The feature extraction network of the lightweight YOLOv5 algorithm adopts the Conv convolution module and MobileNetv2 as the backbone; the neck network of the lightweight YOLOv5 algorithm adopts the FPN+PAN feature pyramid structure to transmit the feature information of targets of different sizes.
9. The crane four-rope grab bucket swing angle calculation system based on YOL0v5 is characterized by: The system comprises: A CCD industrial camera, mounted on the trolley platform of the crane, for capturing real-time images of the four-rope grab; An image acquisition card is used to transmit the image data captured by the CCD industrial camera to an industrial computer; A rotary encoder for measuring the length of the lifting rope of the four-rope grab; A PLC for controlling the operation of the crane and the triggering of the image acquisition card; and A marker is disposed on the top of the four-rope grab and is located directly below the CCD industrial camera, and is used as a reference for assisting image recognition and coordinate positioning.
10. The crane four-rope grab bucket swing angle calculation system based on YOL0v5 according to claim 9 is characterized in that: The marker is an infrared structured light source or a circular reflector.
Citation Information
Cited By
Method and system for calculating torsion angle of four-rope grab bucket of crane based on YOL0v5
CN118929450A
Lifting appliance control method and device of travelling crane and electronic equipment
CN119898689A
Crane grab bucket steel wire rope deviation rectification control method and system
CN119929670A
A method and system for correcting deviation of a steel wire rope of a traveling grab
CN119929670B
Method and system for identifying and calculating swing of grab bucket of port portal crane based on target detection
CN120198490A
