Method for detecting inclined pulling and inclined hoisting of crane based on YOLOV5 image inclination angle detection

Abstract

The invention relates to the technical field of image target detection, in particular to a crane inclined pulling and inclined hoisting detection method based on YOLOV5 image inclination angle detection. The method comprises the steps of determining a target detection area, performing first image acquisition on the target detection area, obtaining a data set of each image, selecting a reference image from the data set, calculating actual inclination angles of other images, performing value classification on all the images of the data set, inputting the images into a YOLOV5 classification neural network, and outputting a classification model. Performing second image acquisition on the target detection area, inputting the images acquired for the second time into the classification model to obtain a category value theta1 corresponding to the images acquired for the second time, acquiring an image when the crane picks up a hoisted object and obtaining an included angle theta2, and calculating an inclined pulling and inclined hoisting angle theta in the image according to theta1 and theta2, and judging whether the inclined pulling and hanging angle theta in the image exceeds a threshold value or not. The method provided by the invention is more accurate and reliable in detection result, higher in real-time performance and lower in cost.

Description

Technical Field

[0001] The present invention relates to the technical field of image target detection, and in particular to a method for detecting a skewed or tilted crane based on YOLOV5 image inclination detection. Background Art

[0002] Cranes, commonly known as cranes, are generally prohibited from tilting or slanting when they are used for lifting operations. The so-called tilting or slanting of a crane means that the wire rope of the crane is not perpendicular to the ground, forming a certain angle. Because if the object is tilted or slanted before it leaves the ground, then the object will inevitably cause the rope to swing under the influence of gravity after it leaves the ground. This will first cause safety hazards to the environment near the object, and secondly, it will continuously wear out the adhesion between the hook and the rope. Over time, the durability of the hook and the rope will decrease, and it will cause frequent safety accidents at the work site. Therefore, it is crucial to accurately detect tilting or slanting and notify the operator in a timely manner.

[0003] The traditional method is to use image target detection technology, by setting the threshold of the angle between the wire rope and the vertical line of the ground, when the angle between the wire rope and the vertical line of the ground is within the threshold range, it is considered that there is no crooked or slanted lifting. However, this method ignores the skew of the camera itself, that is, when the camera itself is not perpendicular to the ground, the image captured by the camera is also tilted, resulting in inaccurate detection results. Patent document CN104106097A discloses a method for detecting the horizontal gravity direction of an image, which involves complex algorithm logic and a large number of complex mathematical functions. The mathematical functions used include local orientation function, image direction function, periodic function, kernel function, receptive field response function and derivative function, etc., and these functions contain a large number of complex parameters. The relevant parameters include attention focus coordinates, significant orientation angle, local orientation angle, weight and horizontal gravity recognition angle pair, etc., which require a high mathematical theoretical foundation of technicians. Ordinary technicians engaged in this field often do not have such ability, so the application threshold of this method is high and the application cost is large. Therefore, in view of the above shortcomings, a crane crooked or slanted lifting detection method based on YOLOV5 image inclination detection is proposed. Summary of the invention

[0004] 1. Technical issues to be resolved The present invention provides a method for detecting a crane's crookedness based on YOLOV5 image inclination detection, so as to overcome the problems in the prior art of large storage space occupancy, low detection efficiency and high application cost, and the technical problem that the prior detection technology ignores the influence of the crooked camera itself on the crookedness detection, resulting in poor accuracy and reliability of the crookedness detection results, which is easy to cause safety accidents.

[0005] (II) Technical solution In order to solve the above problems, the present invention provides a crane tilt detection method based on YOLOV5 image inclination detection, comprising: Step S1: determining a target detection area, performing a first image acquisition of the target detection area through a camera, and obtaining a data set of each image in the first image acquisition; Step S2: Select one image from the data set obtained in step S1 as a reference image, calculate the actual tilt angles of other images in the data set based on the reference image, and classify all images in the data set; Step S3: input all images classified in step S2 into the YOLOV5 classification neural network, and output the classification model after training and learning the YOLOV5 classification neural network; Step S4: perform a second image acquisition on the target detection area, and input the second acquired images into the classification model of step S3, and output the category value corresponding to the second acquired images through the classification model, and the value is set to θ1; Step S5: Capture an image of the crane picking up the object, and obtain the angle θ2 between the line connecting the hook and the pulley center and the vertical direction of the image, and correct the angle θ2 according to θ1 obtained in step S4; Step S6: Based on θ1 obtained in step S4 and θ2 obtained in step S5, calculate the angle θ of the tilt in the image, and determine whether the angle θ of the tilt in the image exceeds a threshold. If so, it proves that the crane in the image is tilted, and an early warning is issued immediately.

[0006] Preferably, in step S1, the captured image should contain at least one person as a reference object for determining the degree of image tilt, and the reference objects include a horizon, an upright person, an upright telephone pole, or a suspended object.

[0007] Preferably, in step S1, the specific method for obtaining the rotation set is to perform multiple rotation operations at different angles and orientations on each image obtained by the first image acquisition, and mark the image after each rotation, with the clockwise rotation angle being recorded as positive and the counterclockwise rotation angle being recorded as negative.

[0008] Preferably, the calculation formula for the number of data sets is: I=m×q Among them, I is the number of data sets; q is the number of original images; m is the number of rotated images corresponding to the original images.

[0009] Preferably, the calculation formula for the number of rotations is: m=R / d Among them, m is the number of rotations; R is the camera tilt angle range; d is the rotation accuracy.

[0010] Preferably, the value of d is smaller than a threshold value of an angle θ of the slanted hanging, and the threshold value of θ is a maximum inclination angle allowed between the wire rope and a perpendicular line to the ground.

[0011] Preferably, in step S2, the actual tilt angle of the reference image is marked as 0°.

[0012] Preferably, in step S2, the classification accuracy of the value classification is set to D, the number of categories and the range of each category are determined according to R and D, and D>d.

[0013] Preferably, in step S5, the interior of the image of the crane picking up the suspended object includes a hook and a pulley of the crane, and a line connecting the center of the hook and the pulley can be obtained through the position of the hook and the position of the pulley in the image.

[0014] Preferably, in step S6, the calculation formula of the angle of the skew in the image is: θ=θ2+θ1 Among them, θ is the angle of the crooked or slanted suspension in the image; θ1 is the category value of the second image acquisition; θ2 is the angle between the line connecting the center of the hook and the pulley and the vertical direction of the image.

[0015] (III) Beneficial effects The crane crooked and slanted detection method based on YOLOV5 image inclination detection provided by the present invention eliminates the influence of the crooked and slanted detection caused by the crookedness of the camera itself, improves the accuracy and reliability of the crooked and slanted detection, and further improves the safety of the work site. At the same time, the algorithm logic involved in the method is simple and easy to implement, and only simple parameters in single digits are involved in the algorithm. Ordinary technicians in the application field can successfully use the method, which reduces the application threshold and application cost. In addition, the method indirectly improves the durability of the crane's hook and rope, and reduces the lifting cost of the crane. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a flow chart of a method for detecting a crooked or slanted crane based on YOLOV5 image inclination detection according to an embodiment of the present invention. Implementation

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] Figure 1Flow chart of a method for detecting a crane tilting and slanting based on YOLOV5 image tilt angle detection according to an embodiment of the present invention. Figure 1 As shown, the present invention provides a crane tilt detection method based on YOLOV5 image tilt detection, including: Step S1: determine the target detection area, perform a first image acquisition on the target detection area through a camera, and obtain a data set of each picture in the first image acquisition.

[0019] In this method, in step S1, the captured image should contain at least one person as a reference object for determining the degree of image tilt, and the reference objects include a horizon, an upright person, an upright telephone pole, or a suspended object.

[0020] In practical applications, the specific method for obtaining the rotation set is to rotate each image obtained by the first image acquisition multiple times at different angles and orientations, and mark the image after each rotation, with the clockwise rotation angle being recorded as positive and the counterclockwise rotation angle being recorded as negative.

[0021] In this method, the calculation formula for the number of data sets is: I=m×q Among them, I is the number of data sets; q is the number of original images; m is the number of rotated images corresponding to the original images.

[0022] In practical applications, the formula for calculating the number of rotations is: m=R / d Among them, m is the number of rotations; R is the camera tilt angle range; d is the rotation accuracy.

[0023] It should be noted that the value of d is smaller than the threshold value of the angle θ of the slanted lifting, and the threshold value of θ is the maximum inclination angle allowed between the wire rope and the perpendicular line of the ground.

[0024] To further illustrate, in practical applications, a rotation operation is performed on each collected image to obtain a data set, and the collected image is used as the original image, and the center point of each original image is used as its own rotation origin. Each original image is rotated m times clockwise or counterclockwise around its own rotation origin at different angles, and each rotation obtains a rotated image, and each rotated image is marked: the angle of clockwise rotation is recorded as positive, and the angle of counterclockwise rotation is recorded as negative. Accordingly, each original image has a corresponding rotated image set, and each image set contains m rotated images corresponding to the original image. According to the calculation formula for the number of data sets, assuming that there are m original images, I is m×q rotated images, and these m×q rotated images constitute the data set. In the above, the number of rotations m is determined by the camera tilt angle range R and the rotation accuracy d, m=R / d, where R and d are determined according to specific business needs, and d is much smaller than the threshold of the angle θ of the crooked lifting. This threshold is the maximum inclination angle allowed between the wire rope and the vertical line of the ground in the actual production demand of using this method to detect the crooked lifting.

[0025] Step S2: Select one image from the data set obtained in step S1 as a reference image, calculate the actual tilt angles of other images in the data set based on the reference image, and classify all images in the data set.

[0026] In this method, in step S2, the actual tilt angle of the reference image is marked as 0°, wherein the classification accuracy of the value classification is set to D, and the number of categories and the range of each category are determined based on R and D, wherein D>d.

[0027] It should be noted that in order to calculate the actual tilt angle of each image in the data set, it is necessary to make a comprehensive judgment based on the multiple reference objects selected in step S1, select a relatively positive image in the acquired data set, count the image as the reference image, and mark the actual tilt angle of the reference image as 0°. Using the actual tilt angle of the image as a reference, calculate the actual tilt angle of each image in the data set.

[0028] In addition, in practical applications, for all images in the data set, the classification accuracy D is set according to specific business needs, and D>d. The number of categories and the range of each category are further determined based on R and D, and the middle value of each category range is taken to assign a value to the category for classification.

[0029] It should be noted that, for a certain number of images in the data set, the smaller the classification accuracy D is, the more categories are divided, and the higher the identifiable accuracy of step S4 is. In practical applications, if the actual production application has low requirements for identifiable accuracy, D can be set to a larger value under the condition of D>d.

[0030] Step S3: Input all images classified in step S2 into the YOLOV5 classification neural network, and output the classification model after training and learning the YOLOV5 classification neural network.

[0031] Step S4: perform a second image acquisition on the target detection area, and input the second acquired images into the classification model of step S3, and output the category value corresponding to the second acquired image through the classification model, and the value is set to θ1.

[0032] Step S5: Capture an image of the crane picking up the object, and obtain the angle θ2 between the line connecting the center of the hook and the pulley and the vertical direction of the image, and correct the angle θ2 according to θ1 obtained in step S4.

[0033] In this method, in step S5, the interior of the image of the crane picking up the hanging object includes the hook and pulley of the crane, and the line connecting the center of the hook and the pulley can be obtained through the position of the hook and the position of the pulley in the image.

[0034] It should be noted that in practical applications, the image target detection technology is used to detect the hook and pulley of the crane in the image, and the line connecting the center of the hook and the pulley is obtained, and the angle θ2 between the line connecting the center of the hook and the pulley and the vertical direction of the image is further calculated, and the angle θ2 is corrected using the θ1 obtained in step S4.

[0035] Step S6: Based on θ1 obtained in step S4 and θ2 obtained in step S5, calculate the angle θ of the tilt in the image, and determine whether the angle θ of the tilt in the image exceeds a threshold. If so, it proves that the crane in the image is tilted, and an early warning is issued immediately.

[0036] In this method, in step S6, the calculation formula of the angle of the skew in the image is: θ=θ2+θ1 Among them, θ is the angle of the image; θ1 is the category value of the second collected image; θ2 is the angle between the line connecting the center of the hook and the pulley and the vertical direction of the image.

[0037] In practical applications, this detection method occupies little space, has low cost, high detection efficiency and accuracy, strong reliability, and low threshold.

[0038] In order to further understand the working principle of the crane tilt detection method based on YOLOV5 image tilt detection, the following describes in detail an embodiment of the application of the detection method. In this embodiment, when detecting the tilt in the lifting operation, the following steps are taken: Step 1: Determine the target detection area, perform the first image acquisition of the target detection area through the camera, and obtain the respective data sets of each picture in the first image acquisition.

[0039] In this embodiment, the captured image should contain at least one human reference object for determining the degree of image tilt, such as a horizon, an upright person, an upright telephone pole, or a suspended object. The image may also contain a crane, suspended objects, etc. In addition, in practical applications, the more images q are captured, the more accurate the final detection result will be. It is assumed that a total of 100 images are captured in this embodiment, that is, q=100.

[0040] In this embodiment, the threshold of θ is 18°, and the camera tilt angle range R is taken as: [-180°, 180°]. It should be further explained that the tilt angle range can be expressed in other ways as needed, such as [0°, 360°], etc., d=1°, m=360 times, so after rotation, each original image has a corresponding rotated image set, and each image set contains 360 images of different angles corresponding to the original image after rotation. According to the calculation formula for the number of data sets, the rotated image set of 100 original images contains a total of 36,000 rotated images, and these 36,000 rotated images constitute the data set.

[0041] In practical applications, take any original image P1 among the 100 original images as an example: Assume that the rotated image set of the original image P1 includes the rotated images p11, p12, ..., p1 360 , among which p13, p19, p1 126 The images obtained by rotating P1 by 3° counterclockwise, 5° clockwise, and 10° clockwise, respectively, and the images obtained by rotating p13, p19, and p1 126 They are marked as -3°, +5°, and +10° respectively.

[0042] Step 2: Select one image from the data set obtained in step 1 as a reference image, calculate the actual tilt angles of other images in the data set based on the reference image, and classify all images in the data set.

[0043] In actual applications, a comprehensive judgment is made based on the aforementioned multiple reference objects. A relatively straight image is selected in the data set. The so-called relatively straight refers to a small degree of inclination. The image is set as the reference image, and the actual inclination angle of the image is marked as 0°. The actual inclination angle of the image is used as a reference to calculate the actual inclination angle of each image in the data set.

[0044] In this embodiment, it is assumed that a relatively positive image is artificially selected as p19, and the actual tilt angle of the image is marked as 0°, then p13, p1 126 The actual tilt angles are -8° and +5° respectively. Similarly, the actual tilt angle of each image in the dataset can be obtained.

[0045] In practical applications, the classification accuracy is set to D=3°, then all images in the data set are divided into R / D=120 categories, and the span of each category range is 3°, so the category ranges are: [-180°, -178°), [-178°, -176°), [-176°, -174°), …, [+178°, +180°), and the corresponding category values ​​are -179°, -177°, -175°, …, 179°.

[0046] In this embodiment, the category values ​​of all images in the data set are classified according to the actual tilt angle, and all images in the data set are classified into these category values ​​according to the actual tilt angle of each image in the data set: -179°, -177°, -175°, ..., 179°.

[0047] Step 3: Input all images classified in step 2 into the YOLOV5 classification neural network. After training and learning the YOLOV5 classification neural network, output the classification model.

[0048] Step 4: Perform a second image acquisition of the target detection area, and input the second acquired images into the classification model of step 3, and output the category value corresponding to the second acquired image through the classification model, which is set to θ1.

[0049] In practical applications, the classification model outputs the specific category value corresponding to the new image, which is the inclination angle θ1 of the second captured image, that is, the inclination angle of the camera that captures the new image. In this embodiment, assuming that the category value corresponding to the second captured image is +9°, the inclination angle of the second captured image is 9 degrees: θ1=+9°.

[0050] Step 5: Collect the image of the crane picking up the object, and obtain the angle θ2 between the line connecting the center of the hook and the pulley and the vertical direction of the image, and correct the angle θ2 according to θ1 obtained in step 4.

[0051] In this embodiment, image target detection technology is used to detect the hook and pulley of the crane in the image, and the line connecting the center of the hook and the pulley is obtained. This line coincides with the wire rope of the crane, and the angle θ2 between the line connecting the center of the hook and the pulley and the vertical direction of the image is calculated. It is assumed that θ2=-22°, and θ1 is applied to correct the angle θ2.

[0052] Step six: Based on θ1 obtained in step four and θ2 obtained in step five, calculate the angle θ of the tilted crane in the image, and determine whether the angle θ of the tilted crane in the image exceeds the threshold. If so, it proves that the crane in the image is tilted, and an early warning is issued immediately.

[0053] In this embodiment, the angle of the tilt in the image is θ=θ2+θ1=-22°+9°= -11°, determine whether the angle θ of the tilted crane in the image exceeds the threshold. If so, the crane in the image appears to be tilted, and an alarm can be issued to prompt the on-site operator; otherwise, the crane in the image does not appear to be tilted.

[0054] In practical applications, the threshold in this embodiment is 18°. It can be seen that θ obtained in step 6 does not exceed the threshold, so it is considered that the crane in the image does not appear to be tilted or slanted.

[0055] The crane crooked and slanted detection method based on YOLOV5 image inclination detection provided by the present invention eliminates the influence of the crooked and slanted detection caused by the crookedness of the camera itself, improves the accuracy and reliability of the crooked and slanted detection, and further improves the safety of the work site. At the same time, the algorithm logic involved in the method is simple and easy to implement, and only simple parameters in single digits are involved in the algorithm. Ordinary technicians in the application field can successfully use the method, which reduces the application threshold and application cost. In addition, the method indirectly improves the durability of the crane's hook and rope, and reduces the lifting cost of the crane.

[0056] The above implementation modes are only used to illustrate the present invention, but not to limit the present invention. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, all equivalent technical solutions also belong to the scope of the present invention. The patent protection scope of the present invention should be defined by the claims.

Claims

1. A crane tilt detection method based on YOLOV5 image inclination detection is characterized by: include: Step S1: determining a target detection area, performing a first image acquisition of the target detection area through a camera, and obtaining a data set of each image in the first image acquisition; Step S2: Select one image from the data set obtained in step S1 as a reference image, calculate the actual tilt angles of other images in the data set based on the reference image, and classify all images in the data set; Step S3: input all images classified in step S2 into the YOLOV5 classification neural network, and output the classification model after training and learning the YOLOV5 classification neural network; Step S4: perform a second image acquisition on the target detection area, and input the second acquired images into the classification model of step S3, and output the category value corresponding to the second acquired images through the classification model, and the value is set to θ1; Step S5: Capture an image of the crane picking up the object, and obtain the angle θ2 between the line connecting the hook and the pulley center and the vertical direction of the image, and correct the angle θ2 according to θ1 obtained in step S4; Step S6: Based on θ1 obtained in step S4 and θ2 obtained in step S5, calculate the angle θ of the tilt in the image, and determine whether the angle θ of the tilt in the image exceeds a threshold. If so, it proves that the crane in the image is tilted, and an early warning 1 is issued immediately.

2. The method for detecting a crane tilting or slanting based on YOLOV5 image tilt angle detection according to claim 1 is characterized in that: In step S1, the captured image should contain at least one person as a reference object for determining the degree of image tilt, and the reference objects include a horizon, an upright person, an upright electric pole, or a suspended object.

3. The method for detecting a crane tilting or slanting based on YOLOV5 image tilt angle detection according to claim 1, characterized in that: In step S1, the specific method for obtaining the rotation set is to rotate each image obtained by the first image acquisition multiple times at different angles and orientations, and mark the image after each rotation, with the clockwise rotation angle being recorded as positive and the counterclockwise rotation angle being recorded as negative.

4. The method for detecting a crane tilting or slanting based on YOLOV5 image tilt angle detection according to claim 3 is characterized in that: The calculation formula for the number of data sets is: I=m×q Among them, I is the number of data sets; q is the number of original images; m is the number of rotated images corresponding to the original images.

5. The method for detecting a crane tilting or slanting based on YOLOV5 image tilt angle detection according to claim 3 is characterized in that: The calculation formula for the number of rotations is: m=R / d Among them, m is the number of rotations; R is the camera tilt angle range; d is the rotation accuracy.

6. The method for detecting a crane tilting or slanting based on YOLOV5 image tilt angle detection according to claim 5, characterized in that: The value of d is smaller than the threshold value of the angle θ of the skewed hanging, and the threshold value of θ is the maximum inclination angle allowed between the wire rope and the perpendicular line of the ground.

7. The method for detecting a crane tilting or slanting based on YOLOV5 image tilt angle detection according to claim 1, characterized in that: In the step S2, the actual tilt angle of the reference image is marked as 0°.

8. The method for detecting a crane tilting or slanting based on YOLOV5 image tilt angle detection according to claim 5, characterized in that: In the step S2, the classification accuracy of the value classification is set to D, and the number of categories and the range of each category are determined according to R and D, and D>d.

9. The method for detecting a crane tilting or slanting based on YOLOV5 image tilt angle detection according to claim 1, characterized in that: In step S5, the interior of the image of the crane picking up the hanging object includes the hook and pulley of the crane, and the line connecting the center of the hook and the pulley can be obtained through the position of the hook and the position of the pulley in the image.

10. The method for detecting a crane tilting or slanting based on YOLOV5 image tilt angle detection according to claim 1, characterized in that: In step S6, the calculation formula of the angle of the skew in the image is: θ=θ2+θ1 Among them, θ is the angle of the crooked or slanted suspension in the image; θ1 is the category value of the second image acquisition; θ2 is the angle between the line connecting the center of the hook and the pulley and the vertical direction of the image.

Images