Monocular measurement method and system suitable for lifting appliance attitude detection

Abstract

The invention belongs to the technical field of hanger attitude detection, and discloses a monocular measurement method and system suitable for hanger attitude detection. According to the method, at least one group of marking units are arranged on the lifting appliance, a monocular camera is vertically mounted right below the center of a frame of a trolley, a real-time image containing a marker is acquired by using the monocular camera, and then real-time pixel coordinates of the marker are acquired by using a target detection model, so that real-time pixel coordinates of the lifting appliance are obtained; comparing the real-time pixel coordinate of the lifting appliance with the static pixel coordinate of the lifting appliance to obtain a pixel deviation value of the lifting appliance; and finally, obtaining the actual offset of the lifting appliance along the moving directions of the trolley and the cart based on the pixel offset value of the lifting appliance, and obtaining the rotation angle of the lifting appliance around the lifting direction based on the real-time pixel coordinates of the marker. According to the invention, the posture of the lifting appliance can be accurately measured.

Description

Technical Field

[0001] The present invention belongs to the technical field of spreader attitude detection, and more specifically, relates to a monocular measurement method and system applicable to spreader attitude detection. Background Art

[0002] Existing spreader attitude detection solutions are mainly implemented based on lidar. Due to the inherent reasons of lidar itself, the following problems will occur: (1) high cost; (2) large sawtooth fluctuations in the output waveform of lidar; (3) in some scenarios, such as at noon, due to the scattering of sunlight, the lidar ranging is unstable, resulting in function failure.

[0003] How to overcome the above problems and achieve accurate detection of the offset of the spreader along the trolley direction and the gantry direction, as well as the rotation angle of the spreader around the lifting direction, is a topic of concern and research in this field. Summary of the Invention

[0004] The present invention provides a monocular measurement method and system applicable to spreader attitude detection, and solves at least one of the problems such as high cost, large output fluctuations, and function failure in spreader attitude detection in the prior art.

[0005] The present invention provides a monocular measurement method applicable to spreader attitude detection, including the following steps:

[0006] Set at least one group of marking units on the spreader, and vertically install a monocular camera directly below the center of the trolley frame; wherein, the trolley is arranged on the gantry, the movement direction of the trolley is perpendicular to the movement direction of the gantry, the spreader is suspended below the trolley, and each group of marking units includes two markers symmetric about the center of the spreader;

[0007] Use the monocular camera to obtain a real-time image containing the markers;

[0008] Based on the real-time image, use an object detection model to obtain the real-time pixel coordinates of the markers;

[0009] Based on the real-time pixel coordinates of the markers, obtain the real-time pixel coordinates of the spreader;

[0010] Compare the real-time pixel coordinates of the spreader with the static pixel coordinates of the spreader to obtain the pixel offset value of the spreader;

[0011] Based on the pixel offset value of the spreader, obtain the actual offset of the spreader along the movement directions of the trolley and the gantry; based on the real-time pixel coordinates of the markers, obtain the rotation angle of the spreader around the lifting direction.

[0012] Preferably, multiple groups of marking units with different colors or textures are provided on the spreader; the focal length of the monocular camera is configured to cover the entire movement range of the spreader between the highest point and the lowest point, and at least one group of marking units can be clearly seen when the monocular camera acquires real-time images.

[0013] Preferably, the target detection model uses YOLO or CenterNet.

[0014] Preferably, obtaining the actual offset based on the pixel offset value includes: calibrating the monocular camera, and combining the distance from the optical center of the monocular camera to the spreader to map the pixel coordinates to a three-dimensional rectangular coordinate system with the camera as the origin.

[0015] Preferably, calibrating the monocular camera includes: after performing distortion correction on the monocular camera, calibrating the actual length represented by a single pixel in the plane where the spreader is located at different heights.

[0016] Preferably, when the spreader is at different heights, respectively obtain the pixel distance between the pixel coordinates of two markers in a certain group of marking units in the image; according to the pixel distance between the pixel coordinates of the two markers in the image and the actual distance between the two markers, calibrate the actual length represented by a single pixel in the plane where the spreader is located at different heights.

[0017] Preferably, take the average value of the pixel coordinates of the two markers in the image to obtain the pixel coordinates of the spreader at different heights.

[0018] Preferably, the distance from the optical center of the monocular camera to the spreader is obtained by the following method: obtain the measured distance from the optical center of the monocular camera to the spreader through a scanning system, and correct the measured distance according to the pixel size and the actual size of the marker in the real-time image.

[0019] Preferably, obtain the angle between the straight line connecting two markers in a certain group of marking units in the real-time image and the horizontal line, and the angle between the straight line connecting the two markers when the spreader is stationary and the horizontal line, and obtain the rotation angle of the spreader around the lifting direction based on the difference between the above two angles.

[0020] On the other hand, the present invention provides a monocular measurement system applicable to spreader attitude detection, including: a monocular camera, at least one group of marking units, and a data processing module; the monocular camera is used to acquire real-time images containing markers; the data processing module is used to execute the steps in the above-mentioned monocular measurement method applicable to spreader attitude detection to obtain the actual offset of the spreader along the movement directions of the trolley and the crane, and the rotation angle of the spreader around the lifting direction.

[0021] One or more technical solutions provided in the present invention have at least the following technical effects or advantages:

[0022] In the present invention, at least one set of marking units is arranged on the spreader, the monocular camera is vertically installed directly below the center of the trolley frame, the monocular camera is used to obtain a real-time image containing the marker, and then the target detection model is used to obtain the real-time pixel coordinates of the marker, and then the real-time pixel coordinates of the spreader are obtained; then the real-time pixel coordinates of the spreader are compared with the static pixel coordinates of the spreader (i.e., the pixel coordinates when the spreader is stationary) to obtain the pixel offset value of the spreader; finally, based on the pixel offset value of the spreader, the actual offset amount of the spreader along the movement directions of the trolley and the gantry is obtained, and the rotation angle of the spreader around the lifting direction is obtained based on the real-time pixel coordinates of the marker. Compared with the spreader attitude detection solution based on lidar, the cost of the present invention is lower, the pixel offset output by the camera image is more stable and smooth, and the marker can still be seen clearly in scenarios such as strong light diffuse reflection at noon, which can ensure the continuous effectiveness of the measurement function. The present invention can accurately measure the actual offset amount of the spreader along the movement directions of the trolley and the gantry and the rotation angle of the spreader around the lifting direction, and realize the precise detection of the spreader attitude. Description of the Drawings

[0023] Figure 1 is the pinhole imaging model;

[0024] Figure 2 is a schematic structural diagram of a monocular measurement system applicable to spreader attitude detection provided in Embodiment 2 of the present invention;

[0025] Figure 3 is a physical diagram containing two markers photographed by a monocular measurement system applicable to spreader attitude detection provided in Embodiment 2 of the present invention.

[0026] Among them, 1 - trolley, 2 - gantry, 3 - monocular camera, 4 - marker, 5 - spreader. Detailed Embodiments

[0027] In order to better understand the above technical solutions, the above technical solutions will be described in detail below in conjunction with the accompanying drawings of the specification and specific embodiments.

[0028] Embodiment 1:

[0029] Embodiment 1 provides a monocular measurement method applicable to spreader attitude detection, including the following steps:

[0030] Set at least one set of marking units on the spreader, and vertically install the monocular camera directly below the center of the trolley frame; wherein, the trolley is arranged on the gantry, the moving direction of the trolley is perpendicular to the moving direction of the gantry, the spreader is suspended below the trolley, and each set of marking units includes two markers symmetric about the center of the spreader;

[0031] Use the monocular camera to obtain a real-time image containing the markers;

[0032] Based on the real-time image, use the object detection model to obtain the real-time pixel coordinates of the markers;

[0033] Based on the real-time pixel coordinates of the markers, obtain the real-time pixel coordinates of the spreader;

[0034] Compare the real-time pixel coordinates of the spreader with the static pixel coordinates of the spreader to obtain the pixel offset value of the spreader;

[0035] Based on the pixel offset value of the spreader, obtain the actual offset amount of the spreader along the moving directions of the trolley and the gantry; based on the real-time pixel coordinates of the markers, obtain the rotation angle of the spreader around the lifting direction.

[0036] Among them, the object detection model can adopt object detection algorithms based on deep learning. For example, YOLO, CenterNet, etc. can be adopted.

[0037] Among them, obtaining the actual offset amount based on the pixel offset value includes: calibrating the monocular camera, and combining the distance from the optical center of the monocular camera to the spreader to map the pixel coordinates to a three-dimensional rectangular coordinate system with the camera as the origin.

[0038] Since the installation position of the camera cannot be known, the three-dimensional rectangular coordinate system with the camera as the origin can be used as the reference system. Because it cannot be guaranteed that the camera is absolutely vertically downward, in the present invention, at different heights, the pixel coordinates of the spreader center in the image are respectively converted into the origin. The pixel coordinates of the markers are obtained through an object detection model based on deep learning, and the pixel coordinates of the markers are converted into the pixel coordinates of the spreader center in the image, that is, the coordinate origin at this height.

[0039] See Figure 1 , in the case of no distortion of the camera, based on the pinhole imaging model, with the camera optical center as the origin, the point P in the three-dimensional rectangular coordinate system with the camera as the origin passes through the pinhole imaging to the point P' in the two-dimensional coordinate system of the image plane. The pixel coordinates of P' are (u, v), and the real coordinates of P are (X, Y, Z). The coordinate conversion is expressed as follows:

[0040]

[0041] Where (u, v) is the pixel coordinate of the marker in the image, (X, Y, Z) is the real coordinate of the marker, and f x is the focal length of the camera in the x direction, f y is the focal length of the camera in the y direction, (C x ,C y ) are the pixel coordinates of the center of the image plane.

[0042] In the present invention, calibrating the monocular camera includes: after performing distortion correction on the monocular camera, calibrating the actual length represented by a single pixel when the sling is at different heights.

[0043] The distortion correction of the monocular camera may be performed using an existing correction solution.

[0044] Considering that the ratio of pixels to actual length is still not necessarily a fixed value after camera distortion correction, the present invention also performs an additional calibration step. The specific calibration method is as follows: when the spreader is at different heights, the pixel distance between the pixel coordinates of two markers in a group of marking units in the image is obtained respectively; based on the pixel distance between the pixel coordinates of the two markers in the image and the actual distance between the two markers, the actual length represented by a single pixel on the plane where the spreader is located when the spreader is at different heights is calibrated.

[0045] The actual distance between the two markers is fixed and known.

[0046] In addition, by taking the average of the pixel coordinates of the two markers in the image, the pixel coordinates of the spreader at different heights can be obtained.

[0047] The present invention specifically obtains the distance from the optical center of the monocular camera to the spreader in the following manner: Because the spreader has a small tilt angle, the spreader's pixels are nearly flat from the camera's perspective. A scanning system is first used to obtain the measured distance from the monocular camera to the spreader. This measured distance is then corrected based on the pixel size of the marker in the real-time image and the actual size of the marker. Obtaining the measured distance from the monocular camera to the spreader using the scanning system can be implemented using existing lidar solutions.

[0048] In a preferred embodiment, the present invention provides a plurality of groups of marking units with different colors or textures on the spreader.

[0049] The monocular camera is installed directly below the center of the trolley frame, and the lens is vertically downward, which can ensure that the spreader in the image captured by the monocular camera is located in the central area of the image, so that the swing range of the spreader detected by the camera image is maximized. The focal length of the monocular camera is configured to cover the entire movement range of the spreader between the highest point and the lowest point. When the monocular camera acquires real-time images, it can clearly see at least one set of marker units. For example, it can clearly see the pixel texture of the marker.

[0050] The marker in the present invention has significant color and texture differentiations compared with other objects or devices on site. The marker can be a self-set pattern, a pulley on the spreader, etc. By setting multiple sets of marker units, it can be ensured that when the spreader swings, even if some sets of marker units on the spreader do not appear in the camera view due to the swing of the spreader, the pixel coordinates of the spreader can still be obtained through other sets of marker units within the camera view. In addition, setting multiple sets of marker units can also ensure that there are markers to be recognized in various scenarios and various lighting conditions.

[0051] Since the angle between the straight line connecting two markers in the image and the horizontal line only differs from the angle between the straight line connecting two markers in the real world and the horizontal line by the rotation angle of the straight line connecting the markers during installation, and this rotation angle is fixed. Therefore, the present invention can obtain the rotation angle of the spreader around the lifting direction according to the angle between the straight line connecting two markers in a certain set of marker units in the real-time image and the rotation angle of the straight line connecting the markers during installation.

[0052] The experimental results of the QQCTU yard crane are as follows: The offset errors of the spreader attitude in the movement directions of the trolley and the gantry can both be controlled within 3 cm, and the fluctuations can be controlled within 1 cm; the rotation angle error of the spreader around the lifting direction can be controlled within 0.5°, and the fluctuations can be controlled within 0.2°.

[0053] In summary, the monocular measurement method for spreader attitude detection provided in Embodiment 1 can accurately measure the actual offset of the spreader along the movement directions of the trolley and the gantry and the rotation angle of the spreader around the lifting direction.

[0054] Embodiment 2:

[0055] Embodiment 2 provides a monocular measurement system for spreader attitude detection, including: a monocular camera, at least one set of marker units, and a data processing module; the monocular camera is used to acquire real-time images containing markers; the data processing module is used to execute the steps in the monocular measurement method for spreader attitude detection as described in Embodiment 1 to obtain the actual offset of the spreader along the movement directions of the trolley and the gantry and the rotation angle of the spreader around the lifting direction.

[0056] See Figure 2 , the trolley 1 is arranged on the large vehicle 2, the moving direction of the trolley 1 is perpendicular to the moving direction of the large vehicle 2, the spreader 5 is suspended below the trolley 1, the monocular camera 3 is vertically installed directly below the center of the frame of the trolley 1, and at least one set of marking units is arranged on the spreader 5, and each set of marking units includes two markers 4 that are symmetric about the center of the spreader 5.

[0057] Figure 3 FIG. is a physical diagram including two markers photographed by the monocular measurement system suitable for spreader attitude detection provided in Embodiment 2, wherein Figure 3 the two markers are framed by the two red boxes in.

[0058] Since the functions of the devices or modules in the monocular measurement system suitable for spreader attitude detection provided in Embodiment 2 correspond to the steps in the monocular measurement method suitable for spreader attitude detection provided in Embodiment 1, reference can be made to the description of Embodiment 1 for understanding, and details are not described herein again.

[0059] Finally, it should be noted that the above specific embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the examples, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A monocular measurement method applicable to sling attitude detection, characterized in that, It includes the following steps: Set at least one set of marking units on the spreader, and vertically install the monocular camera directly below the center of the trolley frame; wherein, the trolley is arranged on the gantry, the moving direction of the trolley is perpendicular to the moving direction of the gantry, the spreader is suspended below the trolley, and each set of marking units includes two markers symmetric about the center of the spreader; Use the monocular camera to obtain a real-time image including the markers; Based on the real-time image, use the object detection model to obtain the real-time pixel coordinates of the markers; Obtain the real-time pixel coordinates of the spreader based on the real-time pixel coordinates of the markers; Compare the real-time pixel coordinates of the spreader with the static pixel coordinates of the spreader to obtain the pixel offset value of the spreader; Based on the pixel offset value of the spreader, obtain the actual offset amount of the spreader along the moving directions of the trolley and the gantry; based on the real-time pixel coordinates of the markers, obtain the rotation angle of the spreader around the lifting direction.

2. The monocular measurement method applicable to sling attitude detection according to claim 1, characterized in that, Set multiple sets of marking units with different colors or textures on the spreader; the focal length of the monocular camera is configured to cover the entire movement range of the spreader between the highest point and the lowest point, and the monocular camera can see at least one set of marking units when obtaining the real-time image.

3. The monocular measurement method applicable to sling attitude detection according to claim 1, characterized in that The object detection model uses YOLO or CenterNet.

4. The monocular measurement method applicable to sling attitude detection according to claim 1, wherein Obtaining the actual offset amount based on the pixel offset value includes: calibrating the monocular camera, and combining the distance from the optical center of the monocular camera to the spreader, and mapping the pixel coordinates to a three-dimensional rectangular coordinate system with the camera as the origin.

5. The monocular measurement method for spreader posture detection according to claim 4, characterized in that: Calibrating the monocular camera includes: after performing distortion correction on the monocular camera, calibrating the actual length represented by a single pixel on the plane where the spreader is located when the spreader is at different heights.

6. The monocular measurement method applicable to sling attitude detection according to claim 5, characterized in that, When the spreader is at different heights, respectively obtain the pixel distance between the pixel coordinates of the two markers in a certain set of marking units in the image; according to the pixel distance between the pixel coordinates of the two markers in the image and the actual distance between the two markers, calibrate the actual length represented by a single pixel on the plane where the spreader is located when the spreader is at different heights.

7. The monocular measurement method applicable to sling attitude detection according to claim 6, characterized in that Take the average value of the pixel coordinates of the two markers in the image to obtain the pixel coordinates of the spreader at different heights.

8. The monocular measurement method applicable to sling attitude detection according to claim 4, characterized in that, Obtain the distance from the optical center of the monocular camera to the spreader in the following way: obtain the measured distance from the optical center of the monocular camera to the spreader through the scanning system, and correct the measured distance according to the pixel size and the actual size of the markers in the real-time image.

9. The monocular measurement method applicable to sling attitude detection according to claim 1, characterized in that Obtain the included angle between the straight line connecting the two markers in a certain set of marking units in the real-time image and the horizontal line, and the included angle between the straight line connecting the two markers when the spreader is stationary and the horizontal line, and obtain the rotation angle of the spreader around the lifting direction based on the difference between the above two included angles.

10. A monocular measurement system applicable to sling attitude detection, characterized in that, It includes: A monocular camera, at least one set of marker units, and a data processing module; the monocular camera is used to obtain a real-time image containing markers; the data processing module is used to execute the steps in the monocular measurement method for detecting the attitude of a spreader applicable as described in any one of claims 1-9 to obtain the actual offset of the spreader in the moving directions of the trolley and the gantry, and the rotation angle of the spreader around the lifting direction.

Images