A single-lamp method for detecting the posture of a sling to prevent wire rope obstruction

Abstract

This invention proposes a single-lamp, wire rope-blocking sling posture detection method, comprising the following steps: S1. Capturing an image of a square infrared lamp; S2. Using the captured image, determining whether the square infrared lamp is obstructed by a wire rope; S3. If there is no obstruction, selecting the contour with the largest area in the image and recording it as the target contour; if there is obstruction, finding the minimum bounding rectangle corresponding to all contours, and obtaining the target contour based on the minimum bounding rectangle; S4. Obtaining the four sides of the target contour and performing linear fitting on each of the four sides, finding the intersection of two adjacent straight lines to obtain the sub-pixel location of the four vertices of the square infrared lamp in the image; S5. Based on the sub-pixel location of the four vertices, obtaining the coordinates of the four vertices in the world coordinate system and calculating the position of the square infrared lamp relative to the camera. This single-lamp, wire rope-blocking sling posture detection method can improve the accuracy and stability of sling posture detection.

Description

Technical Field

[0001] The present invention relates to the technical field of port automation, and in particular to a method for detecting a spreader posture with a single lamp to prevent a steel wire rope from blocking the spreader. Background Art

[0002] With the continuous development of port automation, the automatic container operation system has been continuously improved and upgraded. In order to prevent the port's container grabbing and stacking spreaders from excessive deviation and collision with the line during the process of grabbing and stacking containers, in the traditional automatic operation process, the spreader is often detected and positioned by a camera installed at the bottom of the yard crane trolley. For example, the "A container truck spreader detection method and system" with publication number CN112184800A uses a camera to detect the posture of the spreader and container. However, in the existing solution, three infrared lights (850nm) are installed on the spreader, and the camera captures the infrared lights. Then, the posture of the spreader is detected through image processing algorithms combined with posture estimation algorithms.

[0003] This method for detecting the attitude of the spreader has many shortcomings. First, when the spreader is at a high position from the ground, the three lights are often out of the camera's field of view, resulting in failure in determining the spreader's attitude. Second, in order to ensure that the three infrared lights are within the field of view during the entire lifting process of the spreader, the luminous area of ​​the infrared lights is limited, resulting in the ellipse formed by the infrared lights in the image occupying relatively few pixels, affecting the positioning of the lamp core and, in turn, the measurement accuracy of the spreader's attitude. Finally, there is wire rope obstruction. Since the luminous area of ​​the lights is not large enough, the ellipse formed by the infrared lights in the image occupies relatively few pixels. After the lights in the image are cut in half, it is difficult to reassemble them through the algorithm. Summary of the Invention

[0004] In view of this, the present invention proposes a single-lamp anti-wire rope obstruction spreader posture detection method, which aims to solve many problems existing in the traditional spreader posture detection method based on three lights and improve the spreader posture detection accuracy and stability.

[0005] The technical solution of the present invention is achieved as follows: The present invention provides a single-lamp anti-wire rope obstruction spreader posture detection method, which is implemented based on a detection and positioning system. The detection and positioning system includes a camera installed on a field crane trolley and a single square infrared lamp installed on the field crane spreader. The camera is used to capture images of the square infrared lamp. The detection method includes the following steps:

[0006] S1, collect images of the square infrared light;

[0007] S2, judging whether the square infrared lamp is blocked by the wire rope through the collected image;

[0008] S3. If there is no occlusion, the contour with the largest area in the image is selected and recorded as the target contour; if there is occlusion, the minimum bounding rectangle corresponding to all contours is obtained, and the target contour is obtained based on the minimum bounding rectangle;

[0009] S4. Obtain the four edges of the target contour, perform straight line fitting on the four edges, find the intersection of two adjacent straight lines, and obtain the sub-pixel positioning of the four vertices of the square infrared light in the image;

[0010] S5. Calculate the position of the square infrared light relative to the camera based on the sub-pixel positioning of the four vertices and the coordinates of the four vertices in the world coordinate system.

[0011] On the basis of the above technical solution, preferably, in step S5, the posture of the field crane relative to the camera is solved by calculating the posture transformation from the world coordinate system where the square lamp is located to the camera coordinate system.

[0012] Further preferably, the sub-pixel positioning coordinates of the four vertices obtained in step S4 are P1 (u1, v1), P2 (u2, v2), P3 (u3, v3), and P4 (u4, v4). In step S5, the coordinates of the four vertices in the world coordinate system are P w1 (X w1 , Y w1 , Z w1 )、P w2 (X w2 , Y w2 , Z w2 )、P w3 (X w3 , Y w3 , Z w3 )、P w4 (X w4 , Y w4 , Z w4 ), the calculation formula of the square infrared light's position relative to the camera is:

[0013]

[0014] Among them, u and v are the sub-pixel coordinates of the vertices of the square infrared light in the image coordinate system, and f x With f y is the pixel focal length, the coordinates of the camera optical center in the camera coordinate system are (u0, v0), Z c is the scale factor, M1 is the camera intrinsic parameter, M2 is the camera extrinsic parameter, R is the rotation matrix, and T is the translation matrix.

[0015] Further preferably, any point in the world coordinate system is P W (X W , Y W, Z W ), which is PC (X C , Y C , Z C ),but:

[0016]

[0017] Among them, R is the rotation matrix and T is the translation matrix.

[0018] More preferably, the geometric center of the field bridge hoist coincides with the geometric center of the square infrared lamp, and the coordinate of the geometric center of the field bridge hoist in the camera coordinate system is P c_spreader (X c_spreader , Y c_spreader , Z c_spreader ), that is, the position of the geometric center of the field crane in the camera coordinate system is:

[0019] .

[0020] On the basis of the above technical solution, preferably, before performing step S1, the method further includes correcting the posture of the square infrared lamp by calibrating the camera's external parameters.

[0021] Further preferably, when the height of the field crane is 1-5 container heights, the heights of the field crane are h1, h2, h3, h4, and h5 respectively, and the coordinates of the geometric center of the field crane measured by the camera in the actual coordinate system of the camera are P C1 (X C1 , Y C1 , Z C1 )、P C2 (X C2 , Y C2 , Z C2 )、P C3 (X C3 , Y C3 , Z C3 )、P C4 (X C4 , Y C4 , Z C4 )、P C5 (X C5 , Y C5 , Z C5 ), are both denoted as P Ci , where i is an integer from 1 to 5, and the corresponding coordinates in the camera standard coordinate system are P W1 (0, 0, h1), P W2 (0, 0, h2), P W3 (0, 0, h3), P W4 (0, 0, h4), P W5(0, 0, h5), all denoted as P Wi , where i is an integer from 1 to 5, establish the least squares optimization objective function:

[0022]

[0023] Among them, R W2C is the pose correction matrix.

[0024] Based on the above technical solution, preferably, step S2 includes the following sub-steps:

[0025] S21, recording the collected image as the original image, and copying it to obtain an image copy;

[0026] S22, performing image processing on the original image and the image copy respectively;

[0027] S23, extracting and collecting all contours of the original image and the processed image copy respectively;

[0028] S24, filtering out the circumscribed rectangle corresponding to the largest contour area of ​​the original image;

[0029] S25, calculating the centroid of all contours of the image copy;

[0030] S26. Filter out, from all contours of the image copy, all contours whose centroids are within the circumscribed rectangle corresponding to the maximum contour area of ​​the original image;

[0031] S27. Calculate the number of screened contours, and determine whether the square infrared lamp is blocked by the wire rope based on the number of contours.

[0032] Further preferably, the image processing in step S22 includes grayscale conversion, threshold segmentation and dilation and corrosion processing of the original image, and grayscale conversion and threshold segmentation processing of the image copy.

[0033] More preferably, the image processing in step S22 also includes reducing the original image before grayscale processing, and in step S24, it also includes enlarging the circumscribed rectangle corresponding to the maximum contour area of ​​the original image, wherein the product of the reduction ratio of the original image and the enlargement ratio of the circumscribed rectangle is equal to 1.

[0034] The single-lamp anti-wire rope obstruction spreader posture detection method of the present invention has the following beneficial effects compared with the prior art:

[0035] (1) By setting a single square infrared lamp for attitude detection, the problem in traditional technology that three infrared lamps are often out of the camera's field of view when the spreader is at a high position from the ground can be effectively avoided, or in order to prevent this situation, the luminous area of ​​the infrared lamp is limited to avoid the three infrared lamps being out of the field of view during the entire lifting process of the spreader. However, this will cause the ellipse formed by the infrared lamp to occupy fewer pixels in the image, affecting the positioning of the lamp core and thus affecting the accuracy of the spreader attitude measurement. In other words, this method can improve the accuracy and stability of spreader attitude detection;

[0036] (2) By performing image processing on the original image and the image copy, the number of contours can be screened, and then whether there is a wire rope obstruction can be determined. This can effectively avoid the situation where the captured image is blocked by the wire rope and affects the posture detection of the spreader. At the same time, if there is no wire rope obstruction, the spreader posture can be obtained more quickly than when there is a wire rope obstruction.

[0037] (3) By calibrating the camera's external parameters and correcting the sling's posture, the sling's posture can be further accurately obtained, the installation error can be corrected, and the detection accuracy can be further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 This is a flowchart of the steps of the method for detecting the posture of a sling with a single lamp to prevent the wire rope from blocking it according to the present invention;

[0040] Figure 2 This is a structural diagram of a detection and positioning system for a method for detecting a spreader posture with a single lamp to prevent wire rope obstruction according to the present invention;

[0041] Figure 3 This is a block diagram of the image processing and judgment steps of the method for detecting the posture of a spreader with a single light to prevent the wire rope from blocking the spreader according to the present invention;

[0042] Figure 4 Schematic diagram of the circumscribed rectangle of the single-lamp anti-wire rope obstruction spreader posture detection method without wire rope obstruction of the present invention;

[0043] Figure 5 This is a schematic diagram of the single-lamp anti-wire rope obstruction spreader posture detection method without wire rope obstruction outline of the present invention;

[0044] Figure 6This is a schematic diagram of sub-pixel positioning of four vertices in the method for detecting the posture of a spreader with a single lamp to prevent wire rope obstruction according to the present invention when there is no wire rope obstruction;

[0045] Figure 7-10 Schematic diagram of wire rope obstruction outlines in different situations using the single-lamp anti-wire rope obstruction spreader posture detection method of the present invention;

[0046] Figure 11 Schematic diagram of contour collection when a wire rope is blocked in the method for detecting the posture of a sling with a single lamp to prevent wire rope blocking according to the present invention;

[0047] Figure 12 Schematic diagram of target outline of the method for detecting the posture of a sling with a single lamp to prevent wire rope obstruction under the condition of wire rope obstruction of the present invention;

[0048] Figure 13 A schematic diagram of sub-pixel positioning of four vertices in the method for detecting the posture of a spreader with a single lamp to prevent wire rope obstruction according to the present invention when the spreader is obstructed by a wire rope;

[0049] Figure 14 Schematic diagram of posture settlement of the method for detecting the posture of a spreader with a single lamp to prevent wire rope obstruction according to the present invention;

[0050] Figure 15 The X on the infrared square lamp is the single lamp anti-wire rope blocking sling posture detection method of the present invention. w -Y w Schematic diagram of the floor plan;

[0051] Figure 16 This is a schematic diagram of the actual installation deviation of the camera in the single-lamp anti-wire rope obstruction sling posture detection method of the present invention. DETAILED DESCRIPTION

[0052] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0053] like Figure 1-15 As shown, the single-lamp anti-wire rope obstruction spreader posture detection method of the present invention is implemented based on a detection and positioning system. The detection and positioning system is as follows: Figure 2As shown, A represents a camera, B represents a square infrared lamp, and C represents that the detection and positioning system includes a camera arranged on the field bridge trolley and a single square infrared lamp arranged on the field bridge hoist. The camera is used to capture images of the square infrared lamp. The camera is installed at the bottom of the trolley, and the square infrared lamp is installed on the hoist. In theory, as long as there is extra installation space on the hoist, the square infrared lamp can be installed at any position of the hoist. The camera position is determined by the installation position of the square infrared lamp. The camera can be almost directly above the square infrared lamp. Here, the hoist is installed in the center of the hoist by default. The detection method includes steps S1-S5.

[0054] Step S1: Capture an image of the square infrared light.

[0055] During the operation, the camera on the yard crane trolley takes pictures of the square infrared light to complete the image acquisition operation.

[0056] Step S2: Determine whether the square infrared lamp is blocked by the wire rope based on the collected image.

[0057] Before determining whether the collected square infrared light image is blocked by the wire rope, the image needs to be processed first, and then the number of contours in the image is used to determine whether it is blocked by the wire rope. In this embodiment, when the number of contours in the image is one, it indicates that there is no wire rope blocking. If the number of contours in the image is greater than one, it indicates that there is wire rope blocking. The specific processing and judgment method includes steps S21-S27.

[0058] Step S21: record the collected image as the original image, and copy it to obtain an image copy.

[0059] After collecting the current image of the square infrared lamp, it is copied to obtain the original image and the image copy. The original image and the image copy are exactly the same. It should be noted that the original image and the image copy mentioned in this embodiment are only for the convenience of distinction. The subsequent processing and judgment process can refer to steps S22-S27, and during processing, the processing steps of the original image and the image copy can be interchanged without affecting the final result.

[0060] Step S22: performing image processing on the original image and the image copy respectively.

[0061] The image processing steps for the original image are shrinking, grayscale, threshold segmentation and dilation and corrosion processing of the original image. The image processing steps for the copy image are grayscale and threshold segmentation of the image copy. By shrinking the original image and performing dilation and corrosion processing, the maximum contours are connected and the influence of factors such as wire rope occlusion is removed. The grayscale and threshold segmentation of the original image and the image copy are both preparations for contour extraction.

[0062] Step S23: extracting and collecting all contours of the original image and the processed image copy respectively.

[0063] The contour of the original image after being reduced, grayed, thresholded, and processed by dilation and corrosion is extracted, and all contours are collected. The contour of the image copy after being grayed and thresholded is extracted, and all contours are collected.

[0064] Step S24: Filter out the circumscribed rectangle corresponding to the largest contour area of ​​the original image.

[0065] All the contours collected from the original image are extracted and filtered to obtain the bounding rectangle corresponding to the largest contour area of ​​the original image. It should be noted that the bounding rectangle here is not the contour, but the minimum bounding rectangle outside the largest contour.

[0066] Step S25: Calculate the centroids of all contours of the image copy.

[0067] Step S26: Filter out from all contours of the image copy all contours whose centroids are within the circumscribed rectangle corresponding to the maximum contour area of ​​the original image.

[0068] Since the original image is reduced during image processing, in order to match the contour in the image copy, the circumscribed rectangle corresponding to the maximum contour area of ​​the original image needs to be enlarged. It should be noted that the product of the reduction ratio of the original image and the enlargement ratio of the circumscribed rectangle is equal to 1. In this embodiment, the reduction ratio of the original image is 0.5 and the enlargement ratio is 2.

[0069] After enlarging the outline circumscribed rectangle of the largest outline area of ​​the original image, it is placed in the image copy. Using this circumscribed rectangle, all contours in the image copy whose center of mass is within the circumscribed rectangle are filtered out, thereby removing contours whose center of mass is outside the circumscribed rectangle and eliminating external interference factors. The contours filtered out here include the outermost contour of the square infrared lamp in the image copy, and if there is a wire rope obstruction, the contour obstructed by the wire rope is also included.

[0070] Since the original image has been dilated and eroded, the contour distortion is quite serious. If there is external light interference, the maximum contour extracted will have a large error. Therefore, it is only used here as an auxiliary contour screening. The contour extracted from the image copy has not been dilated and eroded, ensuring the authenticity of the contour.

[0071] Step S27: Calculate the number of screened contours, and use the number of contours to determine whether the square infrared lamp is blocked by the wire rope.

[0072] like Figure 4-5 As shown in the figure, when there is no wire rope occlusion, the obtained contour is a whole, but when there is a wire rope occlusion, it is divided into Figure 7-10The several cases shown include horizontal occlusion, vertical occlusion, cross occlusion and oblique occlusion. Regardless of the occlusion situation, the number of contours obtained will be greater than one.

[0073] Step S3: If there is no occlusion, the contour with the largest area in the image is screened out and recorded as the target contour; if there is occlusion, the minimum bounding rectangle corresponding to all contours is obtained, and the target contour is obtained based on the minimum bounding rectangle.

[0074] When it is judged that there is no occlusion, the small contours are filtered out, and the contour with the largest area is selected from the image copy, that is, the square infrared light contour, which is recorded as Contour here, and this contour is used as the target contour.

[0075] When it is determined that there is occlusion, all the contours collected in step S26 are first expanded and then eroded to obtain the circumscribed rectangle of the square lamp contour, and this circumscribed rectangle is recorded as retangle. It should be noted that this circumscribed rectangle is obtained by processing the image copy, not the circumscribed rectangle obtained by processing the original image in step S26. At this time, all the contours whose centroids are within the retangle among the contours extracted from the image copy are collected. The contours extracted here are not subjected to the expansion and corrosion processing, and the extracted contours are represented by a set, which is recorded as

[0076] ,like Figure 11 As shown, taking cross occlusion as an example, Contains contours Contour1, Contour2, Contour3, and Contour4. Obtain the minimum enclosing rectangle. According to the center, length, width and azimuth of the enclosing rectangle, it is easy to obtain the rectangular target outline. The obtained target outline is as follows: Figure 12 shown.

[0077] Step S4: Obtain the four sides of the target contour, and perform straight line fitting on the four sides respectively, find the intersection points of two adjacent straight lines, and obtain the sub-pixel positioning of the four vertices of the square infrared lamp in the image.

[0078] For the case where there is no wire rope blocking, it is easy to get the four sides of the rectangle through the center point of the circumscribed rectangle of the square lamp outline, as well as the length, width and azimuth of the circumscribed rectangle, such as Figure 5 As shown, the four sides of the extracted rectangular outline are recorded as line1, line2, line3, and line4 respectively. The following is a straight line fitting of the outlines of the four sides. Assume that the first side has n points, and the i-th point pi is recorded as (x 1i ,y 1i ), the set of contour points is recorded as , use the linear regression formula to solve the straight line corresponding to this edge, the formula is as follows:

[0079]

[0080]

[0081] but The desired straight line is denoted as L1.

[0082] Similarly, the other three edges are fitted with straight lines to obtain the straight lines of the other three edges, which are recorded as L2, L3, and L4 respectively. Their expressions are as follows:

[0083]

[0084] Now we can find the intersection of two adjacent straight lines to get the sub-pixel positioning of the four vertices of the square infrared light in the image, which are recorded as P1, P2, P3, and P4 respectively. Figure 6 shown.

[0085] For the case of wire rope occlusion, the four edges of the target contour are recorded as: And perform straight line fitting on the four edges respectively, and obtain the sub-pixel coordinates through the obtained straight lines and intersection points, and record them as P1, P2, P3, and P4 respectively, as shown in Figure 13 shown.

[0086] It should be noted that if the space on the sling allows, the light-emitting surface of the square infrared lamp can be appropriately increased, and the points of the four side contours of the square lamp in the image can be increased to improve the accuracy of straight line fitting, thereby obtaining more accurate sub-pixel intersection coordinates.

[0087] Step S5: Calculate the position of the square infrared light relative to the camera based on the sub-pixel positioning of the four vertices and the coordinates of the four vertices in the world coordinate system.

[0088] like Figure 14 As shown in the figure, the pose settlement is mainly solved by calculating the pose transformation from the world coordinate system Ow where the square lamp is located to the camera coordinate system Oc to solve the pose of the hanger relative to the camera.

[0089] Based on the above obtained four vertex sub-pixel coordinates of the square infrared light in the image P1 (u1, v1), P2 (u2, v2), P3 (u3, v3), P4 (u4, v4), as shown Figure 15 As shown, assuming that the length of the square infrared lamp is a, the width is b, and the geometric center of the square infrared lamp is taken as the origin, X w -Y w The plane is fixed on the upper surface of the square infrared lamp, and the world coordinate system is established. The world coordinates of the four vertices of the square infrared lamp are: P w1 (-0.5a, -0.5b, 0), P w2(0.5a, -0.5b, 0), P w3 (0.5a, 0.5b, 0), P w4 (-0.5a, 0.5b, 0), for the convenience of description, it is recorded as: P w1 (X w1 , Y w1 , Z w1 )、P w2 (X w2 , Y w2 , Z w2 )、P w3 (X w3 , Y w3 , Z w3 )、P w4 (X w4 , Y w4 , Z w4 ).

[0090] According to the camera pinhole model formula:

[0091]

[0092] Among them, u and v are the sub-pixel coordinates of the vertices of the square infrared light in the image coordinate system, and f x With f y is the pixel focal length, the coordinates of the camera optical center in the camera coordinate system are (u0, v0), Z c is the scale factor (Z c is not 0), represents the effective focal length (the distance from the optical center to the image plane), M1 is the camera intrinsic parameter, which can be obtained through camera calibration, and M2 is the camera extrinsic parameter, that is, the transformation matrix of the square infrared lamp world coordinate system relative to the camera coordinate system, which represents the translation and rotation of the square infrared lamp relative to the camera. R is the rotation matrix, and T is the translation matrix.

[0093] The sub-pixel coordinates of the four vertices of the square infrared light in the image are P1 (u1, v1), P2 (u2, v2), P3 (u3, v3), and P4 (u4, v4). In step S5, the coordinates of the four vertices in the world coordinate system are P w1 (X w1 , Y w1 , Z w1 )、P w2 (X w2 , Y w2 , Z w2 )、P w3 (X w3 , Y w3 , Z w3 )、P w4 (X w4 , Y w4 , Zw4 ), the calculation formula of the square infrared light's position relative to the camera is:

[0094]

[0095] By combining the above formulas, we can solve M2, which is the rotation matrix R and the translation matrix T. In this way, we can get the position of the square infrared lamp relative to the camera, that is, the position of the sling relative to the camera.

[0096] That is to say, the world coordinate system coincides with the camera coordinate system after rotation (R) and translation (T). Let any point P in the world coordinate system be W (X W , Y W , Z W ), any point in the world coordinate system is P W (X W , Y W , Z W ), which is PC (X C , Y C , Z C ),but:

[0097] .

[0098] Here, the coordinates of the geometric center of the hanger in the world coordinate system of the square lamp are: P spreader (X spreader ,Y spreader , Z spreader ), the geometric center of the field bridge hoist coincides with the geometric center of the square infrared lamp, that is, the geometric center of the hoist coincides with the world coordinate system, and the coordinate of the geometric center of the field bridge hoist in the camera coordinate system is P c_spreader (X c_spreader , Y c_spreader , Z c_spreader ), whose coordinate in the world coordinate system is P w_spreader (X w_spreader , Y w_spreader ,Z w_spreader ), that is, the position of the geometric center of the field crane in the camera coordinate system is:

[0099]

[0100] By analyzing the rotation matrix, we can get the rotation angle, pitch angle, and roll angle of the spreader (world coordinate system) relative to the camera (camera coordinate system).

[0101] Ideally, the camera's optical axis (the Z axis of the camera coordinate system) is perpendicular to the ground plane. During the hoist's height increase, the X and Y values ​​of the hoist's geometric center in the camera coordinate system remain unchanged. However, during actual camera installation, it's not guaranteed that the camera's optical axis (the Z axis of the camera coordinate system) is perpendicular to the ground plane. As a result, the X and Y coordinates of the hoist's geometric center in the camera coordinate system vary linearly with the hoist's height during the increase in height. This necessitates calibration of the camera's position.

[0102] When the camera is installed at the bottom of the trolley, it cannot ensure that the camera optical axis is perpendicular to the ground plane. This causes the plane where the X-axis and Y-axis of the camera coordinate system are located to be not horizontal, and the Z-axis is not vertically downward. As a result, when the spreader is at different heights, the x-value and y-value of the coordinates of the spreader's geometric center change with the height of the spreader. Therefore, it must be corrected through camera external parameter calibration.

[0103] like Figure 16 As shown, ideally, the horizontal plane is parallel to the ground plane, and the camera is completely perpendicular to the horizontal plane. At this time, in the camera coordinate system, the X axis and the Y axis are in the same horizontal plane, and the Z axis is vertically downward. At this time, the camera coordinate system is O N (Camera standard coordinate system), in fact, due to installation errors, the ideal condition cannot be guaranteed. At this time, the camera coordinate system is O C (Actual camera coordinate system), where the ICP matching calibration and installation error correction can be performed based on the point cloud.

[0104] In the specific calibration process, when the height of the crane is 1-5 container heights, the heights of the crane are h1, h2, h3, h4, and h5 respectively. The coordinates of the geometric center of the crane measured by the camera in the actual coordinate system of the camera are P C1 (X C1 , Y C1 , Z C1 )、P C2 (X C2 , Y C2 , Z C2 )、P C3 (X C3 , Y C3 , Z C3 )、P C4 (X C4 , Y C4 , Z C4 )、P C5 (X C5 , Y C5 , Z C5 ), are both denoted as P Ci , where i is an integer from 1 to 5, and the corresponding coordinates in the camera standard coordinate system are PW1 (0, 0, h1), P W2 (0, 0, h2), P W3 (0, 0, h3), P W4 (0, 0, h4), P W5 (0, 0, h5), all denoted as P Wi , where i is an integer from 1 to 5, establish the least squares optimization objective function:

[0105]

[0106] Among them, R W2C is the pose correction matrix, and the nonlinear optimization library (such as ceres, g2o, nlpot, etc.) is used to perform nonlinear optimization on the objective function to obtain R W2C .

[0107] The coordinates of the geometric center of the spreader in the camera coordinate system are calibrated as P. c_real_spreader (X c_real_spreader , Y c_real_spreader , Z c_real_spreader ) is corrected as follows:

[0108]

[0109] The left side shows the coordinates of the spreader's posture after calibration in the standard coordinate system, and the right side shows the coordinates of the spreader in the actual installation posture. As for the angular posture information of the spreader relative to the camera coordinate system, since the angle is very small and all are relative values, the user using the spreader posture can compensate for it by himself and no calibration is required. In this way, the calibration of the spreader posture data is completed.

[0110] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A single-lamp anti-wire rope obstruction spreader posture detection method, characterized in that: Based on the detection and positioning system, the detection and positioning system includes a camera installed on the field crane trolley and a single square infrared lamp installed on the field crane spreader. The camera is used to capture images of the square infrared lamp. The detection method includes the following steps: S1, collect images of the square infrared light; S2, judging whether the square infrared lamp is blocked by the wire rope through the collected image; S3. If there is no occlusion, the contour with the largest area in the image is selected and recorded as the target contour; if there is occlusion, the minimum bounding rectangle corresponding to all contours is obtained, and the target contour is obtained based on the minimum bounding rectangle; S4. Obtain the four edges of the target contour, perform straight line fitting on the four edges, find the intersection of two adjacent straight lines, and obtain the sub-pixel positioning of the four vertices of the square infrared light in the image; S5. Calculate the position of the square infrared light relative to the camera based on the sub-pixel positioning of the four vertices and the coordinates of the four vertices in the world coordinate system; The step S2 includes the following sub-steps: S21, recording the collected image as the original image, and copying it to obtain an image copy; S22, performing image processing on the original image and the image copy respectively; S23, extracting and collecting all contours of the original image and the processed image copy respectively; S24, filtering out the circumscribed rectangle corresponding to the largest contour area of ​​the original image; S25, calculating the centroid of all contours of the image copy; S26. Filter out, from all contours of the image copy, all contours whose centroids are within the circumscribed rectangle corresponding to the maximum contour area of ​​the original image; S27, calculating the number of screened contours, and judging whether the square infrared lamp is blocked by the wire rope based on the number of contours; The image processing in step S22 includes grayscale conversion, threshold segmentation, and dilation and corrosion processing of the original image, grayscale conversion and threshold segmentation processing of the image copy, and reduction processing of the original image before grayscale conversion; In step S24 , the process further includes enlarging the circumscribed rectangle corresponding to the maximum contour area of ​​the original image, wherein the product of the reduction ratio of the original image and the enlargement ratio of the circumscribed rectangle is equal to 1.

2. The method for detecting the posture of a sling with a single lamp to prevent the wire rope from blocking it according to claim 1, characterized in that: In step S5, the pose of the field crane relative to the camera is solved by calculating the pose transformation from the world coordinate system where the square lamp is located to the camera coordinate system.

3. The method for detecting the posture of a sling with a single lamp to prevent the wire rope from blocking it according to claim 2, wherein: The sub-pixel coordinates of the four vertices obtained in step S4 are P1 (u1, v1), P2 (u2, v2), P3 (u3, v3), and P4 (u4, v4). In step S5, the coordinates of the four vertices in the world coordinate system are P w1 (X w1 , Y w1 , Z w1 )、P w2 (X w2 , Y w2 , Z w2 )、P w3 (X w3 , Y w3 , Z w3 )、P w4 (X w4 , Y w4 , Z w4 ), the calculation formula of the square infrared light's position relative to the camera is: ； Among them, u and v are the sub-pixel coordinates of the vertices of the square infrared light in the image coordinate system, and f x With f y is the pixel focal length, the coordinates of the camera optical center in the camera coordinate system are (u0, v0), Z c is the scale factor, M1 is the camera intrinsic parameter, M2 is the camera extrinsic parameter, R is the rotation matrix, and T is the translation matrix.

4. The method for detecting the posture of a sling with a single lamp to prevent the wire rope from blocking it according to claim 2, wherein: Any point in the world coordinate system is P W (X W , Y W , Z W ), which is PC (X C , Y C , Z C ),but: ； Among them, R is the rotation matrix and T is the translation matrix.

5. The method for detecting the posture of a sling with a single lamp to prevent the wire rope from blocking it according to claim 4, characterized in that: Take the geometric center of the square infrared lamp as the origin, X w -Y w The plane is fixed on the upper surface of the square infrared lamp, and the world coordinate system is established. The geometric center of the field bridge hoist coincides with the geometric center of the square infrared lamp. The coordinate of the geometric center of the field bridge hoist in the camera coordinate system is P c_spreader (X c_spreader , Y c_spreader , Z c_spreader ), that is, the position of the geometric center of the field crane in the camera coordinate system is: 。 6. The method for detecting the posture of a sling with a single lamp to prevent the wire rope from blocking it according to claim 1, characterized in that: Before performing step S1, the position and posture of the square infrared lamp is corrected by calibrating the camera's external parameters.

7. The method for detecting the posture of a sling with a single lamp to prevent the wire rope from blocking it according to claim 6, characterized in that: When the height of the field crane is 1-5 container heights, the heights of the field crane are h1, h2, h3, h4, and h5 respectively. The coordinates of the geometric center of the field crane measured by the camera in the actual coordinate system of the camera are P C1 (X C1 , Y C1 , Z C1 )、P C2 (X C2 , Y C2 , Z C2 )、P C3 (X C3 , Y C3 , Z C3 )、P C4 (X C4 , Y C4 , Z C4 )、P C5 (X C5 , Y C5 , Z C5 ), are both denoted as P Ci , where i is an integer from 1 to 5, and the corresponding coordinates in the camera standard coordinate system are P W1 (0, 0, h1), P W2 (0, 0, h2), P W3 (0, 0, h3), P W4 (0, 0, h4), P W5 (0, 0, h5), all denoted as P Wi , where i is an integer from 1 to 5, establish the least squares optimization objective function: ； Among them, R W2C is the pose correction matrix.

Images