A Cuboid Detection and Parameter Estimation Method Based on Binocular Vision

Abstract

The present invention provides a method for detecting and estimating a cuboid based on binocular vision, wherein the method uses a binocular camera to photograph a target to be measured to obtain a two-dimensional image, and processes the two-dimensional image to obtain the structural parameters and posture parameters of the cuboid target; the process of processing the two-dimensional image specifically includes: extracting the edges of the cuboid; detecting and screening the polygons of the cuboid; identifying the key three-dimensional vertices of the cuboid; and estimating the structural parameters and posture parameters of the cuboid. The advantages of the present invention are: contactless and easy to install the equipment; more efficient than the method of manually extracting feature points or randomly extracting feature points, and the extraction effect is robust; the present application realizes the feature extraction of cuboid targets, and the algorithm can be applied to cuboid detection in a variety of scenarios, including non-cooperative cubic satellites in space, carton classification in industrial sites, etc., and has universal applicability.

Description

Technical Field

[0001] The present invention belongs to the fields of machine vision, target detection and target measurement, and in particular relates to a cuboid detection and parameter estimation method based on binocular vision. Background Art

[0002] In the space sector, emerging CubeSats, based on a 1U rectangular structure, are assembled into various sizes using structural components for small-scale space exploration. Aerial deployment and on-orbit maintenance of CubeSats require close-range, non-contact detection of parameters such as their position, attitude, and morphology. Existing detection technologies typically use laser detection or a combination of laser and visual detection, which are costly and complex. A low-cost, yet highly accurate, non-contact detection system for CubeSats is currently needed.

[0003] In the industrial field, contactless dimensional estimation of rectangular objects facilitates product sorting on assembly lines, but current technologies suffer from low efficiency and poor robustness. Using a single camera combined with structured light requires high precision in parallel mounting of the equipment. Traditional binocular vision solutions rely on manual selection of feature points in two images or random selection using feature point matching methods like SIFT, resulting in low efficiency and poor robustness. Summary of the Invention

[0004] The purpose of the present invention is to overcome the defects of traditional binocular vision solutions, which rely on manual selection of feature points in two images or randomly select corresponding feature points based on feature point matching methods such as SIFT, resulting in low efficiency and poor robustness.

[0005] In order to achieve the above object, the present invention proposes a cuboid detection and estimation method based on binocular vision, the method comprising:

[0006] Step 1: Extract the edges of the cuboid from the two-dimensional image of the target to be measured acquired by binocular vision;

[0007] Step 2: Detect and screen polygons of the cuboid;

[0008] Step 3: Identify the key three-dimensional vertices of the cuboid;

[0009] Step 4: Estimate the structural parameters and pose parameters of the cuboid.

[0010] As an improvement to the above method, step 1 is to extract the edges of the cuboid using the Canny edge detection algorithm.

[0011] As an improvement to the above method, step 2 specifically includes:

[0012] Polygon detection; extract the contour of the edge image, fit the approximate polyline of each contour by judging the threshold, and store the vertices of the continuous polyline segments in the form of a set D;

[0013] D is expressed as the following formula:

[0014] D={p1,p2,…p j ,...p n}

[0015] Among them, p j Represents the two-dimensional coordinates of a vertex of a polygon connected in sequence, n represents the number of sides of polygon D; select all polygons with n greater than 3 and save them into the set;

[0016] Set the vertex arrangement to clockwise as the positive direction;

[0017] Step 2-2: Filter polygons; filter polygons based on the pixel distance information between polygon vertices, retaining polygon information related to the target. The filtering process includes polygon deduplication and filtering out isolated polygons.

[0018] As an improvement of the above method, the approximate broken line fitting of each contour by judging the threshold is specifically as follows: judging whether the point with the largest distance from the head-to-tail line in the current contour is greater than the set threshold; if it is greater than the threshold, the curve is divided into two segments based on the maximum point and processed separately; otherwise, the curve is approximated as a line segment until all contours are approximated by broken line segments.

[0019] As an improvement to the above method, the step 2-2 polygon deduplication specifically includes: judging the distance between all vertices of two polygons. If the number of adjacent points of the polygons exceeds 3, the two polygons are determined to be duplicates, and the polygon with the larger area is retained to achieve deduplication.

[0020] As an improvement to the above method, step 2-2 of filtering out isolated polygons specifically includes: if two polygons belong to the same target, there are more than two adjacent vertices, and their vertex information is combined; if the number of adjacent vertices of the isolated polygon and other polygons is less than two, its vertex information is not combined.

[0021] As an improvement to the above method, step 3 specifically includes:

[0022] A pixel coordinate system is established based on the two-dimensional image acquired by the binocular camera. The origin of the pixel coordinate system is the O0 (0, 0) coordinate point in the upper left corner of the image, and the horizontal and vertical coordinates are the row and column where the image is located respectively;

[0023] In a two-dimensional image, a cuboid target appears as a combination of 2-3 rectangular planes. When the number of polygons in the left and right views is greater than or equal to 2, the center coordinates of each polygon are first calculated. The center coordinates are then connected in pairs to calculate the slope. The two pairs of polygons with the closest slopes in the left and right views are retained as target features. For each camera view, two adjacent polygon lists D1 and D2 are defined. The polygons are labeled according to their positional relationship: the polygon with the smaller sum of the horizontal and vertical center coordinates is set as D1, and the other is set as D2.

[0024] Select the two points with the closest corresponding pixel values ​​in the two polygon lists D1 and D2, which represent the common edge vertices C1 and C2 of the two target rectangles; set it as C1 when the difference between the sum of the horizontal and vertical coordinates of the two vertices is greater than the set threshold; when the difference between the sum of the coordinates of the two vertices is less than the set threshold, take the smaller vertical coordinate as C1; use this constraint to ensure that the vertices and polygons of the left and right views are in one-to-one correspondence;

[0025] After determining the common edge vertices C1 and C2, use the diagonal vertex judgment method, set the constant value N to 2, the weight of each edge to 1, and judge the upper vertex C of the target clockwise along the vertices of polygon D1 from vertex C2. up ; From vertex C1, follow the vertices of polygon D2 clockwise to determine the lower vertex C of the target down ;

[0026] The diagonal vertex determination method is: for a polygon, from a vertex p of the polygon x To the diagonal vertex p x The sum of the number of edges that diag passes through is a fixed value N, and the weight of the edge is set to the number of edges; different starting values ​​and starting vertices are set for polygons with different numbers of vertices, and N is set as the initial value. The step value is reduced by the difference between it and the weight of the edge until the step value is reduced to 0 and the corresponding vertex is reached; it can be expressed as:

[0027]

[0028] Among them, w ij Represents a vertex p x To the diagonal vertex p x The edges between diags.

[0029] As an improvement to the above method, step 3 further includes:

[0030] When the short side interference appears at the vertices of the cuboid target in the two-dimensional image, the number of vertices N of the polygon D If it is greater than 4, add N D The weights of the 4 shortest edges are set to 0, and they are considered invalid vertices when passing through the segment.

[0031] As an improvement to the above method, step 3 further includes:

[0032] When there are only two rectangular planes in the left and right eye views detected by the binocular camera, but only one pair of planes corresponds, and the other pair of planes are perpendicular to each other in space, the slopes of the plane center points cannot be matched; at this time, the judgment rule is first modified for a group of polygons in the left and right views. The rule is changed to: From vertex C1, rotate clockwise along the vertex of polygon D1 to obtain the upper vertex C up , rotate clockwise from vertex C2 along the vertices of polygon D2 to get the next vertex C down , the rules of the other perspective do not change, and judge whether the slopes of the current corresponding body diagonal vertices are consistent. If they are inconsistent, modify the judgment rules under the other view and find the corresponding body diagonal vertices; at this time, for the vertices C1 and C2 under the unmodified rule view, cross a vertex clockwise along the common surface to achieve the correspondence between vertices C1 and C2, and find the next vertex coordinates of C1 and C2 as the new C1 and C2.

[0033] As an improvement to the above method, step 4 specifically includes:

[0034] Construct the camera coordinate system O with the optical center of the camera as the origin of the coordinate system c -X c Y c Z c , X c Axis and Y c The axes are parallel to the horizontal and vertical axes of the image pixel coordinate system respectively; Z c The axis is the camera optical axis, which is perpendicular to the image plane;

[0035] The target coordinate system O is constructed with the geometric center of the cuboid target as the coordinate origin. axis -X axis Y axis Z axis , set X axis is the normal vector of the target front plane; Y axis is the normal vector of the right plane; Z axis is the normal vector of the upper plane; X axis 、Y axis and Z axis Determined by the cross product between points:

[0036]

[0037] in, is the vector expression between two points;

[0038] The structural parameter information of the cuboid target, length, width and height [a, b, H], is represented by the Euclidean distance between the above vertices:

[0039]

[0040] Target coordinate system O axis -X axis Y axis Z axis and the camera coordinate system O c -X c Y c Z c Pose parameters converted between: pitch angle, yaw angle and roll angle The expression is as follows:

[0041]

[0042] Among them, r 11 、r 21 、r 31 、r 32 and r 33 To use the calculated X axis ,Y axis ,Z axis The elements of the rotation matrix R of the rectangular target; X axis ,Y axis ,Z axis The three vectors are all 1×3 vectors, and the rotation matrix R is 3×3. The expression is as follows:

[0043]

[0044] Among them, r 11 For X axis The first element X axis (1), r 12 For X axis The second element X axis (2), r 13 For X axis The third element X axis (3), r 21 Y axis The first element Y axis (1), r 22 Y axis The second element Y axis (2), r 23 Y axis The third element Y axis (3), r 31 Z axis The first element Z axis (1), r 32 Z axis The second element Z axis (2), r33 Z axis The second element Z axis (3).

[0045] Compared with the prior art, the advantages of the present invention are:

[0046] 1. Non-contact, easy to install: The binocular camera is used to detect the target without contact and extract effective information from the image. Compared with the monocular combined with ranging system, the requirements for target placement are more stringent;

[0047] 2. Robustness and efficiency: Through edge detection, polygon detection, and screening, polygon groups representing the target are extracted. Then, a set of designed rectangular feature points is used to extract the target's effective feature points. This method is more efficient and robust than manual or random feature point extraction methods.

[0048] 3. Universality: This application realizes the feature extraction of rectangular targets. The algorithm can be applied to rectangular detection in various scenarios, including non-cooperative cubic satellites in space and cardboard classification in industrial sites, and has universality. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 The figure shows a schematic diagram of a cuboid detection and parameter estimation method based on binocular vision;

[0050] Figure 2 The figure shows a flow chart of a cuboid detection and parameter estimation method based on binocular vision;

[0051] Figure 3 The figure shows a schematic diagram of polygon deduplication;

[0052] Figure 4 The figure shows a schematic diagram of isolated polygon filtering;

[0053] Figure 5 The figure shows a schematic diagram of the two-dimensional image coordinate system obtained by the binocular camera;

[0054] Figure 6 The figure shows the schematic diagram of key point discrimination in the plane correspondence case;

[0055] Figure 7 The figure shows a special case diagram of vertex discrimination failure;

[0056] Figure 8 Schematic diagram of key point discrimination when the planes shown do not correspond;

[0057] Figure 9 Shown is a schematic diagram of the rectangular model. DETAILED DESCRIPTION

[0058] The technical solution of the present invention is described in detail below with reference to the accompanying drawings.

[0059] In response to the problems of high cost, low efficiency, and poor robustness faced by the detection and parameter estimation of rectangular targets, space cubesats, and other targets in the industrial assembly line field, this application proposes a rectangular detection and parameter estimation system based on binocular vision. The detection system is constructed using a binocular camera and a processor. The polygonal border of the target is detected through visual images without contact with the target, and the key stereo vertex discrimination method is used to detect the rectangular target. The position, posture, length, width, and height and other parameters of the target are estimated using binocular detection. It has good real-time, robust, and economical performance.

[0060] This application proposes a cuboid detection and estimation method based on binocular vision, which uses a binocular camera to photograph the target to be measured and processes the generated two-dimensional grayscale image to obtain the specifications and posture of the cuboid target.

[0061] The overall framework of this application is as follows Figure 1 As shown in the figure, the left and right view grayscale images of the binocular camera are used to generate binary edge images through edge extraction method. The polygon group of the rectangular target outline in the left and right view images is obtained through polygon detection and screening calculation. Then, the key vertices of the rectangular target are identified, and the three-dimensional coordinates of the key vertices of the rectangular target are calculated based on the left and right views. Finally, the specifications and posture of the rectangular target are solved.

[0062] like Figure 2 As shown in Figure 2, the cuboid detection and estimation method based on binocular vision includes:

[0063] Step 1: Edge extraction

[0064] The Canny edge detection method is widely used in the target edge extraction process. The geometric features of the target are extracted from the edge image, and subsequent pose estimation can be performed.

[0065] The basic steps to implement the Canny edge detection algorithm include:

[0066] 1) Gaussian filter smoothing;

[0067] 2) First-order derivatives calculate gradient magnitude and direction;

[0068] 3) Perform non-maximum suppression on the gradient amplitude;

[0069] 4) Use double threshold to determine the edge.

[0070] The edge information of the target is obtained through the above method and retained as a binary edge image for subsequent polygon detection and screening.

[0071] Step 2: Polygon detection and screening

[0072] Binarized edge image f edge (x, y) contains the effective information of the target. Polygon detection is performed on the image to extract rectangular features, and then the polygon group containing the target rectangle information is screened out for subsequent judgment.

[0073] Step 2-1: Polygon Detection

[0074] Polygon detection includes two parts: contour extraction and polygon approximation processing:

[0075] 1) Extract contours

[0076] First, contour extraction is performed on the binary edge image. The boundary tracking method proposed by Satoshi Suzuki is used to identify connected regions of binary image pixels, thereby achieving contour extraction. The detected boundaries are stored in an array as the coordinates of the contour inflection points, avoiding the data complexity required to retain all the contour pixels.

[0077] 2) Polygonal approximation processing

[0078] The Douglas-Peucker algorithm is used to iteratively fit the approximate polyline of each contour by judging the threshold: judging whether the point with the largest distance from the end line in the current contour is greater than the given threshold T r If it is greater than the threshold, the curve is divided into two segments at the maximum point and processed separately. Otherwise, the curve is approximated as a line segment until the entire contour is approximated by a broken line segment.

[0079] The vertices of continuous polyline segments are stored in the form of a set, that is, the vertex information of each polygon, polygon D i It is expressed as the following formula:

[0080] D i ={p1,p2,...p j ,…p n} (1)

[0081] where p j The two-dimensional coordinates of a vertex corresponding to the polygon connected in sequence, n corresponds to the polygon D i To retain valid rectangle information, all polygons with n greater than 3 are selected and saved into a collection for subsequent identification.

[0082] In order to ensure the consistency of vertex direction, it is stipulated that the vertices are arranged clockwise as the positive direction, that is, the calculation vector and Check whether the dot product result is less than 0. If it is less than 0, the vertices are arranged in clockwise order; otherwise, the vertices of the polygon are arranged in reverse order.

[0083] Step 2-2: Filter polygons

[0084] After detecting polygon features in the image, the polygons are filtered based on the pixel distance information between the polygon vertices, retaining the polygon information related to the target. The filtering process includes polygon deduplication and filtering out isolated polygons:

[0085] 1) Polygon deduplication

[0086] like Figure 3 As shown in the figure, multiple threshold segmentation in the contour detection process leads to duplication of polygon vertex lists, which interferes with judgment and needs to be deduplicated. Considering that all vertex coordinates of repeated polygons are the same or similar, it is necessary to judge the polygon D i and D j If the number of adjacent points of a polygon exceeds 3, the two polygons are considered duplicates, and the polygon with the larger area is retained to remove duplicates. The judgment formula is as follows:

[0087]

[0088] Among them, N Di Represents polygon D i The number of edges, N Dj Represents polygon D j The number of edges; |D i {p x}D j {p y}| represents polygon D i The pixel point p x With polygon D j The pixel point p y The Euclidean distance between r is the pixel distance threshold set.

[0089] 2) Filter out isolated polygons

[0090] like Figure 4 As shown in the figure, in addition to the target, there may also be isolated polygons that interfere with the judgment. It is necessary to combine the adjacent polygon information and filter out the isolated polygons: if polygon D i and D j If they belong to the same target, there will be more than two adjacent vertices, and their vertex information will be combined; if the number of adjacent vertices of an isolated polygon in the scene and the target plane is less than two, the corresponding polygon will not be combined. The judgment formula is as follows:

[0091]

[0092] Step 3: Identify key solid vertices

[0093] In order to obtain effective target key point information, this paper designs a set of algorithms for distinguishing target common edge vertices and body diagonal vertices based on the special positional relationship between target rectangular features, and specifically analyzes the discrimination methods in different situations.

[0094] like Figure 5 As shown in FIG, a pixel coordinate system is established based on the two-dimensional image obtained by the binocular camera. The origin of the pixel coordinate system is the coordinate point O0 (0, 0) in the upper left corner of the image, and the horizontal and vertical coordinates are the row u and column v where the image is located, respectively.

[0095] In the camera perspective, a cuboid target usually presents a combination of 2-3 rectangular planes. When the number of polygons in the left and right views is greater than or equal to 2, first calculate the center coordinates of each polygon, connect the center coordinates two by two to calculate the slope, and retain the two pairs of polygons with the closest slopes in the left and right views as target features; for each camera view, define two adjacent polygon lists as D i and D j , label the polygons according to their positional relationship: the polygon with the smaller sum of horizontal and vertical coordinates is designated as D1, and the other one is designated as D2.

[0096] Select the two points in the two polygon lists with the closest corresponding pixel values, which represent the common edge vertices C1 and C2 of the two target rectangles. The vertex with the smaller sum of the horizontal and vertical coordinates is specified as C1. When the sum of the coordinates is close to the same, the vertex with the smaller vertical coordinate is prioritized as C1. This is used as a constraint to ensure a one-to-one correspondence between the vertices and polygons of the left and right views.

[0097] like Figure 6 As shown, this application proposes a method for identifying polygon diagonal vertices. For a polygon S, from a vertex p of the polygon x To the diagonal vertex p x The sum of the edges traversed by the diag is a constant value, N. The number of edges is then converted into edge weights. In this case, the weight of each edge is a constant value of 1. For special cases, the edge weights may vary, as detailed in the following sections. For polygons with different numbers of vertices, different starting values ​​and starting vertices are set. N is set as the initial value, and the step value is subtracted from the edge weight until the step value reaches 0, reaching the corresponding vertex.

[0098]

[0099] Among them, w ij Represents a vertex p x To the diagonal vertex p x The edges between diags.

[0100] After determining the common edge vertices C1 and C2, use the diagonal vertex determination method from vertex C2 along the vertices of polygon D1 clockwise to determine the upper vertex C of the target up ; From vertex C1, follow the vertices of polygon D2 clockwise to determine the lower vertex C of the target down In general, polygon D i and D j The number of vertices is 4, N in the diagonal vertex judgment method is set to 2, and the weight of each edge is 1 without change.

[0101] like Figure 7 As shown in the figure, the early morphological processing may cause interference of short edges at the vertices of the target, resulting in the number of vertices N of the polygon D Greater than 4, then the polygon (N D -4) The weight of the shortest edge is set to 0, and it is considered an invalid vertex by default when passing through this segment, which realizes the judgment of special cases.

[0102] When the binocular camera detects only two rectangular planes in the left and right eye views, but only one pair of planes corresponds, and the other pair of planes are perpendicular to each other in space, the slopes of the plane center points cannot be matched, and the above judgment method will result in Figure 7 In the case shown, the diagonal vertices of the body are not one-to-one corresponding, and subsequent judgment cannot be performed.

[0103] like Figure 8 As shown in the figure, for this special case, the solution can be obtained by modifying the correspondence between the above vertices and polygons. First, modify the judgment rule for a group of polygons in the left and right views. The rule is changed to: From vertex C1, rotate clockwise along the vertices of polygon D1 to obtain the upper vertex C up , rotate clockwise from vertex C2 along the vertices of polygon D2 to get the next vertex C down The rules for the other view remain unchanged. The system checks whether the slopes of the corresponding diagonal vertices are consistent. If not, the rules for the other view are modified to find the corresponding diagonal vertices. At this point, for vertices C1 and C2 in the unmodified rule view, it is necessary to cross a vertex clockwise along the common surface to achieve the correspondence between C1 and C2. Since the vertices are already arranged in a clockwise direction, the coordinates of the next vertex after C1 and C2 are found as the new C1 and C2.

[0104] In the special case where only a single plane can be detected from a certain view, the single polygon information cannot satisfy the judgment of each vertex, nor can effective corresponding points be obtained in this case. For this, it is necessary to use a multi-eye joint approach to realize pose estimation; the target specifications can continue to be judged based on the information of a single surface, but there is a risk of misjudgment.

[0105] Step 4: Estimate target size, position and pose

[0106] After the above judgment, the corresponding vertices C1, C2, C up and C down , based on the parallax principle, the three-dimensional coordinate information of the above four vertices can be restored.

[0107] Construct the camera coordinate system O with the optical center of the camera as the origin of the coordinate system c -X c Y c Z c , X c Axis and Y c The axis is parallel to the u and v axes of the image pixel coordinates; Z c The axis is the camera optical axis, which is perpendicular to the image plane.

[0108] The target coordinate system O is constructed with the geometric center of the target as the coordinate origin axis -X axis Y axis Z axis , stipulate X axis is the normal vector of the target front plane; Y axis is the normal vector of the right plane; Z axis is the normal vector of the upper plane. axis 、Y axis and Z axis It can be determined by the cross product between points:

[0109]

[0110] In the formula is the vector expression between two points, since X axis and Y a ' xis Orthogonality is not guaranteed, the last step Y axis The cross product calculation ensures the orthogonality of the three axes.

[0111] For parallel binocular cameras, according to the characteristics of similar triangles, when the baseline length B of the two cameras, the focal length f and the pixel coordinates (x l ,y l ) and (x r ,y r ), derive the three-dimensional coordinates of the target point (X w ,Y w ,Z w ) is shown in formula (6):

[0112]

[0113] Among them, x l -xr Represents the disparity of the image, and x0, y0 represent the coordinates of the center point of the image.

[0114] The center coordinates (X, Y, Z) of the target can be expressed as formula (7), which is the translation matrix T, where (C up (x),C up (y),C up (z)) and (C down (x),C down (y),C down (z)) corresponds to point C up and C down The three-dimensional coordinates of:

[0115]

[0116] Depend on Figure 9 As shown, the target's structural parameter information length, width and height [a, b, H] can be represented by the Euclidean distance between the above vertices:

[0117]

[0118] Using the calculated X axis ,Y axis ,Z axis The rotation matrix R, X that can constitute the target axis ,Y axis ,Z axis The three vectors are all 1×3 vectors, and the rotation matrix R constructed according to formula (9) is 3×3. 11 For X axis The first element X axis (1), r 12 For X axis The second element X axis (2), and so on for the remaining 7 elements.

[0119]

[0120] Target coordinate system O axis -X axis Y axis Z axis and the camera coordinate system O c -X c Y c Z c Pitch, yaw and roll angles converted between It can be obtained by solving formula (10). When the two coordinate systems are relatively translated, the values ​​of the three Euler angles are 0°.

[0121]

[0122] Through the above calculations, the structural parameters [a, b, H] and posture parameters of the target are realized. Estimates.

[0123] An example of the technical solution of this application is to achieve non-contact measurement of a target by looking down at the target with a binocular camera: a binocular camera (such as Realsense D435i) is connected horizontally, with the optical axis tilted, and a certain distance (such as 40-70cm) from the target is maintained. The image of the target to be measured is collected by the camera and transmitted to the processor, which can be a PC or an embedded device (such as an 11th Gen Intel(R) Core(TM) i7-11800H processor), and the target size specification parameters [a, b, H] and the target position and posture parameters are calculated according to the formula derived in the article.

[0124] Finally, it should be noted that the above embodiments are intended only to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions to the technical solutions of the present invention do not depart from the spirit and scope of the technical solutions of the present invention and are intended to be encompassed by the claims of the present invention.

Claims

1. A method for detecting and estimating a cuboid based on binocular vision, the method comprising: Step 1: Extract the edges of the cuboid from the two-dimensional image of the target to be measured acquired by binocular vision; Step 2: Detect and screen polygons of the cuboid; Step 3: Identify the key three-dimensional vertices of the cuboid; Step 4: Estimate the structural parameters and pose parameters of the cuboid; The step 3 specifically includes: A pixel coordinate system is established based on the two-dimensional image acquired by the binocular camera. The origin of the pixel coordinate system is the O0 (0, 0) coordinate point in the upper left corner of the image, and the horizontal and vertical coordinates are the row and column where the image is located respectively; In a two-dimensional image, a cuboid target appears as a combination of 2-3 rectangular planes. When the number of polygons in the left and right views is greater than or equal to 2, the center coordinates of each polygon are first calculated. The center coordinates are then connected in pairs to calculate the slope. The two pairs of polygons with the closest slopes in the left and right views are retained as target features. For each camera view, two adjacent polygon lists D1 and D2 are defined. The polygons are labeled according to their positional relationship: the polygon with the smaller sum of the horizontal and vertical center coordinates is set as D1, and the other is set as D2. Select the two points with the closest corresponding pixel values ​​in the two polygon lists D1 and D2, which represent the common edge vertices C1 and C2 of the two target rectangles; set it as C1 when the difference between the sum of the horizontal and vertical coordinates of the two vertices is greater than the set threshold; when the difference between the sum of the coordinates of the two vertices is less than the set threshold, take the smaller vertical coordinate as C1; use this constraint to ensure that the vertices and polygons of the left and right views are in one-to-one correspondence; After determining the common edge vertices C1 and C2, use the diagonal vertex judgment method, set the constant value N to 2, the weight of each edge to 1, and judge the upper vertex C of the target clockwise along the vertices of polygon D1 from vertex C2. up ; From vertex C1, follow the vertices of polygon D2 clockwise to determine the lower vertex C of the target down ; The diagonal vertex determination method is: for a polygon, from a vertex p of the polygon x To the diagonal vertex p x The sum of the number of edges that diag passes through is a fixed value N, and the weight of the edge is set to the number of edges; different starting values ​​and starting vertices are set for polygons with different numbers of vertices, and N is set as the initial value. The step value is reduced by the difference between it and the weight of the edge until the step value is reduced to 0 and the corresponding vertex is reached; it can be expressed as: Among them, w ij Represents a vertex p x To the diagonal vertex p x The edges between diags.

2. The binocular vision-based cuboid detection and estimation method according to claim 1, characterized in that: The step 1 is to extract the edges of the cuboid using the Canny edge detection algorithm.

3. The binocular vision-based cuboid detection and estimation method according to claim 1, characterized in that: The step 2 specifically includes: Step 2-1: Polygon detection; perform contour extraction on the edge image, fit the approximate polyline of each contour by judging the threshold, and store the vertices of the continuous polyline segments in the form of a set D; D is expressed as the following formula: D={p1,p2,...p j ,...p n } Among them, p j Represents the two-dimensional coordinates of a vertex of a polygon connected in sequence, n represents the number of sides of polygon D; select all polygons with n greater than 3 and save them into the set; Set the vertex arrangement to clockwise as the positive direction; Step 2-2: Filter polygons; filter polygons based on the pixel distance information between polygon vertices, retaining polygon information related to the target. The filtering process includes polygon deduplication and filtering out isolated polygons.

4. The binocular vision-based cuboid detection and estimation method according to claim 3, characterized in that: The method of fitting the approximate broken line of each contour by judging the threshold is specifically as follows: judging whether the point with the largest distance from the head-to-tail line in the current contour is greater than the set threshold; if it is greater than the threshold, the curve is divided into two segments based on the maximum point and processed separately; otherwise, the curve is approximated as a line segment until all contours are approximated by broken line segments.

5. The binocular vision-based cuboid detection and estimation method according to claim 3, characterized in that: The step 2-2 of polygon deduplication specifically includes: determining the distance between all vertices of two polygons, and if the number of adjacent points of the polygons exceeds 3, it is determined that the two polygons are duplicates, and the polygon with the larger area is retained to achieve deduplication.

6. The binocular vision-based cuboid detection and estimation method according to claim 3, characterized in that: The step 2-2 of filtering out isolated polygons specifically includes: if two polygons belong to the same target and have more than two adjacent vertices, their vertex information is combined; if the number of adjacent vertices of the isolated polygon and other polygons is less than two, their vertex information is not combined.

7. The binocular vision-based cuboid detection and estimation method according to claim 1, characterized in that: The step 3 further comprises: When the short side interference appears at the vertices of the cuboid target in the two-dimensional image, the number of vertices N of the polygon D If it is greater than 4, add N D The weights of the 4 shortest edges are set to 0, and they are considered invalid vertices when passing through the segment.

8. The binocular vision-based cuboid detection and estimation method according to claim 1, characterized in that: The step 3 further comprises: When there are only two rectangular planes in the left and right eye views detected by the binocular camera, but only one pair of planes corresponds, and the other pair of planes are perpendicular to each other in space, the slopes of the plane center points cannot be matched; at this time, the judgment rule is first modified for a group of polygons in the left and right views. The rule is changed to: From vertex C1, rotate clockwise along the vertex of polygon D1 to obtain the upper vertex C up , rotate clockwise from vertex C2 along the vertices of polygon D2 to get the next vertex C down , the rules of the other perspective do not change, and judge whether the slopes of the current corresponding body diagonal vertices are consistent. If they are inconsistent, modify the judgment rules under the other view and find the corresponding body diagonal vertices; at this time, for the vertices C1 and C2 under the unmodified rule view, cross a vertex clockwise along the common surface to achieve the correspondence between vertices C1 and C2, and find the next vertex coordinates of C1 and C2 as the new C1 and C2.

9. The binocular vision-based cuboid detection and estimation method according to claim 1, characterized in that: The step 4 specifically includes: Construct the camera coordinate system O with the optical center of the camera as the origin of the coordinate system c -X c Y c Z c , X c Axis and Y c The axes are parallel to the horizontal and vertical axes of the image pixel coordinate system respectively; Z c The axis is the camera optical axis, which is perpendicular to the image plane; The target coordinate system O is constructed with the geometric center of the cuboid target as the coordinate origin. axis -X axis Y axis Z axis , set X axis is the normal vector of the target front plane; Y axis is the normal vector of the right plane; Z axis is the normal vector of the upper plane; X axis 、Y axis and Z axis Determined by the cross product between points: in, is the vector expression between two points; The structural parameter information of the cuboid target, length, width and height [a, b, H], is represented by the Euclidean distance between the above vertices: Target coordinate system O axis -X axis Y axis Z axis and the camera coordinate system O c -X c Y c Z c Pose parameters converted between: pitch angle, yaw angle and roll angle The expression is as follows: Among them, r 11 、r 21 、r 31 、r 32 and r 33 To use the calculated X axis ,Y axis ,Z axis The elements of the rotation matrix R of the rectangular target; X axis ,Y axis ,Z axis The three vectors are all 1×3 vectors, and the rotation matrix R is 3×3. The expression is as follows: Among them, r 11 For X axis The first element X axis (1), r 12 For X axis The second element X axis (2), r 13 For X axis The third element X axis (3), r 21 Y axis The first element Y axis (1), r 22 Y axis The second element Y axis (2), r 23 Y axis The third element Y axis (3), r 31 Z axis The first element Z axis (1), r 32 Z axis The second element Z axis (2), r 33 Z axis The second element Z axis (3).

Images