A Spatial Visual Localization Method Based on the Design of a Bicolor Rectangular Target

Abstract

The present invention discloses a spatial vision positioning method based on the design of a two-color rectangular target. The method includes: designing a two-color rectangular target pattern; performing recognition processing on the two-color rectangular target pattern by the contour corner point ellipse extraction method to obtain the target rectangular corner points in the two-color rectangular target pattern; based on the target rectangular corner points in the two-color rectangular target pattern, performing global numbering positioning processing by the sorting corner point numbering method to obtain the spatial positioning of the two-color rectangular target. The present invention can meet the requirements of spatial positioning and long-distance positioning with the help of the target in a dark environment through a simple target pattern composed of two-color target rectangles. As a spatial vision positioning method based on the design of a two-color rectangular target, the present invention can be widely applied to the field of machine vision technology.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and particularly to a spatial vision positioning method based on the design of a two-color rectangular target. Background Art

[0002] In engineering automation control, support for spatial positioning of controlled objects is often required. However, in reality, GPS signals are not always receivable in all scenarios. Sometimes, GPS signals may be affected by buildings, working environments, etc. and cannot reach. In this case, other positioning technologies are needed to help us achieve spatial positioning. Inertial positioning is a common positioning method, but it usually has the problem of excessive drift. Therefore, vision-assisted positioning is still widely used. Common vision-assisted positioning methods include SLAM technology, target recognition and positioning, etc. However, SLAM not only has complex mathematical theories, high computational complexity, but is also easily affected by the environmental background and moving objects. Target positioning is still widely used under some specific requirements because it is robust enough and easy to understand;

[0003] At present, there are many targets available for positioning, such as QR codes, DataMatrix codes, Aztec codes, MaxiCode codes, DotCode codes, Azalea codes, and AprilTag codes, etc. They are all two-dimensional codes composed of simple geometric patterns. They all use coding methods to encode information into black and white patterns, and all have a certain degree of fault tolerance. A certain number of errors can be repaired through error correction codes or error correction algorithms, thus ensuring the reliability of data. However, due to the fact that these targets are usually encoded with some information, information loss caused by low target pixel resolution, pixel noise, etc. occurs during long-distance shooting, resulting in failed recognition and positioning; in addition, these targets require external light illumination for recognition. In a completely dark scene, the white block area of the target needs to emit light. Since these targets contain a lot of information, it means a large number of blocks, and installing an LED lighting module is time-consuming and laborious. Summary of the Invention

[0004] In order to solve the above technical problems, the purpose of the present invention is to provide a spatial vision positioning method based on the design of a two-color rectangular target, which can meet the requirements of spatial positioning and long-distance positioning with the help of the target in a dark environment through a simple target pattern composed of two colors of rectangles.

[0005] The first technical solution adopted by the present invention is: a spatial vision positioning method based on the design of a two-color rectangular target, including the following steps:

[0006] Design a two-color rectangular target pattern;

[0007] The bicolor rectangular target pattern is recognized and processed by the contour corner point ellipse extraction method to obtain the target rectangular corner points in the bicolor rectangular target pattern;

[0008] Based on the target rectangular corner points in the bicolor rectangular target pattern, global numbering and positioning processing is carried out by the sorting corner point numbering method to obtain the spatial positioning of the bicolor rectangular target.

[0009] Furthermore, the designed bicolor rectangular target pattern includes a first-color target rectangle and a second-color target rectangle. The first-color target rectangle and the second-color target rectangle are rectangles with the same shape but different colors, and the interval between the first-color target rectangle and the second-color target rectangle is d.

[0010] Furthermore, the step of recognizing and processing the bicolor rectangular target pattern by the contour corner point ellipse extraction method to obtain the target rectangular corner points in the bicolor rectangular target pattern specifically includes:

[0011] The image where the bicolor rectangular target pattern is located is subjected to color segmentation processing by the color threshold segmentation method, and the contour of the segmented image is recognized by using the opencv open-source algorithm to obtain a preliminary rectangular contour;

[0012] Based on the preliminary rectangular contour, a preset filtering rule is constructed to filter the preliminary rectangular contour to obtain a candidate rectangular contour;

[0013] The candidate rectangular contour is subjected to rotated rectangle fitting. If the ratio of the short side to the long side of the fitted rotated rectangle is greater than a preset threshold, and the ratio of the contour area to the rotated rectangle area is greater than a preset area threshold, then this contour is retained;

[0014] Pairwise matching calculations are performed on all the retained rectangular contours to obtain scores, and a pair of rectangular contours with a matching score greater than a preset threshold and the highest matching score is selected as the final target rectangular contour;

[0015] The calculation of the matching score includes considering the area ratio, width ratio, height ratio, difference in the rotation angle of the fitted rotated rectangle, and the value of the interval length between the two rectangular contours divided by the width of the rectangle;

[0016] The corner points of the final target rectangular contour are recognized by the contour corner point ellipse extraction method to obtain the target rectangular corner points in the bicolor rectangular target pattern.

[0017] Furthermore, the preset filtering rule is:

[0018] The nesting levels of all rectangular contours are judged, and the rectangular contours with a nesting level of 1 are selected as candidate rectangular contours.

[0019] Further, the step of identifying and processing the final target rectangle contour by the contour corner point ellipse extraction method to obtain the target rectangle corner points in the two-color rectangle target pattern specifically includes:

[0020] Based on the properties of the ellipse focus triangle, the storage of the target rectangle contour data format implemented by the opencv open-source algorithm;

[0021] The contour point farthest from the 0th contour point in the contour is defined as the first corner point;

[0022] The contour point farthest from the first corner point in the contour is defined as the second corner point;

[0023] The point farthest from the sum of the distances from the first corner point and the second corner point in the contour is defined as the third corner point;

[0024] Sort the indices of the first corner point, the second corner point, and the third corner point in the contour from largest to smallest;

[0025] Assume that the index of the first corner point in the contour > the index of the second corner point in the contour > the index of the third corner point in the contour. Then, find the contour point farthest from the third corner point and the second corner point among the contour points from the third corner point index to the second corner point index as the first candidate fourth corner point, find the contour point farthest from the second corner point and the first corner point among the contour points from the second corner point index to the first corner point index as the second candidate fourth corner point, and find the contour point farthest from the first corner point and the third corner point among the contour points from the first corner point index to the third corner point index as the third candidate fourth corner point;

[0026] Define the distance from the first candidate fourth corner point to the straight line connecting the third corner point and the second corner point as d1, the distance from the second candidate fourth corner point to the straight line connecting the second corner point and the first corner point as d2, and the distance from the third candidate fourth corner point to the straight line connecting the first corner point and the third corner point as d3;

[0027] Judge the values of d1, d2, and d3. If the value of d1 is the largest, the first candidate fourth corner point is the required fourth corner point. If the value of d2 is the largest, the second candidate fourth corner point is the required fourth corner point. If the value of d3 is the largest, the third candidate fourth corner point is the required fourth corner point;

[0028] Integrate the first corner point, the second corner point, the third corner point, and the fourth corner point to obtain the target rectangle corner points in the two-color rectangle target pattern.

[0029] Further, the step of performing global numbering and positioning processing by the sorting corner point numbering method based on the target rectangle corner points in the two-color rectangle target pattern to obtain the spatial positioning of the two-color rectangle target specifically includes:

[0030] Based on the target rectangle corner points in the two-color rectangle target pattern, number them according to the sorted corner point numbering method to obtain the global numbers of all target rectangle corner points in the two-color rectangle target pattern;

[0031] Construct a two-color rectangle target coordinate system, and calculate the coordinates of all target rectangle corner points in the two-color rectangle target pattern in the two-color rectangle target coordinate system according to the target rectangle shape of the designed two-color rectangle target pattern;

[0032] According to the global numbers of all target rectangle corner points in the two-color rectangle target pattern, obtain the correspondence between the coordinates of all target rectangle corner points in the two-color rectangle target coordinate system and the pixel coordinates in the image coordinate system;

[0033] According to the correspondence between the coordinates of all target rectangle corner points in the two-color rectangle target pattern in the two-color rectangle target coordinate system and the pixel coordinates in the image coordinate system, use the PnP algorithm to minimize the reprojection error to obtain the rotation matrix and translation vector of the camera relative to the two-color rectangle target coordinate system;

[0034] Invert the rotation matrix to obtain the pose of the two-color rectangle target in the camera coordinate system, and the translation vector is the coordinate of the center point of the two-color rectangle target in the camera coordinate system;

[0035] According to the pose of the two-color rectangle target in the camera coordinate system and the coordinate of the center point of the two-color rectangle target in the camera coordinate system, obtain the spatial positioning of the two-color rectangle target.

[0036] Furthermore, the step of numbering all target rectangle corner points in the two-color rectangle target pattern globally according to the sorted corner point numbering method based on the target rectangle corner points in the two-color rectangle target pattern specifically includes:

[0037] Pre-number the target rectangle corner points in the two-color rectangle target pattern according to the magnitude of the coordinate axes components. The result of the pre-numbering is to define the upper left corner of the target rectangle as corner point 1, the lower left corner as corner point 2, the upper right corner as corner point 3, and the lower right corner as corner point 4;

[0038] Two-color target rectangle corner point numbering correspondence;

[0039] Induce the two-color target rectangle corner point numbering correspondence according to the corner point numbers of the target rectangles in the two-color rectangle target pattern at each rotation angle in the image after pre-numbering. The correspondence has four cases:

[0040] Corner point 1 and corner point 2 of the first-color target rectangle in the two-color rectangle target are juxtaposed and corresponding to corner point 3 and corner point 4 of the second-color target rectangle at the interval of the two-color target rectangle;

[0041] In the two-color rectangular target, the rectangular corner points 3 and 4 of the first-color target are juxtaposed and corresponding to the rectangular corner points 1 and 2 of the second-color target at the interval of the two-color target rectangle;

[0042] In the two-color rectangular target, the rectangular corner points 2 and 4 of the first-color target are juxtaposed and corresponding to the rectangular corner points 1 and 3 of the second-color target at the interval of the two-color target rectangle;

[0043] In the two-color rectangular target, the rectangular corner points 1 and 3 of the first-color target are juxtaposed and corresponding to the rectangular corner points 2 and 4 of the second-color target at the interval of the two-color target rectangle;

[0044] According to the coordinate components of the center points of the two target rectangles of the two-color rectangular target, the rotation posture of the two-color rectangular target in the image is distinguished, and the global numbering is carried out in combination with the corresponding relationship of the corner point numbers of the two-color target rectangle, so as to obtain the global numbers of all the target rectangle corner points in the two-color rectangular target pattern.

[0045] Furthermore, the expression of the PNP algorithm is:

[0046] e = sρ - K(RP w + t)

[0047] In the above formula, e represents the reprojection error, s represents the constant coefficient, ρ represents the pixel coordinates of the target rectangle corner point in the image coordinate system, K represents the internal parameter matrix of the pan-tilt camera, R represents the rotation matrix of the camera relative to the two-color rectangular target coordinate system, P w represents the coordinates of the target rectangle corner point in the two-color rectangular target coordinate system, and t represents the translation vector.

[0048] The beneficial effect of the method of the present invention is that: the present invention designs a simple two-color rectangular target pattern, which is composed of two-color rectangles. This target pattern only contains one kind of information. Further, the contour corner point ellipse extraction method is proposed to identify and process the two-color rectangular target pattern, and the target rectangle corner points in the two-color rectangular target pattern are obtained. This method greatly improves the accuracy of extracting the four corner points of the target rectangle from the target rectangle contour, thus laying a foundation for the high precision of the subsequent target positioning. Then, through the sorting corner point numbering method for global numbering and positioning processing, the spatial positioning of the two-color rectangular target is obtained. It can realize the global numbering of the target rectangle corner points with less time complexity, improve the real-time performance of target positioning, and can achieve the purpose of remote recognition, positioning and simple construction of the lighting module in the dark scene to form a two-color rectangular target pattern. The present invention designs a new pattern style and recognition and positioning solution for the target. Description of the Drawings

[0049] Figure 1 is the step flow chart of a spatial vision positioning method based on the design of a two-color rectangular target of the present invention;

[0050] Figure 2 It is a schematic diagram of the dual - color rectangular target pattern designed by the present invention;

[0051] Figure 3 It is a schematic diagram of the result of color segmentation of the image where the dual - color rectangular target pattern is located by the present invention. Among them, A represents the result of color segmentation of the first - color target rectangular, and B represents the result of color segmentation of the second - color target rectangular;

[0052] Figure 4 It is a schematic diagram of the process of screening the rectangular contour to obtain the dual - color target rectangular contour by the present invention;

[0053] Figure 5 It is a schematic diagram of the result of the present invention for identifying the target rectangular contour and its matching score;

[0054] Figure 6 It is a schematic diagram for explaining that the fitting of the rotated rectangle (box) does not fit well with the real position of the target rectangle by the present invention;

[0055] Figure 7 It is a schematic diagram of the present invention for directly locating four corner points from the two target rectangular contours respectively;

[0056] Figure 8 It is a schematic diagram of the result that the corner points can be located in any quadrilateral by the ellipse extraction method based on the contour corner points in the present invention;

[0057] Figure 9 It is a schematic diagram of the global numbering of the corner points of the target rectangle at various rotation angles in the present invention;

[0058] Figure 10 It is a schematic diagram of the result of pre - numbering the corner points of the target rectangle according to the magnitude of the coordinate axis components in the present invention;

[0059] Figure 11 It is a schematic diagram of the result of the corner point numbering of the target rectangle at various rotation angles in the image after pre - numbering in the present invention;

[0060] Figure 12 It is a schematic diagram for explaining the corresponding relationship of the corner point numbering of the dual - color target rectangle summarized according to the corner point numbering of the target rectangle in the dual - color rectangular target pattern at various rotation angles in the image after pre - numbering in the present invention;

[0061] Figure 13 It is a schematic diagram for explaining the method of determining the rotation posture of the dual - color rectangular target in the image in the present invention;

[0062] Figure 14 It is a schematic diagram of the possible pre - numbering results of the corner points of two dual - color rectangular targets obtained in the specific implementation of the present invention;

[0063] Figure 15It is a schematic diagram of obtaining the global numbering method of the target rectangle in the specific implementation of the present invention;

[0064] Figure 16 It is a schematic diagram of the result of establishing a two-color rectangular target coordinate system with the center point of the two-color rectangular target as the origin in the present invention. Specific implementation mode

[0065] The following further elaborates the present invention in detail in conjunction with the attached drawings and specific embodiments. For the step numbers in the following embodiments, they are only set for the convenience of explanation and illustration, and no limitation is imposed on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0066] Refer to Figure 1 , the present invention provides a spatial vision positioning method based on the design of a two-color rectangular target, and this method includes the following steps:

[0067] S1. Design a two-color rectangular target pattern;

[0068] Specifically, refer to Figure 2 , the target designed by the present invention is composed of two-color rectangles, named the two-color rectangular target. The two target rectangles have the same shape and different colors (such as red and blue). The height and width of the target rectangle are both h and w, and the interval between the two target rectangles is d; the values of h, w, and d can be of any size.

[0069] S2. Perform recognition processing on the two-color rectangular target pattern through the contour corner point ellipse extraction method to obtain the target rectangle corner points in the two-color rectangular target pattern;

[0070] S21. Segmentation of the two-color target rectangle area;

[0071] Specifically, refer to Figure 3 , since the two target rectangles have their own colors, the present invention uses a color segmentation method to identify the target rectangle area. There are many ways to segment colors, such as HSV color segmentation, threshold segmentation, etc. In practical applications, in order to avoid color mis-segmentation caused by factors such as environmental light as much as possible, the target rectangle usually uses colors with obvious RGB channel differences for filling, such as red and blue. Taking the filling of the target rectangle with red and blue as an example, the method for segmenting the red target rectangle area is to subtract the corresponding pixels of the blue channel from the pixels of the red channel. If the pixel difference is greater than the preset threshold, it is considered that the pixel is red. Similarly, the method for segmenting the blue target rectangle area is to subtract the corresponding pixels of the red channel from the pixels of the blue channel. If the pixel difference is greater than the preset threshold, it is considered that the pixel is blue

[0072] S22. Screen out the target rectangle area according to the segmentation map;

[0073] Specifically, referring to Figure 4 , it can be summarized as finding all the contours within the area from the segmentation map and filtering out the impossible contours according to the rules of the number of contour nesting levels, the ratio of the contour area to the area of the fitted rotated rectangle, and the ratio of the short side to the long side of the fitted rotated rectangle. Then, pairwise matching is performed among the remaining contours, and then the matching scores are given according to the similarity of the shapes and the distance relationship between the matching contours. Finally, the matching score is greater than the set threshold and the highest one is the target rectangle contour. As Figure 5 shown, the matching score of the left target rectangle frame and the right target rectangle frame is 3.04972, and the highest score can be set to 5 points. After testing by the present invention, the matching scores of non-target areas usually do not exceed 2.8 points.

[0074] S23. Extract the four corner points of the target rectangle from the target rectangle contour based on the contour corner point ellipse extraction method.

[0075] Specifically, only having the contour cannot locate the spatial position of the target rectangle. Therefore, we need to find the four corner points of the target rectangle from the contour. The simplest method is to fit the minimum area rotated rectangle to the contour, and then use the four corner points of the rotated rectangle as the corner points of the contour. However, when the fit of the extracted contour to the target rectangle is not perfect or the target rectangle is imaged as a non-standard rectangle in the current camera view, there will be a large deviation between the corner points of the fitted rotated rectangle and the true corner points of the target rectangle. As Figure 6 shown, in order to obtain the accurate corner point positions from the contour, the present invention invented an algorithm for calculating the positions of the four corner points in the contour from the contour storage format extracted by opencv according to the properties of the elliptical focal triangle, named "contour corner point ellipse extraction method". The algorithm flow is as follows:

[0076] (1) The contour point farthest from the 0th contour point in the contour is the first corner point, i.e., the first corner point;

[0077] (2) The contour point farthest from the first corner point in the contour is the second corner point, i.e., the second corner point;

[0078] (3) The point farthest from the sum of the distances from the first corner point and the second corner point in the contour is the third corner point, i.e., the third corner point;

[0079] (4) Sort the indices of the first corner point, the second corner point, and the third corner point in the contour from largest to smallest. Here, assume the sorted result is: the index of the first corner point in the contour is greater than the index of the second corner point in the contour is greater than the index of the third corner point in the contour;

[0080] (5) Find the contour point farthest from the third corner point and the second corner point among the contour points from the index of the third corner point to the index of the second corner point as the first candidate for the fourth corner point;

[0081] (6) Find the contour point farthest from corner point two and corner point one among the contour points from the corner point two index to the corner point one index as the second candidate corner point four;

[0082] (7) Find the contour point farthest from corner point one and corner point three among the contour points from the corner point one index to the corner point three index as the third candidate corner point four;

[0083] (8) The distance from the first candidate corner point four to the straight line formed by corner point three and corner point two is d1, the distance from the second candidate corner point four to the straight line formed by corner point two and corner point one is d2, and the distance from the third candidate corner point four to the straight line formed by corner point one and corner point three is d3;

[0084] (9) If d1 is the largest, the first candidate corner point four is the required corner point four; if d2 is the largest, the second candidate corner point four is the required corner point four; if d3 is the largest, the third candidate corner point four is the required corner point four, that is, the fourth corner point;

[0085] If the four corner points of the rotated rectangle are directly taken as the corner points of the target rectangle, there will be a large deviation from the true corner point positions of the target rectangle. However, using the contour corner point ellipse extraction method of the present invention, the extracted corner points fit well with the true corner point positions of the target rectangle, as Figure 7 shown by the dot markers, which is a schematic diagram of the 4 corner points respectively extracted from the two target rectangle contours by the contour corner point ellipse extraction method of the present invention;

[0086] Further, referring to Figure 8 , the "contour corner point ellipse extraction method" of the present invention is not only applicable to standard rectangles, but can directly locate the corner points from the contour for any quadrilateral or irregular quadrilateral with serrations.

[0087] S3. Based on the corner points of the target rectangle in the two-color rectangle target pattern, perform global numbering and positioning processing through the sorting corner point numbering method to obtain the spatial positioning of the two-color rectangle target.

[0088] S31. Number the corner points of the target rectangle;

[0089] Specifically, referring to Figure 9 , to obtain the spatial position of the two-color rectangle target in the camera coordinate system, that is, it is necessary to know the correspondence between the coordinates of each corner point of the target rectangle in the image and the corner point coordinates of the target rectangle in the two-color rectangle target coordinate system. For this reason, the present invention needs to number each corner point. In order to correctly number each corner point, compared with the method of brute-force judgment by full permutation, the present invention has invented a corner point numbering algorithm with less computational complexity, called the "sorting corner point numbering method".

[0090] S311. Pre-number according to the magnitude of the coordinate axis components;

[0091] Specifically, referring toFigure 10 , the four-point coordinates are first sorted from small to large by the x-axis component, and then stored in array indexes 0, 1, 2, and 3 in turn. The coordinates of indexes 0 and 1 are sorted from small to large by the y-axis and then stored in array indexes 0 and 1 in turn; the same operation is applied to indexes 2 and 3. The sorted result makes the upper left corner of the target rectangle corner point 1, the lower left corner corner point 2, the upper right corner corner point 3, and the lower right corner corner point 4. After the above pre-numbering, when the two-color rectangular target is rotated, all possible cases of pre-numbering of the target rectangle corner points at each rotation angle in the image are Figure 11 shown.

[0092] S312, determining the corresponding relationship between the corner point numbers of the two-color target rectangle;

[0093] Specifically, it is necessary to further determine the corresponding relationship between the corner numbering of the two-color target rectangle and analyze Figure 11 , we can summarize the corresponding relationships of the corner point numbers of four two-color target rectangles, such as Figure 12 As shown, the first color refers to Figure 11 The middle gray grid texture rectangle color alias, that is, the first color target rectangle, the second color refers to Figure 11 The color alias for the pure white texture rectangle, that is, the second color target rectangle, and rectangle i refers to the corner point of the rectangle numbered i.

[0094] S313, determining the global numbering of the corner points of the two-color target rectangle according to the rotation posture of the two-color target rectangle in the image;

[0095] According to step S311, the present invention pre-numbers the corner points, and then obtains the corresponding relationship of the corner point numbers of the two-color target rectangle according to step S312 on the basis of the pre-numbering in step S311. Then, the present invention determines the global numbering relationship of the corner points of the target rectangle according to the rotation posture of the current two-color rectangular target in the image, and finally converts the obtained corresponding relationship of the corner point numbers of the two-color target rectangle into a global number according to step S312;

[0096] First color finger Figure 13 Medium gray grid texture rectangular color alias, secondary color refers to Figure 13 The pure white texture rectangle color alias, such as Figure 13 As shown, when the x-axis component of the center point of the second color rectangle is less than the x-axis component of the center point of the first color rectangle, the rotation posture of the two-color rectangular target is as shown in the left frame, and there are three possibilities. In these three possibilities, the second color rectangle is on the left and the first color rectangle is on the right. The same is true when the x-axis component of the center point of the second color rectangle is equal to or greater than the x-axis component of the center point of the first color rectangle. So far, the present invention has completed the distinction of the rotation posture of the two-color rectangular target and uses Figure 12 The global numbering of the corner points of the two-color target rectangle can be obtained by summarizing the corresponding relationship of the corner point numbers of the target rectangle in the pre-numbered two-color rectangular target pattern at each rotation angle in the image;

[0097] Assume that the current rotation posture of the two-color rectangular target in the image is such that the x-axis component of the center point of the second-color rectangle < the x-axis component of the center point of the first-color rectangle, and according to Figure 12 the rules of the present invention, the corresponding relationship between the rectangular corner numbers of the two-color target is shown as follows:

[0098] (1) Corner point 3 of the second-color rectangle corresponds to corner point 1 of the first-color rectangle, and corner point 4 of the second-color rectangle corresponds to corner point 2 of the first-color rectangle;

[0099] (2) Corner point 3 of the second-color rectangle corresponds to corner point 1 of the second-color rectangle, and corner point 4 of the second-color rectangle corresponds to corner point 2 of the second-color rectangle;

[0100] (3) Corner point 1 of the first-color rectangle corresponds to corner point 3 of the first-color rectangle, and corner point 2 of the first-color rectangle corresponds to corner point 4 of the first-color rectangle;

[0101] First, according to Figure 12 the corresponding relationship between the rectangular corner numbers of the two-color target, the possible sorting of the four corner points can be obtained as Figure 14 two cases. Also, because the rotation posture of the two-color rectangular target is such that the x-axis component of the center point of the second-color rectangle < the x-axis component of the center point of the first-color rectangle, from Figure 13 it can be known that the global numbering method should be Figure 15 as shown. Finally, the present invention only needs to judge the magnitude relationship between the y-axis component of the current corner point 3 of the second-color rectangle and the y-axis component of the corner point 4 of the second-color rectangle to obtain the global numbering. Then, let's assume that the y-axis component of the current corner point 3 of the second-color rectangle is greater than the y-axis component of the corner point 4 of the second-color rectangle, that is Figure 14 the corresponding relationship between the rectangular corner numbers of the target in the right figure style of

[0102] Then the corresponding relationship between the rectangular corner numbers of the target is converted into the global numbering method as follows:

[0103] T[3] = corner point 4 of the second-color rectangle, T[4] = corner point 3 of the second-color rectangle, T[5] = corner point 2 of the first-color rectangle, T[6] = corner point 1 of the first-color rectangle, T[1] = corner point 2 of the second-color rectangle, T[2] = corner point 1 of the second-color rectangle, T[7] = corner point 4 of the first-color rectangle, T[8] = corner point 3 of the first-color rectangle;

[0104] where the content of the T array is the pixel coordinates of the corner points, and the index is the global number corresponding to the pixel coordinates of the corner points.

[0105] S32, Spatial positioning.

[0106] Specifically, as Figure 16 shown, with the center point of the target as the origin, a two-color rectangular target coordinate system is further established. Then, according to Figure 2The h, w, and d values of the designed two-color rectangular target can be used to calculate the coordinates of the eight corner points of the two-color rectangular target in the two-color rectangular target coordinate system. Also, since the global numbering of each corner point of the target rectangle has been completed, it is easy to know the correspondence between the coordinates of the eight corner points of the two-color rectangular target in the two-color rectangular target coordinate system and the pixel coordinates. Finally, by using the PnP algorithm to minimize the reprojection error, the rotation matrix and translation vector of the camera relative to the two-color rectangular target coordinate system can be estimated. Taking the inverse of the estimated rotation matrix gives the pose of the two-color rectangular target in the camera coordinate system, and the translation vector is the coordinate of the center point of the two-color rectangular target in the camera coordinate system;

[0107] The expression of the PnP algorithm is as follows:

[0108] e = sρ - K(RP w + t)

[0109] In the above formula, e represents the reprojection error, s represents a constant coefficient, ρ represents the pixel coordinates of the corner point, K represents the internal parameter matrix of the pan-tilt camera, R represents the rotation matrix of the camera relative to the target coordinate system, P w represents the coordinates of the corner point of the target rectangle in the two-color rectangular target coordinate system, and t represents the translation vector;

[0110] By minimizing e using the least squares method, the rotation matrix R and the translation vector t can be obtained. The translation vector t is the coordinate of the center point of the two-color rectangular target in the camera coordinate system, and R -1 is the pose of the two-color rectangular target coordinate system in the camera coordinate system.

[0111] The above is a specific description of the preferred embodiment of the present invention. However, the present invention is not limited to the described embodiment. Those skilled in the art can make various equivalent deformations or substitutions without departing from the spirit of the present invention, and these equivalent deformations or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A spatial vision positioning method based on the design of a two-color rectangular target, characterized in that Including the following steps: Design a two-color rectangular target pattern, including a first-color target rectangle and a second-color target rectangle. The first-color target rectangle and the second-color target rectangle are rectangles with the same shape but different colors, and the interval between the first-color target rectangle and the second-color target rectangle is d; Perform color segmentation processing on the image where the two-color rectangular target pattern is located through the color threshold segmentation method, and use the opencv open-source algorithm to perform contour recognition on the segmented image to obtain a preliminary rectangular contour; Based on the preliminary rectangular contour, construct a preset filtering rule to filter the preliminary rectangular contour to obtain a candidate rectangular contour; Perform rotated rectangle fitting on the candidate rectangular contour. If the ratio of the short side to the long side of the fitted rotated rectangle is greater than a preset threshold, and the ratio of the contour area to the rotated rectangle area is greater than a preset area threshold, then retain the contour; Perform pairwise matching calculation scores on all the retained rectangular contours, and select a pair of rectangular contours with a matching score greater than the preset threshold and the highest matching score as the final target rectangular contour; The matching score calculation includes considering the area ratio, width ratio, height ratio, difference in the rotation angle of the fitted rotated rectangle, and the value of the interval length between the two rectangular contours divided by the width of the rectangle; Perform corner point recognition processing on the final target rectangular contour through the contour corner point ellipse extraction method to obtain the target rectangular corner points in the two-color rectangular target pattern; Pre-number the target rectangular corner points in the two-color rectangular target pattern according to the size of the coordinate axis components. The result of the pre-numbering is that the upper left corner of the target rectangle is numbered as corner point 1, the lower left corner is corner point 2, the upper right corner is corner point 3, and the lower right corner is corner point 4; Induce the corresponding relationship of the two-color target rectangular corner point numbers according to the corner point numbers of the target rectangle in the two-color rectangular target pattern at each rotation angle in the image after pre-numbering; The corresponding relationship includes: corner point 1 and corner point 2 of the first-color target rectangle in the two-color rectangular target are juxtaposed and corresponding to corner point 3 and corner point 4 of the second-color target rectangle at the interval of the two-color target rectangle; corner point 3 and corner point 4 of the first-color target rectangle in the two-color rectangular target are juxtaposed and corresponding to corner point 1 and corner point 2 of the second-color target rectangle at the interval of the two-color target rectangle; corner point 2 and corner point 4 of the first-color target rectangle in the two-color rectangular target are juxtaposed and corresponding to corner point 1 and corner point 3 of the second-color target rectangle at the interval of the two-color target rectangle; corner point 1 and corner point 3 of the first-color target rectangle in the two-color rectangular target are juxtaposed and corresponding to corner point 2 and corner point 4 of the second-color target rectangle at the interval of the two-color target rectangle; Distinguish the rotation posture of the two-color rectangular target in the image according to the coordinate axis components of the center points of the two target rectangles of the two-color rectangular target, and perform global numbering in combination with the corresponding relationship of the two-color target rectangular corner point numbers to obtain the global numbers of all the target rectangular corner points in the two-color rectangular target pattern; Construct a two-color rectangular target coordinate system, and calculate the coordinates of all the target rectangular corner points in the two-color rectangular target pattern in the two-color rectangular target coordinate system according to the shape of the target rectangle of the designed two-color rectangular target pattern; According to the global numbers of all target rectangular corner points in the two-color rectangular target pattern, obtain the correspondence between the coordinates of all target rectangular corner points in the two-color rectangular target coordinate system and the pixel coordinates in the image coordinate system; According to the correspondence between the coordinates of all target rectangular corner points in the two-color rectangular target coordinate system and the pixel coordinates in the image coordinate system, use the PnP algorithm to minimize the reprojection error to obtain the rotation matrix and translation vector of the camera relative to the two-color rectangular target coordinate system; Invert the rotation matrix to obtain the pose of the two-color rectangular target in the camera coordinate system, and the translation vector is the coordinate of the center point of the two-color rectangular target in the camera coordinate system; According to the pose of the two-color rectangular target in the camera coordinate system and the coordinate of the center point of the two-color rectangular target in the camera coordinate system, obtain the spatial positioning of the two-color rectangular target.

2. The spatial vision positioning method based on the design of a two-color rectangular target according to claim 1, characterized in that, The preset filtering rule is: Judge the nesting levels of all rectangular contours, and select the rectangular contour with a nesting level of 1 as the candidate rectangular contour.

3. The spatial vision positioning method based on the design of a two-color rectangular target according to claim 2, wherein The step of performing corner point recognition processing on the final target rectangular contour through the contour corner point ellipse extraction method to obtain the target rectangular corner points in the two-color rectangular target pattern specifically includes: Based on the properties of the elliptical focal triangle, implement the storage of the data format of the target rectangular contour based on the opencv open source algorithm; The contour point farthest from the 0th contour point in the contour is defined as the first corner point; The contour point farthest from the first corner point in the contour is defined as the second corner point; The point farthest from the sum of the distances from the first corner point and the second corner point in the contour is defined as the third corner point; Sort the indices of the first corner point, the second corner point, and the third corner point in the contour from largest to smallest; Assume that the index of the first corner point in the contour > the index of the second corner point in the contour > the index of the third corner point in the contour. Then, find the contour point farthest from the third corner point and the second corner point among the contour points from the third corner point index to the second corner point index as the first candidate fourth corner point, find the contour point farthest from the second corner point and the first corner point among the contour points from the second corner point index to the first corner point index as the second candidate fourth corner point, and find the contour point farthest from the first corner point and the third corner point among the contour points from the first corner point index to the third corner point index as the third candidate fourth corner point; Define the distance from the first candidate fourth corner point to the straight line formed by the third corner point and the second corner point as d1, the distance from the second candidate fourth corner point to the straight line formed by the second corner point and the first corner point as d2, and the distance from the third candidate fourth corner point to the straight line formed by the first corner point and the third corner point as d3; Judge the values of d1, d2, and d3. If the value of d1 is the largest, the first candidate fourth corner point is the required fourth corner point. If the value of d2 is the largest, the second candidate fourth corner point is the required fourth corner point. If the value of d3 is the largest, the third candidate fourth corner point is the required fourth corner point; Integrate the first corner point, the second corner point, the third corner point, and the fourth corner point to obtain the target rectangular corner points in the two-color rectangular target pattern.

4. The spatial vision positioning method based on the design of a two-color rectangular target according to claim 3, wherein The expression of the PnP algorithm is: ； In the above formula, represents the reprojection error, represents the constant coefficient, represents the pixel coordinates of the corner points of the target rectangle in the image coordinate system, represents the internal parameter matrix of the pan-tilt camera, represents the rotation matrix of the camera relative to the coordinate system of the two-color rectangular target, represents the coordinates of the corner points of the target rectangle in the coordinate system of the two-color rectangular target, represents the translation vector.

Images