Binocular stereo matching method based on complex structured light

Through multiple projection devices, complex structured light patterns are emitted, combined with binocular camera matching feature points, the problem of difficulty in feature matching in binocular vision technology is solved, and efficient and real-time three-dimensional reconstruction is achieved, suitable for target object surfaces in complex environments Information acquisition.

CN118570504BActive Publication Date: 2024-11-05OCEAN UNIV OF CHINA
View PDF 10 Cites 0 Cited by

Patent Information

Application Number
CN202411045157.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2024-11-05
Estimated Expiration
2044-08-01

AI Technical Summary

Technical Problem

When obtaining the three-dimensional information of the target object, existing binocular vision technology is limited by insufficient surface texture, insufficient visible light or environmental factors of the target object, making it difficult to effectively match the feature points of the left and right images. The traditional structured light projection method is time-consuming and complex to calculate. And not real-time.

Method used

Multiple projection devices are used to emit linear structured light strips parallel to each other to form complex structured light patterns, collect images through binocular cameras, acquire feature points of end points and intersection points, build feature descriptors, match feature points, and realize three-dimensionality in combination with structured light paths. reconstruction.

Benefits of technology

Improve the real-time and accuracy of the device, simplify operations, and can effectively obtain the surface space information of the target object under complex conditions, reduce the computing resource requirements, strong adaptability, and avoid the information randomness and dense light of traditional parallel structured light. Difficult to match strips.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118570504B_ABST
    Figure CN118570504B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of image recognition technology, and is a binocular stereo matching method based on complex structured light. Each projection device of the structured light projection system emits a group of several mutually parallel straight line structured lights. The structured lights emitted by any two projection devices intersect to generate an intersection, and each structured light strip has two endpoints. The left camera and the right camera simultaneously capture the complex structured light image generated by the structured light projection system on the surface of the object. The left camera image and the right camera image are processed respectively to obtain the endpoint feature points and the intersection feature points. Based on the positional relationship of the feature points, the structured light path between the feature points is determined. The feature points obtained from the left and right images are used to construct a feature descriptor, and the feature points of the left and right images are matched to obtain the matching relationship of the structured light path between the feature points of the left and right images. The three-dimensional reconstruction of the complex structured light is completed, and the surface features of the object are obtained. The present invention simplifies the matching and recognition steps, and can complete the recognition and matching work conveniently and accurately.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a binocular stereo matching method based on complex structured light. Background Art

[0002] At present, in the field of binocular vision, the three-dimensional information of the target object is usually obtained by controlling the left and right cameras to expose at the same time and collect images of the target object. Match the feature points on the left and right images of the target object. Use some objective constraints to pair the feature points at the same position. However, since most target objects have fewer textures or features on the surface, or even no features or textures, or the photographed target objects are in an environment with less or no visible light, it is impossible to effectively extract and match the features on the left and right images of the target object. In order to solve the above problems, the sensor is generally used to actively provide features for the target object to make up for the deficiencies of the target object itself or the interference caused by environmental factors.

[0003] At present, there are two types of sensors that actively provide features for target objects. One is to project a pattern with coded information onto the surface of the object through a projection device, collect images with coded information through a camera, and decode the captured left and right images according to the corresponding coding method. The feature points with the same coded information on the left and right images are matching points. This projection method requires the projected pattern to be encoded in advance, and the subsequent decoding process is relatively complicated, requires greater computing power and consumes more time resources, and cannot obtain the spatial information of the target object in a timely manner.

[0004] Another projection method is to project line structured light through a projection device, including scanning line structured light and line array structured light. Scanning line structured light is driven by a control device to drive the line structured light projection device so that the line structured light scans the surface of the target object, and at the same time controls the binocular camera to continuously expose and take pictures. This scanning method is time-consuming, cannot guarantee real-time performance, and is limited in application scenarios. Line array structured light is to project line array parallel structured light onto the surface of the target object according to rules through a control device, and control the left and right cameras to expose and collect images at the same time. The structured light strips on the collected left and right images are extracted and matched. The information of the target object obtained under this method is generally the part where the structured light is projected onto the target object. If more information on the surface of the target object is to be obtained, more line structured light needs to be projected. However, too many line structured lights will result in a denser distribution of light strips. This is not conducive to the subsequent extraction and matching of light strips. Moreover, the light strips of parallel structured light are parallel to each other, and the parallel structured light has only one direction. When problems such as light strip loss or light strip obstruction occur, it will be difficult to match the light strips on the left and right images, which will affect subsequent three-dimensional reconstruction and other operations.

[0005] Therefore, a method is needed that can not only ensure the real-time and accuracy of the device, but also obtain the surface spatial information of the target object in complex situations by using relatively simple operations. Summary of the invention

[0006] The purpose of the present invention is to solve one of the aforementioned technical problems and to provide a binocular stereo matching method based on complex structured light.

[0007] To achieve the above object, the present invention adopts the following technical solution:

[0008] A binocular stereo matching method based on complex structured light is implemented based on a matching system and is used to obtain surface features of an object;

[0009] The matching system comprises:

[0010] Structured light projection system: The structured light projection system includes a plurality of projection devices, each of which can emit a group of a plurality of mutually parallel linear structured light strips, and the linear structured light strips emitted by any two projection devices intersect, wherein each linear structured light strip emitted by each projection device has two endpoints; the linear structured light strips generate intersection points at the intersection positions;

[0011] Binocular camera: includes a left camera and a right camera; used to simultaneously capture complex structured light images of the object surface to obtain left and right images;

[0012] Processor: used to obtain images collected simultaneously by the binocular cameras, perform image processing, and execute the matching method, the method comprising:

[0013] Processing the images captured by the left camera and the right camera at the same time, respectively obtaining feature points of the left image and the right image, wherein the feature points include: endpoint feature points and intersection feature points;

[0014] Based on the endpoint feature point or the intersection feature point, other feature points are searched along the extension direction of the structured light strip, and the structured light path between the feature points is determined at the same time;

[0015] All feature points extracted from the left image and the right image are used to construct feature descriptors one by one; based on the feature descriptors, the feature points on the left and right images are matched to obtain the matching relationship between the feature points of the left image and the feature points of the right image;

[0016] Based on the correspondence between the feature points of the left image and the feature points of the right image and the structured light path between the feature points, a matching relationship between the structured light path between the feature points of the left image and the structured light path between the feature points of the right image is obtained;

[0017] Extracting the center position of the light strip from the structured light path between the feature points of the left image and the structured light path between the feature points of the right image respectively;

[0018] Based on the matching relationship between the structured light path between the feature points of the left image and the structured light path between the feature points of the right image, the intrinsic and extrinsic parameters of the camera are obtained in combination with the calibration of the binocular camera, and the epipolar constraint is used to complete the matching of the center points of the structured light strips; the three-dimensional reconstruction of complex structured light is realized to obtain the surface features of the object.

[0019] In some embodiments of the present invention, multiple groups of parallel structured lights emitted by multiple projection devices are interlaced to form a complex structured light pattern.

[0020] In some embodiments of the present invention, the method for determining the intersection feature point includes:

[0021] Detecting corner points included in the intersection position of the structured light strips, the number of corner points is related to the number of intersecting light strips, and if M structured light strips intersect at one point, then the intersection position includes 2M corner points;

[0022] Find the minimum covering circle containing all corner points at the intersection position;

[0023] The center of the minimum coverage circle is taken as the intersection feature point where the light strips intersect.

[0024] In some embodiments of the present invention, the method for determining the endpoint feature point includes:

[0025] The endpoints of each structured light strip are taken as feature points.

[0026] In some embodiments of the present invention, a method for determining a structured light path between feature points includes:

[0027] The feature points are connected to each other through structured light strips.

[0028] In some embodiments of the present invention, a method for determining a structured light path between feature points includes:

[0029] S1: Select a feature point as the starting feature point, that is, the K-th level feature point.

[0030] S2: Select a structured light strip at the starting feature point as the starting path.

[0031] S3: Starting from the starting feature point, along the starting path, find the next level feature point on the path, that is, the K+1th level feature point, which is defined as the end point feature point on the structured light path;

[0032] S4: When the K+1th level feature point is found, the path stops searching; the positional relationship between the starting feature point and the ending feature point and the structured light path between the two feature points are obtained;

[0033] S5: Determine the number of structured light strips at the starting feature point, select each structured light strip at the starting feature point as a starting path, and search for the K+1th level feature point on each starting path until all the K+1th level feature points corresponding to the Kth level feature point are determined;

[0034] S6: Select the K+1th level feature points one by one as new starting feature points; repeat steps S1, S2, S3, S4, S5;

[0035] S7: According to the method of step S6, the next level of feature points are gradually searched until all feature points on the image are traversed, and the structured light paths between all feature points and the positional relationships between all feature points are obtained.

[0036] In some embodiments of the present invention, in step S1, a feature point at an endpoint position is selected as a starting feature point.

[0037] Compared with the prior art, the technical advantages of the present invention are:

[0038] 1. By using a structured light projection system including several projection devices, each projection device can project a group of several parallel structured lights, thereby improving the efficiency of the projection device and effectively reducing costs; it can complete complex structured light projection tasks in different scenarios and has strong adaptability.

[0039] 2. Complex structured light patterns are formed by the random arrangement of several projection devices, and the projected linear structured lights are arranged in an interlaced manner, thereby forming a more complex structured light pattern. The structured light that forms the complex pattern is arranged irregularly, so that when the complex pattern is projected onto the surface of the target object, the spatial information of the target object surface can be obtained more reasonably. This avoids the problem that the parallel structured light only has information in one direction, which leads to the randomness of the surface information of the object.

[0040] 3. The linear structured light array only needs to perform one projection task on the target object, avoiding the problem that dense parallel structured light needs to set one projection task as multiple projections, simplifying the experimental operation steps; at the same time, it ensures that the device has high real-time performance.

[0041] 4. Combine the matching relationship between the feature points of the left and right images with the structured light strips between the feature points. By dividing the complex multi-line structured light pattern into several simple structured light segments for matching, the structured light strips on the left and right images can be matched more conveniently and accurately.

[0042] 5. When there are problems such as occlusion or light bar loss on the left and right images, the traditional matching algorithm cannot accurately match the left and right light bars; even if it can match the left and right light bars, the matching success rate and accuracy are low. The binocular matching algorithm of the present invention can still achieve accurate matching of other complete light bars under this condition. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0044] Figure 1a A structured light pattern emitted by a single projection device;

[0045] Figure 1b A complex structured light pattern formed by multiple projection devices emitting simultaneously;

[0046] Figure 2 Schematic diagram of corner point detection at the endpoints and intersections of complex structured light patterns;

[0047] Figure 3 It is a schematic diagram of endpoint feature points;

[0048] Figure 4 It is a schematic diagram of the intersection feature point structure;

[0049] Figure 5a The left image is a complex structured light pattern projected onto the surface of an object;

[0050] Figure 5b The right image is a complex structured light pattern projected onto the surface of an object;

[0051] Figure 6 The schematic diagram of searching from feature point A to feature point B along structured light path 1 for the left image;

[0052] Figure 7 The schematic diagram of searching from feature point B to feature point C along structured light path 2 for the left image;

[0053] Figure 8 The schematic diagram of searching the feature point D from the feature point B along the structured light path 3 in the left image;

[0054] Fig. 9 The schematic diagram of searching the feature point E from the feature point B along the structured light path 4 in the left image;

[0055] Fig.10 It is the matching relationship diagram between the feature points of the left image and the feature points of the right image;

[0056] Fig.11 It is the matching relationship diagram of the structured light path between the feature points of the left and right images;

[0057] Fig.12 It is the light intensity distribution diagram of the cross section of the structured light strip;

[0058] Fig.13a This is the effect diagram of extracting the center of the structured light strip in the left image;

[0059] Fig.13b This is the effect diagram of extracting the center of the structured light strip in the right image;

[0060] Fig.14 This is the schematic diagram of the epipolar constraint condition;

[0061] Fig.15a It is a schematic diagram of the left image light strip center position matching based on the epipolar line constraint condition;

[0062] Fig.15b It is a schematic diagram of the center position matching of the light strip of the right image based on the epipolar line constraint condition;

[0063] Fig.16 This is the flow chart of the complex structured light binocular stereo matching algorithm. DETAILED DESCRIPTION

[0064] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0065] The present invention provides a binocular stereo matching method based on complex structured light, which is used to obtain the surface features of an object, that is, to obtain the three-dimensional information of the surface of the object. The matching method performs matching steps based on the algorithm of the processor. The method specifically includes: a structured light projection system, a binocular camera and a matching process.

[0066] The structured light projection system includes multiple projection devices, which can use multiple lasers. Each projection device emits a group of several parallel straight line structured lights. The structured lights emitted by any two projection devices intersect, wherein each structured light strip emitted by each projection device has two endpoints; the structured light strips generate intersections at the intersection positions.

[0067] Complex multi-line structured light pattern, through N multi-line structured light projection devices, each multi-line structured light projection device can project a group of n parallel straight line structured light strips. The structured light strips projected by any two projection devices intersect. N groups of parallel multi-line structured lights intersect to form a complex multi-line structured light pattern. Figure 1a The figure shows a schematic diagram of light projection by a single multi-line structured light projection device. Figure 1b A complex structured light pattern formed by multiple projection devices emitting simultaneously.

[0068] refer to Figure 1b and Figure 2 , which is a schematic diagram of the structured light projection system projecting structured light. Figure 1b and Figure 2 In the illustrated embodiment, the structured light projection system is capable of emitting three groups of parallel lights, each group including a number of parallel structured light strips, and the adjacent parallel structured light strips in each group are equally spaced. According to the set angles and positions between the parallel structured lights, the light strips in this group of parallel structured lights are parallel to each other and do not intersect, and the parallel structured light strips in this group intersect with the parallel structured light strips in other groups. Any number of groups of structured light strips can intersect. The number of groups of structured light strips, the position angles, the spacing between parallel structured lights in the same group, etc. can all be set as needed. In the embodiment illustrated in the present invention, the parallel structured light strips intersect with each other, and there is no situation where three light strips intersect.

[0069] For the corner point detection results at the endpoints and intersections of the line structured light of the complex structured light pattern, the detection results of the corner points are enlarged and displayed to highlight the display effect, such as Figure 2 , Figure 3 and Figure 4 shown.

[0070] The structured light projection system projects multiple groups of structured light strips onto the surface of an object, and the structured light patterns that present the surface characteristics of the object will be captured by the binocular camera at the same time.

[0071] The binocular camera includes a left camera and a right camera with fixed relative positions; the left camera and the right camera simultaneously collect the complex structured light image generated by the structured light projection system irradiating the surface of an object, and obtain the left image and the right image respectively, such as Figure 5a , Figure 5b As shown in the figure, due to the different viewing angles of the left camera and the right camera, the images of the structured light on the surface of the object collected by the two cameras will be different.

[0072] The binocular matching algorithm is implemented based on a processor. The processor is equipped with a processing program to execute the binocular matching algorithm and process the left image and the right image respectively.

[0073] The processor acquires the image from the binocular camera and performs image processing. The processing steps include:

[0074] Process the left image captured by the left camera and the right image captured by the right camera respectively to obtain image feature points, which include: endpoint feature points and intersection feature points;

[0075] Based on the endpoint feature points and the intersection feature points, other feature points are found along the direction of the structured light strip path to determine the structured light path between the feature points. The structured light path mentioned here refers to the part of the structured light strip between the feature points, that is, the structured light strip is divided according to the feature points.

[0076] All feature points extracted from the left and right images are used to construct feature descriptors one by one. Based on the feature descriptors, the feature points on the left and right images are matched to obtain the matching relationship between the feature points of the left image and the feature points of the right image;

[0077] Based on the correspondence between the feature points of the left image and the feature points of the right image and the structured light path between the feature points, a matching relationship between the structured light path between the feature points of the left image and the structured light path between the feature points of the right camera image is obtained;

[0078] Matching the structured light path between the feature points of the left camera image and the structured light path between the feature points of the right camera image; the specific matching processing method is described in detail in the following implementation manner;

[0079] The surface features of the object are obtained based on the matching results.

[0080] In some embodiments of the present invention, the characteristic points at the intersection positions of the structured light strips are determined based on the following method:

[0081] Detecting corner points included in the intersection of structured light strips, where the corner points are determined according to the number of intersecting light strips. If M structured light strips intersect at one point, then the intersection position contains 2M corner points;

[0082] Find the smallest circle that contains all corner points at the intersection, that is, the smallest covering circle;

[0083] The center of the minimum covering circle is taken as the intersection feature point.

[0084] In some embodiments of the present invention, the endpoint feature points are determined based on the following method:

[0085] The endpoints at both ends of each structured light strip are taken as feature points, namely, endpoint feature points.

[0086] Specifically, feature points are divided into two categories, one is endpoint feature points, and the other is intersection feature points.

[0087] The number of corner points detected at the endpoint of the line structured light is one, and the corner point is selected as the feature point at the endpoint, such as Figure 3 As shown in the figure. M structured light strips intersect at one point, and the intersection contains 2M corner points. It is necessary to divide the corner points detected at the same intersection into the same group and find the minimum covering circle equation of the group of corner points. The principle is as follows Figure 4 shown.

[0088]

[0089] The coordinates of the center of the circle are ( , ), the radius is . Select the center coordinates of the minimum covering circle ( , ) as the feature point at that position.

[0090] In some embodiments of the present invention, the matching process is to determine the relationship between the feature points in the following manner:

[0091] The feature points are connected to each other through structured light strips.

[0092] In some embodiments of the present invention, the structured light path between feature points is determined as follows.

[0093] First, an image to be processed is selected, and the following steps S1 to S7 are performed on the left image and the right image respectively.

[0094] S1: Select a feature point as the starting feature point, that is, the K-th level feature point.

[0095] In some embodiments of the present invention, in step S1, if the feature point includes multiple structured light strips, they are selected one by one as the starting path. In order to simplify the method, in step S1, the feature point corresponding to the endpoint is selected as the starting feature point.

[0096] S2: Select a structured light strip at the starting feature point as the starting path.

[0097] After the starting feature point is determined, the starting path of the starting feature point can be determined based on the structured light strips on the image.

[0098] S3: Starting from the starting feature point, along the starting path, find the next level feature point on the path, that is, the K+1th level feature point, which is defined as the end point feature point on the structured light path;

[0099] S4: When the K+1th level feature point is found, the path stops searching; the positional relationship between the starting feature point and the ending feature point and the structured light path between the two feature points are obtained;

[0100] S5: Determine the number of structured light strips at the starting feature point, select each structured light strip at the starting feature point as a starting path, and search for the K+1th level feature point on each starting path until all the K+1th level feature points corresponding to the Kth level feature point are determined;

[0101] S6: Select the K+1th level feature points one by one as new starting feature points; repeat steps S1, S2, S3, S4, S5;

[0102] S7: According to the method of step S6, the next level of feature points are gradually searched until all feature points on the image are traversed, and the structured light paths between all feature points and the positional relationships between all feature points are obtained.

[0103] The above steps S1 to S7 are described in detail as follows.

[0104] After extracting all feature points from the image, it is necessary to obtain the relationship between all feature points. Select a feature point as the starting feature point, and use a structured light strip of the starting feature point as the starting path. Starting from the starting feature point, search for the next level feature point on the path along the starting path, that is, the end feature point. When the end feature point is found on the structured light strip path, the search for this path will stop. Select feature point A as the starting point, and stop after finding feature point B along the structured light path 1. Figure 6 shown.

[0105] When the starting point contains unsearched structured light strips, select each unsearched structured light strip as the starting path to find the next-level feature point. When the starting point does not contain an unsearched structured light strip path, use the next-level feature point found as the new starting feature point and follow the above steps. If feature point A does not contain other unsearched structured light strips, use feature point B as the new starting feature point. Use a structured light strip at the starting feature point B as the starting path, i.e., structured light path 2. Stop after finding feature point C along structured light path 2 from starting feature point B. Figure 7 As shown. Determine whether there are other unsearched structured light strips at feature point B. There are two unsearched structured light strips, which are used as new starting paths, namely structured light paths 3 and 4. Start from feature point B and stop after finding feature points D and E along structured light paths 3 and 4, respectively. Figure 8 and 9 As shown. If feature point B does not contain other unsearched structured light strips, feature points C, D, and E are used as new starting feature points, and the above steps are followed to search for new next-level feature points. This continues until all feature points on the image are found, and the positional relationships of all feature points and the structured light paths between feature points are obtained.

[0106] In some embodiments of the present invention, the matching process of the structured light path between the feature points of the left camera image and the structured light path between the feature points of the right camera image is performed as follows:

[0107] The step of determining the structured light path between the feature points is performed on the left camera image and the right camera image respectively, so that the structured light path between the feature points of the left camera image and the structured light path between the feature points of the right camera image can be obtained respectively; since the complex structured light pattern is obtained by irradiating the surface of the object with the same structured light projection system, although the viewing angles of the left and right cameras are different, the images collected by the cameras at different positions with the same feature point and the same feature light path have different performances, but there is still correspondence between the feature points on the left image and the feature points on the right image;

[0108] All feature points extracted from the left and right images are used to construct feature descriptors one by one. Based on the feature descriptors, the feature points on the left and right images are matched to obtain the matching relationship between the feature points of the left image and the feature points of the right image;

[0109] Based on the correspondence between the feature points of the left image and the feature points of the right image and the structured light path between the feature points, a matching relationship between the structured light path between the feature points of the left image and the structured light path between the feature points of the right image is obtained;

[0110] Extracting the center position of the light strip from the structured light path between the feature points of the left image and the structured light path between the feature points of the right image respectively;

[0111] Based on the matching relationship between the optical structure path between the feature points of the left image and the structured light path between the feature points of the right image, the camera's intrinsic and extrinsic parameters are obtained by combining the binocular camera calibration, and the epipolar constraint is used to complete the stereo matching of the center point of the structured light strip; the three-dimensional reconstruction of complex structured light is realized to obtain the surface features of the object. The epipolar constraint condition is an important constraint in image matching. It is based on the geometric relationship in the spatial sequence image and is determined by the relationship between the camera optical center, the object point and the image plane.

[0112] Specifically, the feature points extracted from the left and right images of the target object are matched, such as Fig.10 According to the matching relationship between the structured light paths between the feature points in the image and the feature points in the left and right images, the matching relationship between the structured light paths between the feature points can be obtained, as shown in Fig.11 shown.

[0113] The light intensity distribution of the cross section of the structured light strip conforms to the Gaussian distribution. The light intensity at the center of the light strip is the strongest. The light intensity distribution of the cross section of the structured light strip is as follows: Fig.12 As shown in the figure, the center of the structured light strip to be extracted is located at the extreme point of the structured light intensity, which can reflect the structured light characteristics more accurately. Fig.13a , Fig.13b shown.

[0114] After the extraction is completed, the intrinsic and extrinsic parameters of the camera are obtained by combining the binocular camera calibration, and the stereo matching of the center point of the structured light strip is completed using the epipolar constraint condition; thus obtaining the three-dimensional information of the object surface. L and right camera optical center O R , three non-collinear points determine a plane, namely the epipolar plane. This plane intersects with the left camera image plane and the right camera image plane respectively, and the intersection lines are the epipolar lines T L , outer polar line T R The projection point of the midpoint P in space on the left camera image plane is P L , the projection point on the right camera image plane is P R ,like Fig.14 As shown. Based on the epipolar constraint, the center point P of the light strip extracted on the left image L The center point of the light strip extracted on the right image must be on the outer polar line T R On, that is, the projection point P R Similarly, the center point P of the light strip extracted on the right image R The center point of the light strip extracted on the right image must be on the outer polar line T L On, that is, the projection point P L ,like Fig.15a , Fig.15b As shown. The center point P of the light strip extracted on the left image L The center point P of the light strip extracted from the right image R That is, the matching point pair is obtained by combining the intrinsic and extrinsic parameters of the camera with the binocular camera calibration, and the three-dimensional coordinates of point P can be solved.

[0115] Compared with projector equipment, the present invention adopts multi-line structured light, which does not require complicated encoding preparation work in the early stage and is easy to use; in the later stage, it does not require complicated decoding process, does not require large computing power, saves computing resources, and has low hardware requirements. Compared with the scanning line structured light and linear array structured light projection methods, the present invention only needs to perform a projection task on the target object once to complete the acquisition work; it greatly reduces the time cost and ensures the real-time performance of the system. Compared with the traditional parallel structured light, each of the projection devices used in the present invention can project a number of parallel structured light strips, and the projection device is more efficient and cost-saving. The complex multi-line structured light pattern of the present invention is formed by the intersection of multiple groups of parallel multi-line structured lights, which contains information in multiple directions on the surface of the target object, and obtains more and more comprehensive three-dimensional information on the surface of the target object as much as possible. It makes up for the deficiency that traditional parallel structured light can only obtain information in a certain direction on the surface of the target object.

[0116] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A binocular stereo matching method based on complex structured light, characterized in that: It is implemented based on a matching system and is used to obtain surface features of objects; The matching system comprises: Structured light projection system: The structured light projection system includes a plurality of projection devices, each of which can emit a group of a plurality of mutually parallel linear structured light strips, and the linear structured light strips emitted by any two projection devices intersect, wherein each linear structured light strip emitted by each projection device has two endpoints; the linear structured light strips generate intersection points at the intersection positions; Binocular camera: includes a left camera and a right camera; used to simultaneously capture complex structured light images of the object surface to obtain left and right images; Processor: used to obtain images collected simultaneously by the binocular cameras, perform image processing, and execute the matching method, the method comprising: Processing the images captured by the left camera and the right camera at the same time, respectively obtaining feature points of the left image and the right image, wherein the feature points include: endpoint feature points and intersection feature points; Based on the endpoint feature point or the intersection feature point, searching for other feature points along the extension direction of the structured light strip, and determining the structured light path between the feature points, wherein the structured light path is the portion of the structured light strip between the feature points; All feature points extracted from the left image and the right image are used to construct feature descriptors one by one; based on the feature descriptors, the feature points on the left and right images are matched to obtain the matching relationship between the feature points of the left image and the feature points of the right image; Based on the correspondence between the feature points of the left image and the feature points of the right image, a matching relationship between the structured light path between the feature points of the left image and the structured light path between the feature points of the right image is obtained; Extracting the center position of the light strip from the structured light path between the feature points of the left image and the structured light path between the feature points of the right image respectively; Based on the matching relationship between the structured light path between the feature points of the left image and the structured light path between the feature points of the right image, the intrinsic and extrinsic parameters of the camera are obtained in combination with the binocular camera calibration, and the stereo matching of the center point of the structured light strip is completed using the epipolar constraint condition; the three-dimensional reconstruction of complex structured light is realized to obtain the surface features of the object; The method for determining the structured light path between the feature points includes connecting the feature points to each other through structured light strips, and further includes: S1: Select a feature point as the starting feature point, i.e. the K-th level feature point; S2: Select a structured light strip at the starting feature point as the starting path; S3: Starting from the starting feature point, along the starting path, find the next level feature point on the path, that is, the K+1th level feature point, which is defined as the end point feature point on the structured light path; S4: When the K+1th level feature point is found, the path stops searching; the positional relationship between the starting feature point and the ending feature point and the structured light path between the two feature points are obtained; S5: Determine the number of structured light strips at the starting feature point, select each structured light strip at the starting feature point as the starting path one by one, and find the K+1th level feature point on each starting path until all K+1th level feature points corresponding to the Kth level feature point are determined; S6: Select the K+1th level feature points one by one as new starting feature points; repeat steps S1, S2, S3, S4, S5; S7: According to the method of step S6, the next level of feature points are gradually searched until all feature points on the image are traversed, and the structured light paths between all feature points and the positional relationships between all feature points are obtained.

2. The binocular stereo matching method based on complex structured light according to claim 1, characterized in that: Multiple groups of parallel structured lights emitted by multiple projection devices are intertwined to form a complex structured light pattern.

3. The binocular stereo matching method based on complex structured light according to claim 1, characterized in that: Methods for determining intersection feature points include: Detecting the corner points included in the intersection position of the structured light strips, the number of the corner points is related to the number of intersecting light strips, and if M structured light strips intersect at one point, then the intersection position contains 2M corner points; Find the minimum covering circle containing all corner points at the intersection position; The center of the minimum coverage circle is taken as the intersection feature point where the light strips intersect.

4. The binocular stereo matching method based on complex structured light according to claim 1, characterized in that: Methods for determining endpoint feature points include: The endpoints of each structured light strip are taken as feature points.

5. The binocular stereo matching method based on complex structured light according to claim 1, characterized in that: In step S1, a feature point at the end point position is selected as a starting feature point.

Citation Information

Patent Citations

  • Binocular visual multi-line projection structured light calibration method

    CN106600647A

  • Three-dimensional scanner, system and three-dimensional reconstruction method

    CN112985307A

  • Multi-line laser scanning method and system based on binocular vision

    CN114723828A

  • Binocular vision three-dimensional reconstruction method and system

    CN114842144A

  • Multi-line structured light binocular vision stereo matching method and system

    CN115731301A