Phase shift and multi-line fused composite stripe structured light three-dimensional reconstruction method

Abstract

The invention discloses a phase shift and multi-line fused composite stripe structured light three-dimensional reconstruction method, which comprises the following steps: encoding phase shift stripes to a green channel, encoding multi-line stripes to a red channel, and carrying out single shooting after projection to obtain a fused image; respectively extracting a phase shift stripe and a multi-line stripe through channel separation, obtaining an absolute phase of the phase shift stripe by adopting a three-frequency heterodyne method, reconstructing a three-dimensional shape, and generating a stripe series matrix at the same time; the method comprises the following steps: aiming at a phase shift method failure reflection area, utilizing multi-line fringe supplementary measurement, adopting a Steger algorithm to extract a center line, carrying out line group classification, fracture line combination and global coding sorting based on phase shift fringe series prior information, and reconstructing a reflection area point cloud through binocular stereo vision; and finally fusing the two reconstruction results to generate a complete three-dimensional model. According to the method, high-precision three-dimensional reconstruction can be realized through single shooting, advantages of a phase shift method and a multi-line method are complementary, the problem of multi-line stripe fracture sorting is effectively solved by utilizing stripe series prior, and the measurement efficiency and the reconstruction quality of a reflective area are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional measurement technology, and in particular to a three-dimensional reconstruction method using composite stripe structured light that integrates phase shift and multi-line, which is especially suitable for application scenarios with reflective areas on the surface. Background Technology

[0002] Structured light 3D reconstruction methods, with their non-contact, high precision, and high efficiency, have been widely used in industrial inspection, medical imaging, and interactive entertainment. Among them, phase-shifting fringe structured light 3D reconstruction is one of the mainstream measurement methods. However, when there are reflective areas on the surface of the object being measured, this technique is prone to local overexposure, resulting in light spots and distortion of phase information, which severely limits the measurement effect under reflective conditions.

[0003] To alleviate the aforementioned problems, multi-line fringe structured light 3D reconstruction methods are commonly used. While this method can reduce reflective interference to some extent, the measurement accuracy in non-reflective areas is highly dependent on fringe density and sharpness. Although multi-line fringe structured light 3D reconstruction can reduce the number of projected images, when measuring objects with complex geometric features, deep holes, or surfaces with abrupt curvature, the projected fringes are prone to breakage due to occlusion or deformation, preventing the camera from capturing a complete image. This further requires the decoding algorithm to correctly connect and sort the broken fringes, and any error in this process can cause distortion or voids in the 3D model. Notably, the sorting and decoding of broken fringes is computationally complex, typically requiring algorithms such as logical encoding or mean nearest neighbor to track the fringe centerline, which increases the difficulty of algorithm implementation and the uncertainty of results. If phase-shifting fringe method and multi-line fringe method are used to measure the non-reflective and reflective areas of the same object respectively, multiple sets of images need to be acquired, leading to reduced measurement efficiency, increased requirements for ambient light stability, and extremely complex algorithms for aligning and fusing images from different exposures. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a three-dimensional reconstruction method for composite stripe structured light that integrates phase shift and multi-line structures.

[0005] To achieve the above objectives, the technical solution provided by this invention is as follows:

[0006] A method for three-dimensional reconstruction using composite fringe structured light that integrates phase shifting and multi-line framing includes the following steps:

[0007] Generate a fused image of phase-shifted stripes and multi-line stripes;

[0008] The generated phase-shifted fringe and multi-line fringe fusion image is projected onto the object surface through a projector, and the fringe fusion image modulated by the object surface is captured using an image acquisition device.

[0009] The image acquisition device acquires a fused stripe image modulated by the object surface, and performs color channel separation on the fused stripe image to extract the phase-shifted stripe image and multi-line stripe image modulated by the object surface.

[0010] The three-dimensional shape of the object is reconstructed using a phase-shifted fringe image modulated by the object's surface, and the phase-shifted fringe reconstruction result is obtained.

[0011] The three-dimensional topography of the reflective area on the object surface is reconstructed using a multi-line stripe image modulated by the object surface, and the multi-line stripe reconstruction result is obtained.

[0012] The phase-shifting fringe reconstruction results are fused with the multi-line fringe reconstruction results to obtain the final three-dimensional topography reconstruction model.

[0013] Further, a phase-shifting fringe and multi-line fringe fusion image is generated, including:

[0014] Generate phase-shifted fringe images:

[0015] In an image with the same resolution as the projector, a set of three-step phase-shifted fringes is generated. The grayscale change of each fringe follows a sinusoidal law, and its brightness is determined by the pixel's lateral position, the set fringe period, and the phase shift, which are respectively... ;

[0016] Generate a multi-line stripe image:

[0017] The grayscale values ​​of the generated phase-shifted fringe image are normalized to the range of 0 to 1. Then, based on each normalized image, two corresponding multi-line fringe black-and-white binary images are generated: in the first multi-line fringe black-and-white binary image, pixels with a grayscale value of 1 in the original image are set to white, and the rest are set to black; in the second multi-line fringe black-and-white binary image, pixels with a grayscale value close to 0 in the original image are set to white, and the rest are set to black.

[0018] Next, a completely black RGB image is created as a carrier, and the generated phase-shifted stripe image is encoded into its green channel; the two multi-line stripe black and white binary images are superimposed and encoded into the red channel; the blue channel is kept at 0; thus, a phase-shifted stripe and multi-line stripe fused image is synthesized.

[0019] Furthermore, the three-dimensional topography of the reflective areas on the object's surface is reconstructed using a multi-line stripe image modulated by the object's surface, including:

[0020] Locate the stripe order on the reflective area of ​​an object's surface;

[0021] Extract the center line of the multi-line stripes;

[0022] Preliminary grouping of the subpixel-level centerline coordinates of the multi-line stripes;

[0023] Using stripe series as an auxiliary classification method;

[0024] Within each line group class, multiple line groups belonging to the same line stripe are merged;

[0025] Merge the set of line groups of all stripe levels in the same period;

[0026] Encode and sort the center lines of the multi-line stripes;

[0027] The binocular 3D shape reconstruction is completed by mapping the multi-line sorting of the projector with the multi-line sorting of the camera image.

[0028] Furthermore, the number of stripe grades on the reflective areas of the object's surface is determined, including:

[0029] By comparing the fringe series matrix determined by phase-shifted fringes with the positions of reflective areas on the object's surface, the set of fringe series containing the reflective areas on the object's surface can be found. ,set up ,in Represent a stripe level; let the image coordinates be... For integer pixel positions, generate a binary array based on the stripe series matrix, the first... A binary array is defined as follows:

[0030] .

[0031] Furthermore, the sub-pixel-level centerline coordinates of the multi-line stripes are preliminarily grouped, including:

[0032] First, the multi-line stripes located in the reflective area are extracted, and the Steger algorithm is used to calculate the sub-pixel coordinates of the center point of each multi-line stripe;

[0033] Suppose the obtained sub-pixel coordinate dataset of the center points of the multi-line stripes is:

[0034]

[0035] Among them, index variables Used to identify phase shifts belonging to different periods. Corresponding period Three different phase shifts, Corresponding period Three different phase shifts, Corresponding period Three different phase shifts; Let the coordinates of the points on the center line of the multi-line stripe be given; we begin by initially grouping the sub-pixel center line point sets of the multi-line stripes under different sinusoidal phase shift periods: let the remaining coordinate set be given. Grouping results In the remaining coordinate set Select The point with the largest coordinates :

[0036]

[0037] by Iteratively find neighboring points to form a line group from the starting point: Construct temporary line groups , starting point From the set Remove from the middle and insert a temporary wire group. Select middle The point with the smallest coordinates :

[0038]

[0039] Find neighboring points condition:

[0040]

[0041] in and The distance tolerance threshold between the center points of the two fringes is set as follows: Add line group and from Remove from the list, then recursively execute the selection. Select new And find its neighboring points The steps continue until no point satisfies the nearest neighbor condition, then the temporary line group... Once completed, it will be marked as... The above line group construction process updates the remaining point set. Iterate until it is empty; eventually, a series of non-intersecting line groups are generated, denoted as:

[0042]

[0043] in Indicate each The number of line groups included. Index it; For dataset Line groups after initial internal grouping;

[0044] After initial grouping, the grouped data are merged into three main line groups. :

[0045] ;

[0046] in This represents the period of different sinusoidal phase shift fringes.

[0047] Furthermore, the classification of line groups is aided by the stripe series, including:

[0048] right The coordinates of the center points of multi-line stripes are classified according to the stripe level:

[0049] First, extract the coordinates contained in each stripe level and put them into the coordinate array corresponding to each stripe level. :

[0050]

[0051] Select Each line group contained within inside Minimum value and maximum value Time Point , :

[0052]

[0053] in and They are respectively the corresponding Extreme points Coordinates; if and The corresponding coordinates are all in the coordinate array If it is inside, then it is classified into the stripe level. Corresponding line group class ;like The corresponding coordinates are in the coordinate array Inside, but The corresponding coordinates are in the coordinate array If it is inside, then it is classified into the stripe level. +1 corresponds to the line group type The final result is:

[0054]

[0055] .

[0056] Furthermore, within each line group class, multiple line groups belonging to the same line stripe are merged, including:

[0057] For each line group class Internal The line groups are initially sorted: Let... , For each line group class Internal wiring harness, For line group class Internal set of line groups; calculate each line group Maximum point:

[0058]

[0059] in correspond Maximum point coordinate, express The inner line group, all of them Sort in ascending order to obtain an ordered sequence. And an index arrangement , The sequence satisfies:

[0060]

[0061] according to Sort pairs Corresponding line group Reorder:

[0062]

[0063] Reorder the line groups Reassign the value back to the original line group class. For each line group class Line group Let the set of remaining line groups be... merging result set ;choose The first array As the current processing line group and from Remove, calculate the current line group of Maximum coordinate point:

[0064]

[0065] calculate One of the line groups of Minimum coordinate point:

[0066]

[0067] like

[0068]

[0069] Then the line group and merge and from Removed from the middle and The corresponding difference threshold is used; the merged line group continues to search for whether... Line groups belonging to the same line will be merged and added to the result set. In the middle, then the current The first array as Repeat the above merging steps until no line groups that meet the conditions can be merged. Recorded as The above line combination merging process updates the remaining line group set. Iterate through the process until it becomes empty; finally, you get the line group class. Line group class collection after merging line groups :

[0070]

[0071] in, For the stripe level The number of new line groups obtained after combining the inner lines corresponds to the number of stripe levels. The number of complete lines; This is the merged line group.

[0072] Furthermore, the center lines of the multi-line stripes are encoded and sorted, including:

[0073] The same period The set of line groups of all stripe levels By merging, we obtain three sets of line groups with different periods. ,right All coordinate values ​​are rounded down to the nearest integer, and the image coordinates are set as follows. For integer pixel positions; begin encoding and sorting the center line: for each period and stripe levels The number of center lines is For each component The minimum value method was used to calculate the horizontal coordinates representing the centerline. :

[0074]

[0075] according to Sort the center line in ascending order to obtain an ordered list. Define the initial encoded label image. :

[0076]

[0077] in It is an online group collection After sorting, the center lines are sequentially numbered, and duplicate center lines are detected and processed simultaneously: for each cycle. and stripe levels Define a binary centered image ,in:

[0078]

[0079] For each Create a superimposed image for detecting repeating center lines. :

[0080]

[0081] Identify pixels with values ​​greater than 1 in the overlay image; these pixels indicate that at least two center lines of different periods coincide at this location. Process the repeating center lines and define the processed binary image. :for Always keep:

[0082]

[0083] for Eliminate and Repeated parts:

[0084]

[0085] for Eliminate and Repeated parts:

[0086]

[0087] Then, perform internal sorting within the period, first determining the maximum label value. :

[0088]

[0089] Define offset To avoid confusion when merging centerline labels from different periods, the following measures are taken: and Offset the centerline label to create a new label pattern. and :

[0090]

[0091]

[0092] Ensure that the labels of the three periods do not overlap; create a periodically merged image. The pixel values ​​of this image are obtained by multiplying the binary images processed in each period by their corresponding offset label images point by point and then summing them to obtain an image where each center line has a unique identifier:

[0093]

[0094] exist In this context, pixels with a value of 0 represent those that do not belong to any center line, while non-zero pixels belong to a specific center line, and their pixel values ​​uniquely identify that center line. Find... All non-zero pixel values ​​that appear in the image; these unique non-zero values ​​constitute a set. ,in For set The number of internal elements; for each unique value The representative horizontal coordinates of its corresponding centerline are calculated using the minimum value method. :

[0095]

[0096] Sort by geometric features: based on the calculated representative horizontal coordinates For a set of unique values Sort in ascending order to obtain an ordered list. ;in The corresponding center line is on the far left. The corresponding center line is on the far right. This represents the total number of global centerlines; it defines the final global label image. :

[0097]

[0098] Among them, the label A globally unified encoding sequence number is assigned to the center line, reflecting the overall order of all center lines from left to right; finally, before reconstructing the 3D point cloud based on the binocular stereo vision model, the following steps are taken: and Subtract the two numbers and you get a value that is greater than zero. That is The first one to be removed from the corresponding projector image Multiple lines, by and Subtract the two numbers and you get a value that is greater than zero. That is The first one to be removed from the corresponding projector image Multiple lines ensure the projected multi-line image and The center lines correspond one-to-one, among which and To calculate the difference images separately and The index markers of the redundant stripes that are identified and need to be removed from the original projected image.

[0099] Compared with existing technologies, the principles and advantages of this technical solution are as follows:

[0100] 1. By using color channel fusion, no additional data acquisition is required, which is far less than with pure multi-line stripe schemes, greatly improving measurement efficiency.

[0101] 2. Combining the advantages of phase-shifting method and multi-line method, phase-shifting method can obtain high-precision results in unsaturated regions, while multi-line method can compensate for the surface reconstruction results in saturated regions.

[0102] 3. Using the fringe series provided by the phase shift method as prior information can effectively assist in the sorting problem of the multi-line method and avoid the problem that the traditional multi-line method has difficulty in locating the absolute order. Attached Figure Description

[0103] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0104] Figure 1 This is a schematic diagram of the structured light projection three-dimensional measurement system used in an embodiment of the present invention;

[0105] Figure 2 This is a flowchart illustrating the principle of a three-dimensional reconstruction method for composite stripe structured light that integrates phase shifting and multi-line structures, according to an embodiment of the present invention.

[0106] Figure 3 This is a schematic diagram illustrating the data processing of an image acquired by an image acquisition device after it is transmitted into a computer, as described in an embodiment of the present invention.

[0107] Figure 4 This is a schematic diagram of the centerline coding and sorting in an embodiment of the present invention;

[0108] Figure 5To separate the green channel image and the red channel image from the stripe fusion image acquired by the image acquisition device (a is the stripe fusion image acquired by the image acquisition device; b is the separated green channel image; c is the separated red channel image; the green channel image contains phase-shifted stripes, and the red channel image contains multi-line stripes). Detailed Implementation

[0109] The present invention will be further described below with reference to specific embodiments:

[0110] like Figure 1 As shown, the structured light projection three-dimensional measurement system used in this embodiment includes a computer 1, a projector 2, and a color industrial camera 3.

[0111] like Figure 2 As shown in this embodiment, a three-dimensional reconstruction method for composite stripe structured light that integrates phase shifting and multi-line structures includes the following steps:

[0112] S1. A phase-shifting fringe and multi-line fringe fusion image is generated by computer 1;

[0113] This step includes:

[0114] Generate phase-shifted fringe images:

[0115] In an image with the same resolution as the projector, a set of three-step phase-shifted fringes is generated. The grayscale change of each fringe follows a sinusoidal law, and its brightness is determined by the pixel's lateral position, the set fringe period, and the phase shift, which are respectively... ;

[0116] Generate a multi-line stripe image:

[0117] The grayscale values ​​of the generated phase-shifted fringe image are normalized to the range of 0 to 1. Then, based on each normalized image, two corresponding multi-line fringe black-and-white binary images are generated: in the first multi-line fringe black-and-white binary image, pixels with a grayscale value of 1 in the original image are set to white, and the rest are set to black; in the second multi-line fringe black-and-white binary image, pixels with a grayscale value close to 0 in the original image are set to white, and the rest are set to black.

[0118] Next, a completely black RGB image is created as a carrier, and the generated phase-shifted stripe image is encoded into its green channel; the two multi-line stripe black and white binary images are superimposed and encoded into the red channel; the blue channel is kept at 0; thus, a phase-shifted stripe and multi-line stripe fused image is synthesized.

[0119] S2. The phase-shifted fringe and multi-line fringe fusion image generated by computer 1 is projected onto the object surface through projector 2, and the fringe fusion image modulated by the object surface is captured using image acquisition equipment (color industrial camera 3).

[0120] S3, Computer 1 acquires a stripe fusion image modulated by the object surface from color industrial camera 3, and performs color channel separation on the stripe fusion image (e.g., Figure 5 As shown), extract the phase-shifted fringe image and multi-line fringe image modulated by the object surface;

[0121] S4. Computer 1 uses the phase-shifted fringe image modulated on the object surface to reconstruct the three-dimensional shape of the object and obtain the phase-shifted fringe reconstruction result.

[0122] S5. Computer 1 uses the multi-line stripe image modulated on the object surface to reconstruct the three-dimensional shape of the reflective area on the object surface, and obtains the multi-line stripe reconstruction result.

[0123] This step includes:

[0124] S5-1. Determine the stripe order on the reflective area of ​​the object's surface:

[0125] By comparing the fringe series matrix determined by phase-shifted fringes with the positions of reflective areas on the object's surface, the set of fringe series containing the reflective areas on the object's surface can be found. ,set up ,in Represent a stripe level; let the image coordinates be... For integer pixel positions, generate a binary array based on the stripe series matrix, the first... A binary array is defined as follows:

[0126] .

[0127] S5-2. Extract the center line of the multi-line stripes;

[0128] S5-3. Perform preliminary grouping of the sub-pixel-level centerline coordinates of the multi-line stripes:

[0129] First, the multi-line stripes located in the reflective area are extracted, and the Steger algorithm is used to calculate the sub-pixel coordinates of the center point of each multi-line stripe;

[0130] Suppose the obtained sub-pixel coordinate dataset of the center points of the multi-line stripes is:

[0131]

[0132] Among them, index variables Used to identify phase shifts belonging to different periods. Corresponding period Three different phase shifts, Corresponding period Three different phase shifts, Corresponding period Three different phase shifts; Let the coordinates of the points on the center line of the multi-line stripe be given; we begin by initially grouping the sub-pixel center line point sets of the multi-line stripes under different sinusoidal phase shift periods: let the remaining coordinate set be given. Grouping results In the remaining coordinate set Select The point with the largest coordinates :

[0133]

[0134] by Iteratively find neighboring points to form a line group from the starting point: Construct temporary line groups , starting point From the set Remove from the middle and insert a temporary wire group. Select middle The point with the smallest coordinates :

[0135]

[0136] Find neighboring points condition:

[0137]

[0138] in and The distance tolerance threshold between the center points of the two fringes is set as follows: Add line group and from Remove from the list, then recursively execute the selection. Select new And find its neighboring points The steps continue until no point satisfies the nearest neighbor condition, then the temporary line group... Once completed, it will be marked as... The above line group construction process updates the remaining point set. Iterate until it is empty; eventually, a series of non-intersecting line groups are generated, denoted as:

[0139]

[0140] in Indicate each The number of line groups included. Index it; For dataset Line groups after initial internal grouping;

[0141] After initial grouping, the grouped data are merged into three main line groups. :

[0142] ;

[0143] in This represents the period of different sinusoidal phase shift fringes.

[0144] S5-4. Classification using stripe series as an auxiliary line group:

[0145] right The coordinates of the center points of multi-line stripes are classified according to the stripe level:

[0146] First, extract the coordinates contained in each stripe level and put them into the coordinate array corresponding to each stripe level. :

[0147]

[0148] Select Each line group contained within inside Minimum value and maximum value Time Point , :

[0149]

[0150] in and They are respectively the corresponding Extreme points Coordinates; if and The corresponding coordinates are all in the coordinate array If it is inside, then it is classified into the stripe level. Corresponding line group class ;like The corresponding coordinates are in the coordinate array Inside, but The corresponding coordinates are in the coordinate array If it is inside, then it is classified into the stripe level. +1 corresponds to the line group type The final result is:

[0151]

[0152] .

[0153] S5-5. Within each line group class, merge multiple line groups belonging to the same line stripe:

[0154] For each line group class Internal Perform preliminary sorting of the line groups: Let... , For each line group class Internal wiring harness, For line group class Internal set of line groups; calculate each line group Maximum point:

[0155]

[0156] in correspond Maximum point coordinate, express The inner line group, all of them Sort in ascending order to obtain an ordered sequence. And an index arrangement , The sequence satisfies:

[0157]

[0158] according to Sort pairs Corresponding line group Reorder:

[0159]

[0160] Reorder the line groups Reassign the value back to the original line group class. For each line group class Line group Let the set of remaining line groups be... merging result set ;choose The first array As the current processing line group and from Remove, calculate the current line group of Maximum coordinate point:

[0161]

[0162] calculate One of the line groups of Minimum coordinate point:

[0163]

[0164] like

[0165]

[0166] Then the line group and merge and from Removed from the middle and The corresponding difference threshold is used; the merged line group continues to search for whether... Line groups belonging to the same line will be merged and added to the result set. In the middle, then the current The first array as Repeat the above merging steps until no line groups that meet the conditions can be merged. Recorded as The above line combination merging process updates the remaining line group set. Iterate through the process until it becomes empty; finally, you get the line group class. Line group class collection after merging line groups :

[0167]

[0168] in, For the stripe level The number of new line groups obtained after combining the inner lines corresponds to the number of stripe levels. The number of complete lines; This is the merged line group.

[0169] S5-6. Merge the line group sets of all stripe levels in the same period;

[0170] S5-7, such as Figure 4 As shown, the center lines of the multi-line stripes are encoded and sorted:

[0171] The same period The set of line groups of all stripe levels By merging, we obtain three sets of line groups with different periods. ,right All coordinate values ​​are rounded down to the nearest integer, and the image coordinates are set as follows. For integer pixel positions; begin encoding and sorting the center line: for each period and stripe levels The number of center lines is For each component The minimum value method was used to calculate the horizontal coordinates representing the centerline. :

[0172]

[0173] according to Sort the center line in ascending order to obtain an ordered list. Define the initial encoded label image. :

[0174]

[0175] Where n is the set of online groups After sorting, the center lines are sequentially numbered, and duplicate center lines are detected and processed simultaneously: for each cycle. and stripe levels Define a binary centered image ,in:

[0176]

[0177] For each Create a superimposed image for detecting repeating center lines. :

[0178]

[0179] Identify pixels with values ​​greater than 1 in the overlay image; these pixels indicate that at least two center lines of different periods coincide at this location. Process the repeating center lines and define the processed binary image. :for Always keep:

[0180]

[0181] for Eliminate and Repeated parts:

[0182]

[0183] for Eliminate and Repeated parts:

[0184]

[0185] Then, perform internal sorting within the period, first determining the maximum label value. :

[0186]

[0187] Define offset To avoid confusion when merging centerline labels from different periods, the following measures are taken: and Offset the centerline label to create a new label pattern. and :

[0188]

[0189]

[0190] Ensure that the labels of the three periods do not overlap; create a periodically merged image. The pixel values ​​of this image are obtained by multiplying the binary images processed in each period by their corresponding offset label images point by point and then summing them to obtain an image with a unique identifier for each center line:

[0191]

[0192] exist In this context, pixels with a value of 0 represent those that do not belong to any center line, while non-zero pixels belong to a specific center line, and their pixel values ​​uniquely identify that center line. Find... All non-zero pixel values ​​that appear in the image; these unique non-zero values ​​constitute a set. ,in For set The number of internal elements; for each unique value The representative horizontal coordinates of its corresponding centerline are calculated using the minimum value method. :

[0193]

[0194] Sort by geometric features: based on the calculated representative horizontal coordinates For a set of unique values Sort in ascending order to obtain an ordered list. ;in The corresponding center line is on the far left. The corresponding center line is on the far right. This represents the total number of global centerlines; it defines the final global label image. :

[0195]

[0196] Among them, the label A globally unified encoding sequence number is assigned to the center line, reflecting the overall order of all center lines from left to right; finally, before reconstructing the 3D point cloud based on the binocular stereo vision model, the following steps are taken: and Subtract the two numbers and you get a value that is greater than zero. That is The first one to be removed from the corresponding projector image Multiple lines, by and Subtract the two numbers and you get a value that is greater than zero. That is The first one to be removed from the corresponding projector image Multiple lines ensure the projected multi-line image and The center lines correspond one-to-one, among which and To calculate the difference images separately and The index markers of the redundant stripes that are identified and need to be removed from the original projected image.

[0197] S5-8. Map the multi-line sorting of the projector with the multi-line sorting of the camera image to complete the binocular 3D shape reconstruction.

[0198] S6. Computer 1 fuses the phase-shifting fringe reconstruction results with the multi-line fringe reconstruction results to obtain the final three-dimensional topography reconstruction model.

[0199] The above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Therefore, any changes made in accordance with the shape and principle of the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for three-dimensional reconstruction using composite stripe structured light that integrates phase shifting and multi-line techniques, characterized in that, Includes the following steps: Generate a fused image of phase-shifted stripes and multi-line stripes; The generated phase-shifted fringe and multi-line fringe fusion image is projected onto the object surface through a projector, and the fringe fusion image modulated by the object surface is captured using an image acquisition device. The image acquisition device acquires a fused stripe image modulated by the object surface, and performs color channel separation on the fused stripe image to extract the phase-shifted stripe image and multi-line stripe image modulated by the object surface. The three-dimensional shape of the object is reconstructed using a phase-shifted fringe image modulated by the object's surface, and the phase-shifted fringe reconstruction result is obtained. The three-dimensional topography of the reflective area on the object surface is reconstructed using a multi-line stripe image modulated by the object surface, and the multi-line stripe reconstruction result is obtained. The phase-shifting fringe reconstruction results are fused with the multi-line fringe reconstruction results to obtain the final three-dimensional topography reconstruction model.

2. The method for three-dimensional reconstruction of composite striped structured light combining phase shift and multi-line structures according to claim 1, characterized in that, Generate a phase-shifted fringe and multi-line fringe fusion image, including: Generate phase-shifted fringe images: In an image with the same resolution as the projector, a set of three-step phase-shifted fringes is generated. The grayscale change of each fringe follows a sinusoidal law, and its brightness is determined by the pixel's lateral position, the set fringe period, and the phase shift, which are respectively... ; Generate a multi-line stripe image: The grayscale values ​​of the generated phase-shifted fringe image are normalized to the range of 0 to 1. Then, based on each normalized image, two corresponding multi-line fringe black-and-white binary images are generated: in the first multi-line fringe black-and-white binary image, pixels with a grayscale value of 1 in the original image are set to white, and the rest are set to black; in the second multi-line fringe black-and-white binary image, pixels with a grayscale value close to 0 in the original image are set to white, and the rest are set to black. Next, a completely black RGB image is created as a carrier, and the generated phase-shifted stripe image is encoded into its green channel; the two multi-line stripe black and white binary images are superimposed and encoded into the red channel; the blue channel is kept at 0; thus, a phase-shifted stripe and multi-line stripe fused image is synthesized.

3. The method for three-dimensional reconstruction of composite striped structured light combining phase shift and multi-line structures according to claim 1, characterized in that, The three-dimensional topography of reflective areas on an object's surface is reconstructed using a multi-line stripe image modulated by the object's surface, including: Locate the stripe order on the reflective area of ​​an object's surface; Extract the center line from the multi-line stripes; Preliminary grouping of the subpixel-level centerline coordinates of the multi-line stripes; Using stripe series as an auxiliary classification method; Within each line group class, multiple line groups belonging to the same line stripe are merged; Merge the set of line groups of all stripe levels in the same period; Encode and sort the center lines of the multi-line stripes; The binocular 3D shape reconstruction is completed by mapping the multi-line sorting of the projector with the multi-line sorting of the camera image.

4. The method for three-dimensional reconstruction of composite striped structured light combining phase shift and multi-line structures according to claim 3, characterized in that, Locate the stripe order on the reflective areas of an object's surface, including: By comparing the fringe series matrix determined by phase-shifted fringes with the positions of reflective areas on the object's surface, the set of fringe series containing the reflective areas on the object's surface can be found. ,set up ,in Represent a stripe level; let the image coordinates be... For integer pixel positions, generate a binary array based on the stripe series matrix, the first... A binary array is defined as follows: 。 5. The method for three-dimensional reconstruction of composite stripe structured light that integrates phase shifting and multi-line structures according to claim 4, characterized in that, The subpixel-level centerline coordinates of the multi-line stripes are initially grouped, including: First, the multi-line stripes located in the reflective area are extracted, and the Steger algorithm is used to calculate the sub-pixel coordinates of the center point of each multi-line stripe; Suppose the obtained sub-pixel coordinate dataset of the center points of the multi-line stripes is: Among them, index variables Used to identify phase shifts belonging to different periods. Corresponding period Three different phase shifts, Corresponding period Three different phase shifts, Corresponding period Three different phase shifts; Let the coordinates of the points on the center line of the multi-line stripe be given; we begin by initially grouping the sub-pixel center line point sets of the multi-line stripes under different sinusoidal phase shift periods: let the remaining coordinate set be given. Grouping results In the remaining coordinate set Select The point with the largest coordinates : ； by Iteratively find neighboring points to form a line group from the starting point: Construct temporary line groups , starting point From the set Remove from the middle and insert a temporary wire group. Select middle The point with the smallest coordinates : ; Find neighboring points condition: ; in and The distance tolerance threshold between the center points of the two fringes is set as follows: Add line group and from Remove from the list, then recursively execute the selection. Select new And find its neighboring points The steps continue until no point satisfies the nearest neighbor condition, then the temporary line group... Once completed, it will be marked as... The line group construction process updates the remaining point set. Iterate until it is empty; eventually, a series of non-intersecting line groups are generated, denoted as: ； Where s represents each The number of line groups included. Index it; For dataset Line groups after initial internal grouping; After initial grouping, the grouped data are merged into three main line groups. : ;in This represents the period of different sinusoidal phase shift fringes.

6. The method for three-dimensional reconstruction of composite stripe structured light combining phase shift and multi-line structures according to claim 5, characterized in that, Using stripe series as an auxiliary line group for classification, including: The coordinates of the center points of multi-line stripes are classified according to the stripe level: First, extract the coordinates contained in each stripe level and put them into the coordinate array corresponding to each stripe level. : ; Select Each line group contained within inside Minimum value and maximum value Time Point , : ； in and They are respectively the corresponding Extreme points Coordinates; if and The corresponding coordinates are all in the coordinate array If it is inside, then it is classified into the stripe level. Corresponding line group class ;like The corresponding coordinates are in the coordinate array Inside, but The corresponding coordinates are in the coordinate array If it is inside, then it is classified into the stripe level. +1 corresponds to the line group type The final result is: ； 。 7. The method for three-dimensional reconstruction of composite stripe structured light integrating phase shift and multi-line structures according to claim 6, characterized in that, Within each line group class, multiple line groups belonging to the same line stripe are merged, including: For each line group class Internal The line groups are initially sorted: Let... , For each line group class Internal wiring harness, For line group class Internal set of line groups; calculate each line group Maximum point: ； in correspond Maximum point coordinate, express The inner line group, all of them Sort in ascending order to obtain an ordered sequence. And an index arrangement , The sequence satisfies: ； according to Sort pairs Corresponding line group Reorder: ； Reorder the line groups Reassign the value back to the original line group class. For each line group class Line group Let the set of remaining line groups be... merging result set ;choose The first array As the current processing line group and from Remove, calculate the current line group of Maximum coordinate point: ； calculate One of the line groups of Minimum coordinate point: ； like ； Then the line group and merge and from Removed from the middle and The corresponding difference threshold is used; the merged line group continues to search for whether... Line groups belonging to the same line will be merged and added to the result set. In the middle, then the current The first array as Repeat the merging steps until no more line groups that meet the conditions can be merged. Then, at this point... Recorded as The line group merging process updates the set of remaining line groups. Iterate through the process until it becomes empty; finally, you get the line group class. Line group class collection after merging line groups : ； in, For the stripe level The number of line groups obtained after combining the inner lines corresponds to the number of stripe levels. The number of complete lines; This is the merged line group.

8. The method for three-dimensional reconstruction of composite stripe structured light integrating phase shift and multi-line structures according to claim 7, characterized in that, Encoding and sorting the center lines of multi-line stripes includes: The same period The set of line groups of all stripe levels By merging, we obtain three sets of line groups with different periods. ,right All coordinate values ​​are rounded down to the nearest integer, and the image coordinates are set as follows. For integer pixel positions; begin encoding and sorting the center line: for each period and stripe levels The number of center lines is For each component The minimum value method was used to calculate the horizontal coordinates representing the centerline. : ； according to Sort the center line in ascending order to obtain an ordered list. Define the initial encoded label image. : ； in It is an online group collection After sorting, the center lines are sequentially numbered, and duplicate center lines are detected and processed simultaneously: for each cycle. and stripe levels Define a binary centered image ,in: ； For each Create a superimposed image for detecting repeating center lines. : ； Identify pixels with values ​​greater than 1 in the overlay image; these pixels indicate that at least two center lines of different periods coincide at this location. Process the repeating center lines and define the processed binary image. :for Always keep: ； for Eliminate and Repeated parts: ； for Eliminate and Repeated parts: ； Then, perform internal sorting within the period, first determining the maximum label value. : ； Define offset To avoid confusion when merging centerline labels from different periods, the following measures are taken: and Offset the centerline label to create a new label pattern. and : ； ； Ensure that the labels of the three periods do not overlap; create a periodically merged image. The pixel values ​​of this image are obtained by multiplying the binary images processed in each period by their corresponding offset label images point by point and then summing them to obtain an image where each center line has a unique identifier: ； exist In this context, pixels with a value of 0 represent those that do not belong to any center line, while non-zero pixels belong to a specific center line, and their pixel values ​​uniquely identify that center line. Find... All non-zero pixel values ​​that appear in the image; these unique non-zero values ​​constitute a set. ,in For set The number of internal elements; for each unique value The representative horizontal coordinates of its corresponding centerline are calculated using the minimum value method. : ； Sort by geometric features: based on the calculated representative horizontal coordinates For a set of unique values Sort in ascending order to obtain an ordered list. ;in The corresponding center line is on the far left. The corresponding center line is on the far right. This represents the total number of global centerlines; it defines the final global label image. : ； Among them, the label A globally unified encoding sequence number is assigned to the center line, reflecting the overall order of all center lines from left to right; finally, before reconstructing the 3D point cloud based on the binocular stereo vision model, the following steps are taken: and Subtract the two numbers and you get a value that is greater than zero. That is The first one to be removed from the corresponding projector image Multiple lines, by and Subtract the two numbers and you get a value that is greater than zero. That is The first one to be removed from the corresponding projector image Multiple lines ensure the projected multi-line image and The center lines correspond one-to-one, among which and To calculate the difference images separately and The index markers of the redundant stripes that are identified and need to be removed from the original projected image.

Images