Structured light fast three-dimensional imaging and measuring device and method

Abstract

The application relates to the field of optical three-dimensional measurement technology and discloses a structured light fast three-dimensional imaging and measuring device and method, which comprises a camera, a plurality of projectors and a computer device, wherein the computer device is pre-scanned with a low-frequency geometric sub-sequence and an evaluation phase shift sub-sequence with the same frequency as formal measurement to form a region-direction observability confidence distribution and a correlation mapping; a projection combination is selected according to coverage, maximum confidence and the number of directions, and coding is configured according to the pixel proportion of a low-observability region; confidence is used to guide phase unwrapping, three-dimensional fusion and multi-projection boundary phase selection, and the unobservable region is compensated in combination with geometric continuity constraints, so that formal measurement redundancy image acquisition is reduced, phase unwrapping error rate is reduced, three-dimensional point cloud integrity is improved, and compensation boundary continuity is improved; when a target object and a measuring posture thereof are unchanged, subsequent measurement reuses the above distribution, correlation mapping, projection combination and coding configuration, so that repeated pre-scanning and scheduling are reduced, and the measuring time is shortened.

Description

Technical Field

[0001] This invention relates to the field of optical three-dimensional measurement technology, specifically to a structured light rapid three-dimensional imaging and measurement device and method. Background Technology

[0002] Structured light 3D imaging and measurement projects phase-shifted fringes and coded patterns onto a target object using a projector. A camera then captures structured light images modulated on the target surface and reconstructs the 3D shape of the target object based on absolute phase, system calibration parameters, and optical triangulation. When the target object has deep holes, concave structures, steep steps, or significant changes in surface normals, a single projection direction can easily cause light path obstruction. Highly reflective or lowly reflective surfaces may also cause grayscale saturation or insufficient fringe modulation, leading to phase unfolding errors and missing 3D point clouds.

[0003] To reduce occlusion caused by a single projection direction, CN118960618A discloses a single-camera multi-projection structured light measurement system and a 3D reconstruction method, which obtains height data through multiple projection directions and fuses unoccluded areas. This scheme can supplement some occluded areas, but when the combination of projection directions is fixed in advance, or when the same encoding, acquisition, and processing are performed on all projection directions, it is easy to generate repeated projection, repeated acquisition, and repeated calculation.

[0004] Different target surface regions exhibit variations in fringe modulation, grayscale response stability, and spatial orientation under different projection directions, with the fringe modulation varying with the fringe spatial frequency. If response evaluation results are obtained using only low-spatial-frequency fringes, while higher-spatial-frequency fringes are used in the formal measurement, the evaluation results may not accurately reflect the effective response during the formal measurement phase. Periodic error propagation may also occur when phase unwrapping passes through low-quality regions. For regions still missing after multi-projection measurements, the B-spline surface point cloud hole-filling method disclosed in CN1945626A primarily relies on neighborhood point clouds for surface fitting, failing to utilize the still valid absolute phase information at the boundaries of missing regions under different projection directions.

[0005] Therefore, it is necessary to simultaneously obtain low spatial frequency phase data for geometric reconstruction and structured light response data with the same spatial frequency as the formal measurement during the pre-scanning stage, and to continuously use the same associated mapping formed therefrom for projection direction selection, encoding parameter configuration, phase unfolding path determination, multi-projection 3D data fusion, and compensation boundary phase selection, so as to reduce redundant projection and image acquisition in subsequent measurements while meeting measurement coverage requirements, and improve the boundary continuity of compensation results for residual unobservable areas. Summary of the Invention

[0006] The technical problem to be solved by this invention is: in structured light measurement using a single camera, multiple projectors, and a computer, how to ensure that the structured light response obtained from the pre-scan corresponds to the fringe spatial frequency used in the formal measurement, how to select projection resources and encoding parameters based on the actual structured light response of the target surface under different projection directions, and how to use the observability confidence in the same associated mapping to organize phase unfolding, multi-projection 3D data fusion, and compensation for residual unobservable regions, so as to reduce redundant acquisition and repeated processing in the formal measurement stage, reuse the aforementioned results in subsequent measurements where the target object and its measurement posture remain unchanged, and improve the integrity of 3D data of complex surfaces.

[0007] To address the aforementioned technical problems, according to a first aspect of the present invention, the present invention first provides a structured light rapid three-dimensional imaging and measurement device, comprising a camera, multiple projectors, and a computer device. The multiple projectors are respectively disposed at different positions of the camera, and the projection optical axes of the multiple projectors have different directions relative to the observation optical axis of the camera. The projection areas of the multiple projectors at least partially overlap with the imaging field of view of the camera. The camera and the multiple projectors are all communicatively connected to the computer device. The computer device includes the following modules: A calibration module is used to calibrate the spatial parameters of the camera and the multiple projectors. An observability evaluation module is used to control the multiple projectors to sequentially project a pre-scanned structured light sequence onto the target object, and to control the camera to synchronously acquire pre-scanned image sequences corresponding to each projection direction, dividing the target surface corresponding to the pre-scanned image sequence into multiple target surface regions. The pre-scanned structured light sequence includes a first phase-shifted fringe sub-sequence, a first Gray code sub-sequence corresponding to the first phase-shifted fringe sub-sequence, and a second phase-shifted fringe sub-sequence. The fringe spatial frequency of the first phase-shifted fringe sub-sequence is lower than that of the second phase-shifted fringe sub-sequence, and the fringe spatial frequency of the second phase-shifted fringe sub-sequence is the same as that of the phase-shifted fringe spatial frequency in the structured light coding sequence used for formal measurement. Based on the fringe modulation and grayscale response stability of each target surface region under each projection direction determined by the second phase-shifted fringe sub-sequence, and the directional consistency determined by the first phase-shifted fringe sub-sequence, the first Gray code sub-sequence, and the spatial parameter calibration results, the corresponding observability confidence is determined, and a direction-related spatial observability distribution is formed from the observability confidence of each target surface region under each projection direction. The dynamic scheduling and acquisition module is used to establish an association mapping based on the direction-related spatial observability distribution. This association mapping uses the target surface region number and projection direction number as a joint index to record the corresponding fringe modulation, grayscale response stability, direction consistency, observability confidence, effective structured light response pixel information, and projection coverage information. It also sets fields for projection direction selection results, low observability region pixel proportion, and encoding configuration results. The effective structured light response pixels are determined based on whether the pixel fringe modulation determined by the second phase-shifted fringe subsequence meets a preset modulation condition, and whether the DC component fitted by the second phase-shifted fringe subsequence meets a preset unsaturation condition. For a non-empty candidate projection direction set, the module determines the effective structured light response pixel coverage rate of the candidate projection direction set in each target surface region, and the coverage rate of each target surface region. The maximum observability confidence of the surface region in the candidate projection direction set and the number of projection directions in the candidate projection direction set are used to weight the effective structured light response pixel coverage and the maximum observability confidence according to the proportion of the number of pixels in each target surface region to the total number of pixels in all target surface regions. This weights the weighted effective structured light response pixel coverage and the maximum observability confidence, and constructs a projection direction selection index that increases with the weighted effective structured light response pixel coverage and the maximum observability confidence, and decreases with the proportion of the number of projection directions to the total number of projection directions. The candidate projection direction set that meets the measurement coverage constraint that the weighted effective structured light response pixel coverage is not lower than a preset coverage threshold and has the largest projection direction selection index is determined as the projection direction combination. The projection direction selection result, which characterizes whether each projection direction belongs to the projection direction combination, is written into the projection direction selection result field.Configure corresponding structured light coding parameters according to the association mapping and the observability confidence distribution corresponding to each projection direction in the projection direction combination, and write the configuration result of the structured light coding parameters into the coding configuration result field; control the projector corresponding to the projection direction combination to sequentially project their respective structured light coding sequences, and control the camera to synchronously acquire the corresponding structured light images; the pre-scan, the determination of the projection direction combination, and the configuration of the structured light coding parameters are performed in the first measurement cycle after the target object enters the measurement space or the measurement posture of the target object changes; in subsequent measurement cycles where the target object and its measurement posture remain unchanged, reuse the direction-related spatial observability distribution, the association mapping, the projection direction combination, and the configuration result of the structured light coding parameters obtained in the first measurement cycle, and control the corresponding projector and the camera to complete the subsequent formal measurement according to the reuse result; The phase decoding and reconstruction module is used to calculate the wrapping phase based on the structured light image corresponding to each projection direction in the projection direction combination, determine the phase unfolding starting region and phase unfolding path of the projection direction based on the observability confidence of each target surface region under the same projection direction, so as to obtain the absolute phase of the projection direction; reconstruct the effective three-dimensional data under the corresponding projection direction based on the absolute phase of each projection direction in the projection direction combination and the result of the spatial parameter calibration, and fuse the effective three-dimensional data according to the corresponding observability confidence to obtain the initial three-dimensional morphology; The compensation and fusion module is used to identify unobservable regions as connected regions formed by camera pixels that have not obtained effective 3D points in the initial 3D topography; for unobservable regions at their boundaries where effective absolute phase data in at least two different projection directions can be obtained, the module uses the neighboring 3D point cloud, surface geometric continuity constraints, and multi-projection phase derived boundary constraints formed by selecting and fusing the effective absolute phase data according to the corresponding observability confidence recorded by the associated mapping, combined with the spatial parameter calibration results, to solve the B-spline surface compensation model to generate a compensation point cloud; for unobservable regions at their boundaries where effective absolute phase data in at least two different projection directions cannot be obtained, they are marked as regions to be retested; the module fuses the initial 3D topography with the compensation point cloud to obtain the 3D topography of the target object.

[0008] Further, the observability evaluation module is used to: determine the pre-scan absolute phase of each projection direction based on the first phase-shifted fringe sub-sequence and the first Gray code sub-sequence; obtain effective pre-scan three-dimensional data based on the pre-scan absolute phase and the spatial parameter calibration results; and determine the local surface unit normal vector of each target surface region based on the effective pre-scan three-dimensional data; perform phase-shifted fringe grayscale model fitting on the second phase-shifted fringe sub-sequence corresponding to each projection direction; determine the fringe modulation degree of each target surface region under each projection direction based on the ratio of the fitted modulation amplitude to the DC component; and normalize the fringe modulation degree to obtain the normalized fringe modulation degree; use the normalized residual between the actual grayscale and the fitted grayscale as the grayscale response fluctuation; and determine the normalized grayscale response stability based on the grayscale response fluctuation, wherein the smaller the grayscale response fluctuation, the greater the normalized grayscale response stability. The orientation consistency index is determined by the product of a first non-negative direction cosine and a second non-negative direction cosine. The first non-negative direction cosine is the larger of the sum of the dot product of a first unit vector pointing from the target surface region to the corresponding projector and the local surface unit normal vector of the target surface region, and zero. The second non-negative direction cosine is the larger of the sum of the dot product of a second unit vector pointing from the target surface region to the camera and the local surface unit normal vector, and zero. The direction of the local surface unit normal vector is chosen such that its dot product with the second unit vector is not less than zero. Preset positive weighting coefficients are used to weight the normalized fringe modulation, the normalized grayscale response stability, and the orientation consistency index, respectively. An observability confidence level for each target surface region under each projection direction is obtained through a monotonically increasing normalization function. For target surface regions whose local surface unit normal vectors cannot be determined from the effective pre-scanned 3D data, the corresponding orientation consistency index is set to zero.

[0009] Further, the dynamic scheduling and acquisition module is used to: when the same pixel is covered by multiple projection directions from the same set of candidate projection directions, count only once when determining the effective structured light response pixel coverage rate; for each projection direction in the projection direction combination, determine the pixel proportion of the low observability region within the projection coverage range of the projector corresponding to that projection direction on the target surface, write the pixel proportion into the low observability region pixel proportion field, configure a low redundancy coding sequence when the pixel proportion is lower than a preset proportion threshold, and configure an enhanced coding sequence when the pixel proportion is not lower than the preset proportion threshold; the structured light coding parameters include at least one of phase shift steps and Gray code bits, the low redundancy coding sequence and the enhanced coding sequence are both used to cooperate with the phase unfolding processing to determine the absolute phase of the target surface region that meets the phase unfolding condition in their respective effective coverage areas, at least one structured light coding parameter of the low redundancy coding sequence is less than the same type of structured light coding parameter of the enhanced coding sequence, and the remaining structured light coding parameters are not greater than the same type of structured light coding parameter of the enhanced coding sequence; the low observability region is the target surface region where the observability confidence under that projection direction is lower than a preset confidence threshold.

[0010] Further, the phase decoding and reconstruction module is used to: for each projection direction in the projection direction combination, establish a regional adjacency relationship based on the spatial adjacency relationship between the target surface regions; determine the adjacency relationship between two adjacent target surface regions where the proportion of corresponding pixel pairs with effective wrapping phases on both sides of the shared boundary is not less than a preset effective proportion threshold, and the median of the absolute value of the wrapping phase difference obtained after equivalent conversion of the corresponding pixel pairs with effective wrapping phases on both sides according to a complete phase period is not greater than a preset boundary phase threshold; determine the target surface regions with observability confidence not less than a preset unfolding threshold under the projection direction as the phase unfolding starting region set; and determine the integer number of wrapping phase cycles in the phase unfolding starting region set according to the decoding result of the structured light coding sequence corresponding to the projection direction, so as to obtain the absolute phase of the phase unfolding starting region set; After obtaining the absolute phase of the set of phase unfolding starting regions, the following process is repeated until there are no un-unfolded target surface regions that form a propagable adjacency with the target surface regions that have completed phase unfolding: From the un-unfolded target surface regions that form a propagable adjacency with the target surface regions that have completed phase unfolding, the target surface region with the highest observability confidence under the projection direction is selected as the current target surface region; From the target surface regions that form a propagable adjacency with the current target surface region and have completed phase unfolding, the target surface region with the highest observability confidence under the projection direction is selected as the reference region; Based on the absolute phase continuity at the shared boundary between the reference region and the current target surface region, the integer number of phase unfolding cycles of the current target surface region is determined, and the wrapping phase within the current target surface region is completed according to the integer number of phase unfolding cycles. The phase unfolding of the current target surface region is completed by compensating for integer multiples of the phase period. For cases where only one projection direction provides valid 3D data for the same camera pixel, this valid 3D data is retained as a valid 3D point in the initial 3D topography. For cases where multiple projection directions provide valid 3D data for the same camera pixel, when the sum of the observability confidence scores for the target surface region corresponding to the multiple projection directions is greater than zero, the observability confidence scores for each projection direction are divided by the sum of the observability confidence scores to obtain corresponding normalized fusion weights. These normalized fusion weights are then used to weight and fuse the valid 3D data corresponding to the same camera pixel. When the sum of the observability confidence scores is zero, the valid 3D data corresponding to the camera pixel is not written into the initial 3D topography, and the camera pixel is marked as a pixel that has not obtained a valid 3D point.

[0011] Further, the compensation and fusion module is used to: extract effective 3D point clouds adjacent to the unobservable region, and determine the local surface unit normal vector and the neighborhood average absolute principal curvature based on the effective 3D point cloud, wherein the neighborhood average absolute principal curvature is the average of the absolute values ​​of the two principal curvatures at each 3D point in the effective 3D point cloud; determine the number of region pixels of the unobservable region in the imaging plane of the camera, and use a first compensation parameter group when the number of region pixels is less than a preset pixel number threshold and the neighborhood average absolute principal curvature is less than a preset curvature threshold, otherwise use a second compensation parameter group, wherein at least one of the B-spline surface number and the number of iterations in the second compensation parameter group is greater than the same type of parameter in the first compensation parameter group, and the remaining same type of parameter is not less than the same type of parameter in the first compensation parameter group; form the surface geometric continuity constraint based on the spatial distance between the 3D points in the effective 3D point cloud and the point to be compensated, and the consistency between the local surface unit normal vector and the estimated normal vector of the point to be compensated. For the unobservable region, the effective absolute phase data of different projection directions obtainable at its boundary are statistically analyzed. When the effective formally measured absolute phase data in the projection direction combination comes from at least two different projection directions, the effective formally measured absolute phase data is directly used. When the number of different projection directions corresponding to the effective formally measured absolute phase data is less than two, the effective pre-scan absolute phase data of the remaining projection directions are supplemented according to the observability confidence of the associated mapping record from high to low. After completing the above selection and necessary supplementation, when effective absolute phase data of at least two different projection directions have been obtained, each effective absolute phase data is converted into a three-dimensional boundary point in the camera coordinate system according to the spatial parameter calibration result. When the observability confidence of each three-dimensional boundary point participating in the fusion is... When the sum of the observability confidence scores is greater than zero, each observability confidence score is divided by the sum of the observability confidence scores to obtain the corresponding normalized weights, and the three-dimensional boundary points are weighted and fused according to the normalized weights; when the sum of the observability confidence scores is zero, the three-dimensional boundary points are fused with equal weights; the fused three-dimensional boundary points are used to form the multi-projection phase derived boundary constraints; after completing the above selection and necessary supplementation, if at least two different projection directions of effective absolute phase data cannot be obtained, the unobservable region is marked as a region to be retested, and no compensation point cloud is generated for the unobservable region; for unobservable regions where at least two different projection directions of effective absolute phase data have been obtained, under the surface geometric continuity constraints and the multi-projection phase derived boundary constraints, the B-spline surface compensation model is solved according to the first compensation parameter set or the second compensation parameter set to generate the compensation point cloud.

[0012] According to a second aspect of the present invention, the present invention also provides a measurement method for the above-mentioned measuring device, comprising: S1: calibrating the spatial parameters of the camera and the plurality of projectors; S2: Control the multiple projectors to sequentially project a pre-scanned structured light sequence onto the target object, and the camera synchronously acquires pre-scanned image sequences corresponding to each projection direction; the pre-scanned structured light sequence includes a first phase-shifted fringe sub-sequence, a first Gray code sub-sequence corresponding to the first phase-shifted fringe sub-sequence, and a second phase-shifted fringe sub-sequence, wherein the fringe spatial frequency of the first phase-shifted fringe sub-sequence is lower than that of the second phase-shifted fringe sub-sequence, and the fringe spatial frequency of the second phase-shifted fringe sub-sequence is the same as that of the phase-shifted fringe spatial frequency in the structured light coding sequence used for formal measurement; divide the target surface corresponding to the pre-scanned image sequence into multiple target surface regions, and determine the corresponding observability confidence based on the fringe modulation and grayscale response stability of each target surface region under each projection direction determined by the second phase-shifted fringe sub-sequence, as well as the directional consistency determined by the first phase-shifted fringe sub-sequence, the first Gray code sub-sequence, and the spatial parameter calibration results, and form a direction-related spatial observability distribution based on the observability confidence of each target surface region under each projection direction; S3: Establish an association mapping based on the direction-related spatial observability distribution. The association mapping uses the target surface region number and the projection direction number as a joint index to record the corresponding fringe modulation degree, grayscale response stability, direction consistency, observability confidence, effective structured light response pixel information, and projection coverage information. It also sets a projection direction selection result field, a low observability region pixel ratio field, and an encoding configuration result field. The effective structured light response pixels are determined based on whether the pixel fringe modulation degree determined by the second phase-shifted fringe subsequence meets a preset modulation degree condition, and whether the DC component fitted by the second phase-shifted fringe subsequence meets a preset unsaturation condition. S4: For a non-empty candidate projection direction set, determine the effective structured light response pixel coverage rate of the candidate projection direction set in each target surface region, the maximum observability confidence of each target surface region in the candidate projection direction set, and the number of projection directions in the candidate projection direction set. Based on the proportion of the number of pixels in each target surface region to the total number of pixels in all target surface regions, weight the effective structured light response pixel coverage rate and the maximum observability confidence, and construct a projection direction selection index that increases with the increase of the weighted effective structured light response pixel coverage rate and the maximum observability confidence, and decreases with the increase of the proportion of the number of projection directions to the total number of projection directions.The set of candidate projection directions that meet the measurement coverage constraint that the weighted effective structured light response pixel coverage rate is not lower than a preset coverage threshold and has the largest projection direction selection index is determined as the projection direction combination, and the projection direction selection result that characterizes whether each projection direction belongs to the projection direction combination is written into the projection direction selection result field; S5: Configure the corresponding structured light coding parameters according to the association mapping and the observability confidence distribution of each projection direction in the projection direction combination, write the low observability region pixel ratio of each projection direction into the low observability region pixel ratio field, and write the configuration result of the structured light coding parameters into the coding configuration result field; control the projector corresponding to the projection direction combination to project their respective structured light coding sequences in sequence, and the camera synchronously acquires the corresponding structured light images; The structured light coding parameters in steps S2, S3, S4, and S5 are configured to be executed in the first measurement cycle after the target object enters the measurement space or the measurement posture of the target object changes. In subsequent measurement cycles where the target object and its measurement posture remain unchanged, the configuration results of the orientation-related spatial observability distribution, the association mapping, the projection direction combination, and the structured light coding parameters obtained in the first measurement cycle are reused, and the structured light projection and image acquisition in step S5 are directly executed according to the reuse results. S6: The wrapping phase is calculated according to the structured light image corresponding to each projection direction in the projection direction combination. The phase unfolding start region and phase unfolding path of the projection direction are determined according to the observability confidence of each target surface region under the same projection direction to obtain the absolute phase of the projection direction. The effective three-dimensional data under the corresponding projection direction is reconstructed according to the absolute phase of each projection direction in the projection direction combination and the spatial parameter calibration results. The effective three-dimensional data is fused according to the corresponding observability confidence to obtain the initial three-dimensional morphology. S7: The connected regions formed by camera pixels that have not obtained effective 3D points in the initial 3D topography are identified as unobservable regions; for unobservable regions at the boundary where effective absolute phase data in at least two different projection directions can be obtained, the boundary constraints are derived by deriving the multi-projection phase based on the neighboring 3D point cloud, surface geometric continuity constraints, and the corresponding observability confidence of the effective absolute phase data according to the associated mapping record, combined with the spatial parameter calibration results, and solving the B-spline surface compensation model to generate a compensation point cloud; for unobservable regions at the boundary where effective absolute phase data in at least two different projection directions cannot be obtained, they are marked as regions to be retested; S8: The initial 3D topography and the compensation point cloud are fused to obtain the 3D topography of the target object.

[0013] Further, step S2 includes: The pre-scan absolute phase of each projection direction is determined based on the first phase shift fringe subsequence and the first Gray code subsequence. Effective pre-scan three-dimensional data is obtained based on the pre-scan absolute phase and the spatial parameter calibration results. The local surface unit normal vector of each target surface region is determined based on the effective pre-scan three-dimensional data. Determine the first according to formula (1) The target surface region in the first Observability confidence under each projection direction : Equation (1) The first is determined according to formula (2). The target surface region is in the first Orientation consistency index under each projection direction : Equation (2) in, Represents a monotonically increasing normalized function. This means taking the maximum value among them. Represents the dot product of two vectors; The number of the target surface regions. The number of all projection directions. , ; For the first The target surface region is in the first The normalized fringe modulation degree under the projection direction, the fringe modulation degree being based on the normalized fringe modulation degree of the first projection direction. The ratio of the modulation amplitude to the DC component is determined by fitting the phase-shifted fringe grayscale model to the second phase-shifted fringe subsequence corresponding to each projection direction. For the first The target surface region is in the first The normalized grayscale response stability under each projection direction is determined based on the normalized residual between the actual grayscale and the fitted grayscale of the second phase-shifted fringe subsequence. A smaller normalized residual indicates a more stable response. The larger; For the first The target surface region points to the first The unit vector of the projector corresponding to each projection direction. For the first A unit vector pointing from the target surface region to the camera. The local surface unit normal vector is determined from the effective pre-scan three-dimensional data obtained based on the pre-scan absolute phase determined by the first phase-shifting fringe subsequence and the first Gray code subsequence, as well as the spatial parameter calibration results. The direction is taken to make The direction; for target surface regions where the local surface unit normal vector cannot be determined from the effective pre-scanned 3D data, the corresponding Set to zero; , , The preset positive weighting coefficients are given, and the sum of the three is 1. Step S4 includes: determining the set of non-empty candidate projection directions according to equation (3). Projection direction selection index : Equation (3) In equation (3), the summation symbol represents the summation of the sums ... From 1 to Sum of corresponding terms, Indicates that within the set of candidate projection directions Take from each projection direction The maximum value; It is the set of candidate projection directions consisting of at least one projection direction from all projection directions. For the first The proportion of pixels in one target surface region to the total number of pixels in all target surface regions. The set of candidate projection directions In the first Effective structured light response pixel coverage in each target surface region The set of candidate projection directions The number of projection directions in the image. , , The preset positive weighting coefficients are used; the effective structured light response pixels are pixels whose pixel fringe modulation degree, determined according to the second phase-shifted fringe sub-sequence, is not lower than a preset modulation degree threshold and whose fitted DC component is within a preset unsaturated range. The same pixel is represented by the candidate projection direction set. When multiple projection directions are covered, only one count is made; The candidate projection direction set is determined according to equation (4). Does the measurement coverage constraint meet? Equation (4) The projection direction selection index satisfies equation (4) and Largest set of candidate projection directions Determined as the combination of projection directions ,in, The preset coverage threshold is defined as follows.

[0014] Further, step S5 includes: For each projection direction in the projection direction combination, the target surface area with an observability confidence level not lower than a preset confidence threshold under that projection direction is determined as a high observability area, and the target surface area with an observability confidence level lower than the preset confidence threshold under that projection direction is determined as a low observability area; the pixel ratio of the low observability area within the projection coverage range of the projector corresponding to that projection direction on the target surface is determined, and the pixel ratio is written into the low observability area pixel ratio field; when the pixel ratio is lower than a preset ratio threshold, a low redundancy coding sequence is configured, and when the pixel ratio is not lower than the preset ratio threshold, an enhanced coding sequence is configured; the structured light coding parameters include at least one of phase shift steps and Gray code bits; both the low redundancy coding sequence and the enhanced coding sequence are used to cooperate with the phase unfolding processing in step S6 to determine the absolute phase of the target surface area that meets the phase unfolding condition in their respective effective coverage areas; at least one structured light coding parameter of the low redundancy coding sequence is less than the same type of structured light coding parameter of the enhanced coding sequence, and the remaining structured light coding parameters are not greater than the same type of structured light coding parameter of the enhanced coding sequence.

[0015] Further, step S6 includes: For each projection direction in the projection direction combination, a regional adjacency relationship is established based on the spatial adjacency relationship between the target surface regions. The adjacency relationship between two adjacent target surface regions is determined as a propagable adjacency relationship, where the proportion of corresponding pixel pairs with effective wrapping phases on both sides of the shared boundary to all corresponding pixel pairs at the shared boundary is not less than a preset effective proportion threshold, and the median of the absolute value of the wrapping phase difference of the corresponding pixel pairs with effective wrapping phases on both sides after equivalent conversion according to a complete phase period is not greater than a preset boundary phase threshold. The target surface regions with observability confidence not less than a preset unfolding threshold under the projection direction are determined as the phase unfolding starting region set. The integer number of wrapping phase cycles in the phase unfolding starting region set is determined based on the decoding result of the structured light coding sequence corresponding to the projection direction, so as to obtain the absolute phase of the phase unfolding starting region set. After obtaining the absolute phase of the set of phase unfolding starting regions, the following process is repeated until there are no un-unfolded target surface regions that form a propagable adjacency with the target surface regions that have completed phase unfolding: from the un-unfolded target surface regions that form a propagable adjacency with the target surface regions that have completed phase unfolding, the target surface region with the highest observability confidence under the projection direction is selected as the current target surface region; from the target surface regions that form a propagable adjacency with the current target surface region and have completed phase unfolding, the target surface region with the highest observability confidence under the projection direction is selected as the reference region; based on the absolute phase continuity at the shared boundary between the reference region and the current target surface region, the integer number of phase unfolding cycles of the current target surface region is determined, and the wrapped phase in the current target surface region is compensated for an integer multiple of the complete phase cycle according to the integer number of phase unfolding cycles to complete the phase unfolding of the current target surface region; For a single camera pixel with only one valid 3D data point in a single projection direction, this valid 3D data point is retained as a valid 3D point in the initial 3D topography. For a single camera pixel with valid 3D data points from multiple projection directions, when the sum of the observability confidence scores for the target surface region where the camera pixel is located for the multiple projection directions is greater than zero, the observability confidence scores for each projection direction are divided by the sum of the observability confidence scores to obtain corresponding normalized fusion weights. These normalized fusion weights are then used to perform weighted fusion on each valid 3D data point corresponding to the same camera pixel. When the sum of the observability confidence scores is zero, the valid 3D data points corresponding to the camera pixel are not written into the initial 3D topography, and the camera pixel is marked as a pixel that has not obtained a valid 3D point.

[0016] Further, step S7 includes: Extract the effective 3D point cloud adjacent to the unobservable region, and determine the local surface unit normal vector and the neighborhood average absolute principal curvature based on the effective 3D point cloud. The neighborhood average absolute principal curvature is the average of the absolute values ​​of the two principal curvatures at each 3D point in the effective 3D point cloud. Determine the number of region pixels of the unobservable region in the imaging plane of the camera. If the number of region pixels is less than a preset pixel number threshold and the neighborhood average absolute principal curvature is less than a preset curvature threshold, a first compensation parameter group is used; otherwise, a second compensation parameter group is used. At least one of the B-spline surface number and the number of iterations in the second compensation parameter group is greater than the same type of parameter in the first compensation parameter group, and the remaining same type of parameters are not less than the same type of parameter in the first compensation parameter group. Form the surface geometric continuity constraint based on the spatial distance between the 3D points in the effective 3D point cloud and the point to be compensated, and the consistency between the local surface unit normal vector and the estimated normal vector of the point to be compensated. For the unobservable region, the effective absolute phase data of different projection directions obtainable at its boundary are statistically analyzed. When the effective formally measured absolute phase data in the projection direction combination comes from at least two different projection directions, the effective formally measured absolute phase data is directly used. When the number of different projection directions corresponding to the effective formally measured absolute phase data is less than two, the effective pre-scan absolute phase data of the remaining projection directions are supplemented according to the observability confidence of the associated mapping record from high to low. After completing the above selection and necessary supplementation, when effective absolute phase data of at least two different projection directions have been obtained, the results of the spatial parameter calibration are used. Each obtained effective absolute phase data is converted into a 3D boundary point in the camera coordinate system. When the sum of the observability confidence scores of the 3D boundary points involved in the fusion is greater than zero, each observability confidence score is divided by the sum of the observability confidence scores to obtain a corresponding normalized weight, and the 3D boundary points are weighted and fused according to the normalized weight. When the sum of the observability confidence scores is zero, the 3D boundary points are fused with equal weight. The fused 3D boundary points are used to form the multi-projection phase derived boundary constraint. If, after completing the above selection and necessary supplementation, effective absolute phase data of at least two different projection directions cannot be obtained, the unobservable region is marked as a region to be retested, and no compensation point cloud is generated for the unobservable region. For unobservable regions where effective absolute phase data of at least two different projection directions have been obtained, the B-spline surface compensation model is solved according to the first compensation parameter set or the second compensation parameter set under the surface geometric continuity constraint and the multi-projection phase derived boundary constraint to generate the compensation point cloud.

[0017] Compared with the prior art, the beneficial effects of the present invention are reflected in: 1. The pre-scanning structured light sequence of the present invention includes a low-frequency geometric sub-sequence for obtaining the pre-scanning absolute phase and local surface geometric relationship, and an evaluation phase shift sub-sequence with the same fringe spatial frequency as the formal measurement; the fringe modulation and grayscale response stability are determined by the same-frequency evaluation phase shift sub-sequence, and the observability confidence is formed by combining the directional consistency, which can avoid directly inferring the high-frequency formal measurement response from the low-frequency fringe response, and make the directional correlation spatial observability distribution closer to the formal measurement conditions.

[0018] 2. This invention establishes an association mapping using the target surface area and projection direction as a joint index. Under the premise of meeting the measurement coverage requirements, it selects projection direction combinations with good coverage, high observability, and a small number of combinations, and configures the encoding parameters according to the pixel ratio of low observability areas in each direction. In subsequent measurements where the target object and its measurement posture remain unchanged, the orientation-related spatial observability distribution, association mapping, projection direction combination, and encoding configuration results obtained in the first measurement cycle are reused, which can reduce the amount of repeated projection, image acquisition, and scheduling processing in subsequent measurements.

[0019] 3. This invention determines the phase unrolling starting region, propagation order, and multi-projection 3D data fusion weights by using the observability confidence of the associated mapping record. This can reduce periodic misjudgment and error propagation caused by low-quality phase data, and reduce the impact of low-confidence 3D data on the fusion results.

[0020] 4. For unobservable regions that still lack effective 3D data after measurement in the selected projection direction, this invention prioritizes formal measurement of the absolute phase, and supplements the pre-scanned absolute phase according to the observability confidence of the associated mapping record when there are fewer than two projection directions. When at least two effective absolute phases can be obtained, the B-spline surface compensation model is solved by combining surface geometric continuity constraints and multi-projection phase derived boundary constraints. When there are still fewer than two directions, the corresponding region is marked as a region to be remeasured, thereby improving the continuity of the compensation boundary and avoiding the generation of compensation point clouds with unclear confidence when constraints are insufficient. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the hardware architecture and multi-projection direction optical path of the structured light rapid three-dimensional imaging and measurement device in an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the connection relationship of the functional modules and the data flow of the computer device in an embodiment of the present invention; Figure 3 This is an overall flowchart of the structured light fast three-dimensional imaging and measurement method in an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the evaluation and distribution formation process of directional spatial observability in an embodiment of the present invention. Figure 5 This is a schematic diagram of the projection direction combination selection and structured light coding parameter adaptive configuration process in an embodiment of the present invention; Figure 6 This is a schematic diagram of the phase unrolling starting region, phase unrolling path, and non-propagating boundary based on observability confidence in an embodiment of the present invention. Figure 7 This is a schematic diagram of the effective 3D data reconstruction and confidence-weighted fusion process in multiple projection directions in an embodiment of the present invention; Figure 8 This is a schematic diagram illustrating the selection of compensation parameters, constraint generation, and compensation point cloud fusion process for unobservable regions in an embodiment of the present invention.

[0022] The following are the reference numerals in the attached figures: 10-Camera; 20-Projector; 21-First Projector; 22-Second Projector; 23-Third Projector; 30-Computer Equipment; 31-Processor; 32-Memory; 40-Target Object; 41-Target Surface; 42-Target Surface Region; 50-Calibration Board; 100-Calibration Module; 101-Spatial Parameter Calibration Results; 200-Observability Evaluation Module; 201-Pre-scan Image Sequence; 202-Observability Confidence; 203-Directional Correlation Spatial Observability Distribution; 300-Dynamic Scheduling and Acquisition Module; 301-Association Mapping; 302-Projection Direction Combination; 303-Low Redundancy Coding Sequence; 304-Enhanced Coding Sequence; 305 - Structured light image; 400 - Phase decoding and reconstruction module; 401 - Wrapped phase; 402 - Phase unfolding starting region; 403 - Phase unfolding path; 404 - Absolute phase; 405 - Effective 3D data; 406 - Initial 3D shape; 407 - Propagable adjacency relationship; 408 - Non-propagable boundary; 500 - Compensation and fusion module; 501 - Unobservable region; 502 - Neighborhood effective 3D point cloud; 503 - Surface geometric continuity constraint; 504 - Multi-projection phase derived boundary constraint; 505 - Compensated point cloud; 506 - Target object 3D shape; 507 - First compensation parameter group; 508 - Second compensation parameter group. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the following is combined with... Figures 1 to 8 The present invention will be further described in detail below. Embodiments 1 and 2 illustrate the same inventive concept from the perspectives of device structure and method flow, respectively. Experimental Example 1 is used to verify the technical effects of the described device and method. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0024] This specific implementation uses a fixed set of device parameters, thresholds, and processing procedures. The processing branches involving low-redundancy coding sequences and enhanced coding sequences, as well as the first compensation parameter group and the second compensation parameter group, are all uniquely determined by preset thresholds and input data, and are not selected by the operator.

[0025] Example 1: Structured light rapid 3D imaging and measurement device: I. Device Hardware Structure like Figure 1 As shown, this embodiment provides a structured light rapid three-dimensional imaging and measurement device, including a camera 10, a projector 20 and a computer device 30.

[0026] The projector 20 includes a first projector 21, a second projector 22 and a third projector 23.

[0027] Camera 10 is a global shutter monochrome industrial camera with a resolution of 2448×2048 pixels, a quantization bit depth of 12 bits, and a maximum acquisition frequency of 120 frames per second, and is equipped with a fixed-focus lens with a focal length of 16 mm.

[0028] The first projector 21, the second projector 22, and the third projector 23 all use digital light processing projectors with a resolution of 1140×912 pixels and a grayscale pattern output frequency of 120 Hz. Each projector uses an effective projection area of ​​1024×768 pixels.

[0029] The computer device 30 includes an eight-core 3.2 GHz processor 31, 32 GB of memory 32, and 8 GB of graphics processing memory.

[0030] A camera coordinate system is established with the optical center of camera 10 as the origin and the camera's observation optical axis as the Z-axis. The center coordinates of the target measurement area are (0 mm, 0 mm, 800 mm). The optical center coordinates of the first projector 21, the second projector 22, and the third projector 23 are (-350 mm, 0 mm, 0 mm), (350 mm, 0 mm, 0 mm), and (0 mm, -300 mm, 0 mm), respectively, and the projection optical axes of the three projectors all point to the center of the target measurement area.

[0031] The projection areas of the three projectors and the imaging field of view of the camera 10 together cover a measurement space of 300 mm × 250 mm × 150 mm.

[0032] Camera 10 and three projectors are connected to computer device 30 via a hardware trigger controller. The three projectors operate sequentially in the order of first projector 21, second projector 22, and third projector 23, with only one projector outputting structured light at a time. The camera exposure time is fixed at 3.0 ms, the analog gain is fixed at 0 dB, and the camera acquisition time is synchronized with the stable output range of the projected pattern.

[0033] II. Functional Modules of Computer Equipment like Figure 2 As shown, the computer device 30 executes program instructions in the memory 32 through the processor 31 to realize the calibration module 100, the observability evaluation module 200, the dynamic scheduling and acquisition module 300, the phase decoding and reconstruction module 400, and the compensation and fusion module 500.

[0034] 1. Calibration Module The calibration module 100 is used to calibrate the spatial parameters of the camera 10 and the three projectors, and outputs the spatial parameter calibration result 101.

[0035] The calibration plate 50 is a planar ceramic checkerboard calibration plate with 12×9 interior corner points and a center-to-center distance of 10.000 mm between adjacent corner points. The calibration module 100 controls the camera 10 to acquire checkerboard images of the calibration plate 50 in 20 fixed postures. Based on the pinhole imaging model and the distortion model containing three radial distortion parameters and two tangential distortion parameters, the intrinsic parameters, distortion parameters, and calibration plate posture parameters of the camera 10 are determined.

[0036] Under each calibration plate orientation, the calibration module 100 controls three projectors to project five-step phase-shift fringes and four-bit Gray code in sequence, using each projector as a reverse camera to determine the intrinsic parameters, distortion parameters, and extrinsic parameters of each projector relative to the camera coordinate system.

[0037] The spatial parameter calibration result 101 includes the intrinsic parameters and distortion parameters of the camera 10 and the three projectors, the extrinsic parameters of the projectors relative to the camera coordinate system, and the transformation parameters between the absolute phase and the effective projection column coordinates of the projectors.

[0038] 2. Observability Evaluation Module The observability evaluation module 200 is used to control three projectors to project a pre-scanned structured light sequence onto the target object 40 in sequence, and to control the camera 10 to simultaneously acquire a pre-scanned image sequence 201.

[0039] The pre-scanning structured light sequence projected by each projector consists of a low-frequency geometric subsequence and a co-frequency evaluation phase-shift subsequence. The low-frequency geometric subsequence includes five first phase-shift fringes and four first Gray codes: the period of the first phase-shift fringes is 64 projector pixels, forming 16 complete periods within a 1024-pixel effective projection width, with each of the 16 periods identified by a four-bit first Gray code. The co-frequency evaluation phase-shift subsequence includes five second phase-shift fringes, each with a period of 16 projector pixels, identical to the phase-shift fringes period in the low-redundancy coding sequence and enhancement coding sequence used in the formal measurement. The pre-scanning structured light sequence for each projection direction comprises 14 images.

[0040] The observability evaluation module 200 divides the target object mask into target surface regions 42 in 16×16 pixels within the camera imaging plane.

[0041] For each target surface region, the observability evaluation module 200 processes the pre-scanned images corresponding to the three projection directions and performs the following fixed processing: The pre-scan absolute phase is determined based on the first phase shift fringe and the first Gray code; The pre-scan three-dimensional data is reconstructed based on the pre-scan absolute phase and spatial parameter calibration results 101, and the local surface unit normal vector is determined based on the pre-scan three-dimensional data. Five second phase-shifted fringe images were fitted with a phase-shifted fringe grayscale model, and the fringe modulation degree was determined based on the ratio of the fitted modulation amplitude to the DC component. The stability of grayscale response is determined based on the normalized residual between the actual grayscale and the fitted grayscale of the second phase-shifted fringe image. The orientation consistency is determined based on the projection direction, the camera observation direction, and the unit normal vector of the local surface. By fusing the normalized fringe modulation, normalized grayscale response stability, and directional consistency determined by the second phase-shift fringes with fixed weights of 0.45, 0.25, and 0.30 respectively, an observability confidence level of 202 is obtained.

[0042] For target surface regions with fewer than 64 valid pre-scanned 3D points and where the local surface unit normal vector cannot be determined, the directional consistency of the corresponding region is set to zero.

[0043] The observability evaluation module 200 forms a direction-dependent spatial observability distribution 203 based on the observability confidence 202 of all target surface areas in three projection directions.

[0044] 3. Dynamic Scheduling and Data Acquisition Module The dynamic scheduling and acquisition module 300 establishes a correlation mapping 301 between the target surface region 42 and the projection direction based on the directional spatial observability distribution 203.

[0045] The association mapping 301 includes a regional orientation evaluation record and a projection orientation configuration record. The regional orientation evaluation record uses the target surface region number and projection orientation number as a joint index to record the fringe modulation and grayscale response stability determined by the same-frequency evaluation phase shift subsequence, as well as the corresponding orientation consistency, observability confidence, effective structured light response pixel count, and projection coverage pixel count. The projection orientation configuration record uses the projection orientation number as an index and sets a projection orientation selection result field, a low observability region pixel proportion field, and an encoding configuration result field. When establishing association mapping 301, all three fields are yet to be determined and are in a pending write state.

[0046] The dynamic scheduling and acquisition module 300 denotes the three projection directions corresponding to the first projector 21, the second projector 22 and the third projector 23 as P1, P2 and P3 respectively, and enumerates the set of seven non-empty candidate projection directions composed of P1, P2 and P3.

[0047] Pixels with a fringe modulation degree of not less than 0.10 and a fitted DC component between 205 and 3890 are identified as valid structured light response pixels. Both the fringe modulation degree and the fitted DC component are determined by a co-frequency evaluation phase shifter sequence. If the same pixel is valid in more than two projection directions within a set of candidate projection directions, it is counted only once.

[0048] The dynamic scheduling and acquisition module 300 determines the projection direction combination 302 according to the following fixed relationship: The weight of the effective structured light response pixel coverage is 0.55; The maximum observability confidence weight in the candidate set is 0.35; The penalty weight for the number of projection directions is 0.10; The weighted effective structured light response pixel coverage shall not be less than 0.95.

[0049] Among the candidate sets that satisfy the coverage constraint, the set with the largest projection direction selection index is determined as projection direction combination 302. This projection direction combination 302 is then determined as the current measurement projection direction combination. .

[0050] The difference in the projection direction selection index between the two candidate sets shall not exceed When the number of projection directions is small, select the set with fewer projection directions; when the number of projection directions is the same, select the set according to... , , The numbering order determines a unique projection direction combination 302. After completing the above comparison, the result of whether each projection direction belongs to the projection direction combination 302 is written into the projection direction selection result field in the association mapping 301. When there is no set of candidate projection directions with a weighted effective structured light response pixel coverage of not less than 0.95, the projection direction is selected. , and The complete set of projection directions is determined as the backup projection direction combination, and the backup projection direction combination is determined as the current measurement projection direction combination. The result of selecting all three projection directions is written into the projection direction selection result field in the associated mapping 301, and the measurement coverage insufficient flag is output; the alternative projection direction combination is not recorded as the projection direction combination 302 that satisfies the measurement coverage constraint.

[0051] When the current measurement projection direction combination When the projection direction combination is 302, for the current measurement projection direction combination For each projection direction, the dynamic scheduling and acquisition module 300 counts the number of mask pixels contained in the target surface area with an observability confidence level below 0.60 within the projection coverage area of ​​the projector corresponding to that projection direction, and calculates the proportion of mask pixels in the target object mask area within the projection coverage area. This proportion is then written into the low observability region pixel percentage field in the association mapping 301. When this pixel percentage is less than 20%, a low redundancy coding sequence 303 is configured; when the pixel percentage is not less than 20%, an enhanced coding sequence 304 is configured. When the current measurement projection direction combination... When combining the alternative projection directions, the pixel ratio of the low observability area is not calculated, and the "undetermined" flag is written into the low observability area pixel ratio field. Enhanced coding sequence 304 is configured for all three projection directions, and the corresponding coding configuration results are written into the coding configuration result field.

[0052] The low-redundancy coding sequence 303 consists of six Gray codes and three phase-shifted fringes, for a total of nine images. The enhanced coding sequence 304 consists of six Gray codes and five phase-shifted fringes, for a total of eleven images.

[0053] The dynamic scheduling and acquisition module 300 completes the current measurement projection direction combination. After configuring the encoding parameters for each projection direction, the configuration results of the low-redundancy encoding sequence 303 or the enhanced encoding sequence 304 corresponding to each projection direction are written into the encoding configuration result field of the associated mapping 301.

[0054] The dynamic scheduling and acquisition module 300 controls the current measurement projection direction combination. The corresponding projectors output the structured light coding sequence one by one according to the number order, and control the camera 10 to synchronously acquire the structured light image 305.

[0055] 4. Phase Decoding and Reconstruction Module The phase decoding and reconstruction module 400 calculates the wrapping phase 401 of each projection direction based on the structured light image 305, and establishes a region adjacency map based on the spatial adjacency relationship between the target surface regions 42.

[0056] Two adjacent target surface regions are determined to have a propagable adjacency relationship when they meet the following conditions: 407 The shared boundary length is no less than 4 camera pixels; The proportion of pixels with effective wrapping phase on both sides of the shared boundary is no less than 80%; The wrapping phase difference of corresponding pixel pairs with effective wrapping phase on both sides of the shared boundary. The median of the absolute values ​​of the phase differences obtained after equivalent conversion is no greater than .

[0057] Adjacent regions that do not meet the above conditions form a non-propagable boundary 408.

[0058] The phase decoding and reconstruction module 400 determines the target surface area with an observability confidence of not less than 0.75 and a six-bit Gray code effective pixel ratio of not less than 80% as the phase unfolding starting region 402.

[0059] Phase unwrapping employs an observability confidence priority queue. Unwrapped regions with higher observability confidence are processed first, and the adjacent region with the highest confidence that has already undergone phase unwrapping is used as a reference region. The integer number of cycles is determined by the absolute phase continuity at the shared boundary, thereby forming the phase unwrapping path 403 and obtaining the absolute phase 404.

[0060] The phase decoding and reconstruction module 400 converts the absolute phase 404 into column coordinates within the effective projection area of ​​the projector and determines the corresponding isophase projection light plane. Three-dimensional points are obtained by calculating the intersection points of the spatial rays corresponding to camera pixels and the isophase projection light plane.

[0061] Three-dimensional points in the camera imaging plane or projector phase coordinates with a reprojection error greater than 0.50 pixels, a fringe modulation degree less than 0.10, or a DC component exceeding 205 to 3890 are discarded, and the remaining data constitute the effective three-dimensional data 405 for each projection direction.

[0062] When a valid 3D point exists for a single camera pixel in only one projection direction, the phase decoding and reconstruction module 400 retains that valid 3D point.

[0063] When a single camera pixel has valid 3D points in two or more projection directions, and the sum of the observability confidence scores corresponding to each valid projection direction is greater than zero, the phase decoding and reconstruction module 400 divides the observability confidence scores corresponding to each valid projection direction by the sum of the observability confidence scores to obtain the corresponding normalized fusion weights, and performs weighted fusion on the coordinates of the valid 3D points in each projection direction according to the normalized fusion weights to obtain the initial 3D shape 406.

[0064] When the sum of the observability confidence of each effective projection direction corresponding to the same camera pixel is zero, the effective 3D points corresponding to that camera pixel are not written into the initial 3D topography 406, and the camera pixel is marked as a pixel that has not obtained effective 3D points.

[0065] 5. Compensation and Fusion Module The compensation and fusion module 500 defines the four-connected regions within the target object mask where no effective 3D points are obtained as unobservable regions 501, and extracts an annular neighborhood with a width of 8 camera pixels outside each unobservable region to obtain the effective 3D point cloud of the neighborhood 502.

[0066] The compensation and fusion module 500 uses the 30 nearest points in the spatial distance around each neighborhood 3D point to perform principal component analysis to determine the local surface unit normal vector; then it fits a quadratic surface to determine the two principal curvatures and the neighborhood average absolute principal curvature.

[0067] The threshold for the number of pixels in the unobservable region is fixed at 400, and the threshold for the average absolute principal curvature of the neighborhood is fixed at [missing value]. .

[0068] The number of pixels in the unobservable region is less than 400 and the average absolute principal curvature of the neighborhood is less than In this case, the first compensation parameter group 507 is used. The second compensation parameter group 508 is used for the remaining unobservable areas.

[0069] The first compensation parameter set 507 includes a quadratic tensor product B-spline, an 8-pixel control point spacing, and 20 Gaussian-Newton iterations. The second compensation parameter set 508 includes a cubic tensor product B-spline, a 4-pixel control point spacing, and 80 Gaussian-Newton iterations. The first compensation parameter set 507 is suitable for unobservable regions with a small number of pixels and a low neighborhood mean absolute principal curvature. The lower B-spline surface degree, larger control point spacing, and fewer iterations are used to reduce computation while meeting the fitting requirements for low curvature regions. The second compensation parameter set 508 is suitable for unobservable regions with a large number of pixels or a high neighborhood mean absolute principal curvature. The higher B-spline surface degree, smaller control point spacing, and more iterations are used to improve the fitting freedom and convergence accuracy of the compensation surface for larger regions or higher curvature surfaces.

[0070] The compensation and fusion module 500 establishes surface geometric continuity constraints 503 based on the positions of the three-dimensional points in the effective three-dimensional point cloud 502 in the neighborhood and the local surface unit normal vector.

[0071] For each unobservable region 501, the compensation and fusion module 500 statistically analyzes the effective absolute phase data of different projection directions obtainable at its boundary. First, it starts from the combination of the current measured projection directions. The formal measurement absolute phase 404 is extracted. When the number of different projection directions corresponding to the effective formal measurement absolute phase is not less than two, the effective formal measurement absolute phase is directly adopted. When the number of different projection directions corresponding to the effective formal measurement absolute phase is less than two, the effective pre-scan absolute phases of the remaining projection directions are supplemented according to the observability confidence 202 recorded by the association mapping 301 from high to low. After completing the above selection and necessary supplementation, when at least two effective absolute phase data of different projection directions have been obtained, they are converted into three-dimensional boundary points in the camera coordinate system according to the fringe period and spatial parameter calibration results 101 corresponding to each effective absolute phase. When the sum of the observability confidence 202 corresponding to the three-dimensional boundary points participating in the fusion is greater than zero, each observability confidence 202 is divided by the sum of the observability confidence to obtain the corresponding normalized weight, and the three-dimensional boundary points are weighted and fused according to the normalized weight. When the sum of the observability confidence is zero, the three-dimensional boundary points are fused with equal weight. The fused 3D boundary points are used to form multi-projection phase derived boundary constraints 504. The above equal-weight fusion is only a degenerate treatment when all the data participating in the fusion are valid absolute phase data, but the sum of their corresponding observability confidences is zero and normalized weights cannot be calculated; invalid absolute phase data are not included in the fusion. If, after completing the above selection and necessary supplementation, valid absolute phase data in at least two different projection directions still cannot be obtained, the corresponding unobservable region 501 is marked as a region to be remeasured, and no compensation point cloud 505 is generated for this region.

[0072] The objective function of the compensation model consists of three parts: The distance term from the point to be compensated to the local tangent plane of the effective 3D points in the neighborhood has a weight of 0.35. The consistency term between the estimated normal vector of the point to be compensated and the normal vector of the effective 3D points in the neighborhood has a weight of 0.25; the estimated normal vector of the point to be compensated is obtained by the B-spline surface at the camera image coordinates corresponding to the point to be compensated, respectively along... direction and The cross product of the first-order partial guide vectors in the direction is obtained by normalization.

[0073] The distance term between the compensation point at the boundary of the unobservable region and the phase-derived 3D boundary point has a weight of 0.40.

[0074] For the unobservable region that has already formed a multi-projection phase derived boundary constraint 504, the compensation and fusion module 500 solves the B-spline surface compensation model according to the compensation parameter set corresponding to the unobservable region under the surface geometric continuity constraint 503 and the multi-projection phase derived boundary constraint 504, and generates a compensation point cloud 505.

[0075] The compensation and fusion module 500 writes the compensation points from the compensation point cloud 505 into the positions of the initial 3D topography 406 where no valid 3D points have been obtained, according to the corresponding camera pixel positions. For pixel positions in the initial 3D topography 406 where valid initial 3D points already exist, no compensation points are written. After writing, the compensation point cloud 505 is fused with the initial 3D topography 406 to obtain the 3D topography 506 of the target object.

[0076] Each three-dimensional point in the three-dimensional shape 506 of the target object is marked with a data source label as either a "direct measurement point" or a "compensation point". Compensation points are not marked as directly measured real three-dimensional data.

[0077] III. The working process of the device When the device in this embodiment is working, the calibration module 100 first obtains the spatial parameter calibration result 101; the observability evaluation module 200 then acquires the pre-scan image sequence 201 and forms the orientation-related spatial observability distribution 203; the dynamic scheduling and acquisition module 300 determines the current measurement projection direction combination according to the association mapping 301. The projection direction selection result is written into the projection direction selection result field, the encoding parameters for each projection direction are configured, the pixel ratio of low observability areas or the "undetermined" flag and the encoding configuration result are written into the corresponding fields, and structured light images 305 are acquired; the phase decoding and reconstruction module 400 obtains the absolute phase 404, effective 3D data 405 and initial 3D shape 406 for each projection direction; the compensation and fusion module 500 generates a compensation point cloud 505 for the unobservable areas 501 that can obtain at least two directions of effective boundary absolute phase, marks the remaining unobservable areas as areas to be remeasured, and outputs the 3D shape 506 of the target object. In subsequent measurement cycles where the target object 40 and its measurement posture remain unchanged, the dynamic scheduling and acquisition module 300 reuses the direction-related spatial observability distribution 203, the association mapping 301 and the current measurement projection direction combination obtained in the first measurement cycle. The encoding configuration results are used to directly perform structured light projection and image acquisition based on the reuse results.

[0078] Example 2: This embodiment provides a method for rapid three-dimensional imaging and measurement using structured light. This method is executed using the rapid three-dimensional imaging and measurement device for structured light described in Embodiment 1. Figure 3 As shown, the method includes steps S1 to S8.

[0079] Step S1: Perform spatial parameter calibration A planar ceramic checkerboard calibration plate 50 with 12×9 interior corner points and a center distance of 10.000 mm between adjacent corner points is used to set 20 fixed calibration postures in the measurement space.

[0080] Camera 10 acquires checkerboard images in various poses to determine the intrinsic parameters, distortion parameters, and calibration plate pose parameters of camera 10.

[0081] At each calibration orientation, three projectors project five-step phase-shift fringes and four-bit Gray code, respectively. The phases of the five-step phase shift are 0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 10 ... , , and .

[0082] By treating each projector as a reverse camera, its intrinsic parameters, distortion parameters, and extrinsic parameters relative to the camera coordinate system are determined, and spatial parameter calibration results 101 are obtained.

[0083] The average reprojection error of camera 10 is 0.1 pixels, and the average reprojection errors of the first projector 21, the second projector 22, and the third projector 23 are 0.2 pixels, 0.2 pixels, and 0.3 pixels, respectively.

[0084] Step S2: Pre-scan and determine the spatial observability distribution of orientation correlation like Figure 4 As shown, the target object 40 is fixed within the measurement space. For ease of indexing during implementation, this embodiment uniformly uses subscripts to represent target surface area numbers and superscripts to represent projection direction numbers, i.e., ... , , and They represent the first The target surface region in the first Normalized fringe modulation, normalized grayscale response stability, directional consistency, and observability confidence under each projection direction. The first projector 21, the second projector 22, and the third projector 23 sequentially project a pre-scanned structured light sequence onto the target object 40, and the camera 10 simultaneously acquires a pre-scanned image sequence 201.

[0085] The pre-scan structured light sequence for each projection direction includes a low-frequency geometric subsequence and a co-frequency evaluation phase shift subsequence. The low-frequency geometric subsequence consists of five first phase shift fringes and four first Gray codes. The period of the first phase shift fringes is 64 projector pixels, and the four-bit first Gray code encodes each of the 16 fringe periods within the 1024-pixel effective projection width. The co-frequency evaluation phase shift subsequence consists of five second phase shift fringes, with a period of 16 projector pixels, the same as the phase shift fringes used in the formal measurement in step S5. Fourteen pre-scan images are acquired for each projection direction, for a total of 42 pre-scan images across the three projection directions. The phases of both sets of five-step phase shifts are sequentially 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 14, 15, 16, 17, 18, 19 ... , , and .

[0086] The target object mask is obtained based on the pre-scanned images from three projection directions, and then divided into 16×16 pixel sections within the camera imaging plane. A target surface area.

[0087] No. Pixel weights of each target surface region , which is the ratio of the number of pixels in the mask of this region to the total number of pixels in the mask of the entire target surface region.

[0088] For the The first projection direction The second phase-shifted fringe image, in pixels ( , The following grayscale model is used to fit the area at (). , in, For the first The projection direction of the first The second phase-shifted fringe image in pixels grayscale at the location; ; The DC component of the second phase-shifted fringe image. The modulation amplitude of the second phase-shifted fringe image. This is the wrapping phase of the second phase-shifted fringe image.

[0089] The pixel stripe modulation degree is calculated according to the following formula: .

[0090] Pixel normalized stripe modulation is calculated using the following formula: .

[0091] in, This means that the value is limited to between 0 and 1.

[0092] No. The target surface region in the first Normalized fringe tone under each projection direction , which is the total number of mask pixels in this region average value.

[0093] Divide the root mean square residual between the actual gray level of the second phase-shift fringe image and the gray level of the fitted image by... The pixel normalized fitting residuals are obtained. And determine the stability of pixel grayscale response: .

[0094] No. The target surface region in the first Normalized grayscale response stability under each projection direction , which is the total number of mask pixels in this region average value.

[0095] When decoding the low-frequency geometric subsequence, the DC component of the grayscale model of the first phase-shifted fringe of the same pixel is used as the binary determination threshold of the first Gray code. When the grayscale of the first Gray code image is not lower than the DC component, the corresponding code bit is recorded as 1; when it is lower than the DC component, the corresponding code bit is recorded as 0. Then, the low-frequency fringe period number is determined according to a fixed Gray code to binary code conversion relationship to obtain the pre-scan absolute phase.

[0096] The pre-scan 3D data is reconstructed based on the pre-scan absolute phase and spatial parameter calibration results 101. When the same camera pixel has valid pre-scan 3D points in multiple projection directions, the data determined by the second phase shift fringes is retained. and The 3D point reconstructed by the projection direction with the largest product.

[0097] Principal component analysis was performed on the effective pre-scanned 3D points of each target surface region and its adjacent ring region to obtain the local surface unit normal vector. .

[0098] The The direction is taken to make The direction, among which, For the first The three-dimensional centroid of the target surface region is a unit vector pointing to the optical center of the camera.

[0099] For target surface regions with fewer than 64 effective pre-scanned 3D points, local surface unit normals are not calculated, and the orientation of the corresponding region is kept consistent. Set it to zero.

[0100] Determine the first according to formula (1) The target surface region in the first Observability confidence under each projection direction : Equation (1) in, For interval The identity normalization function within.

[0101] Determine directional consistency according to formula (2) : Equation (2) in, For the first The three-dimensional centroid of the target surface region points to the first The unit vector of the optical center of the projector; Let be the unit normal vector of the local surface; For the first A unit vector pointing from the three-dimensional centroid of the target surface region to the optical center of camera 10; max indicates taking the maximum value among the values ​​in parentheses; · indicates the vector dot product.

[0102] According to all Formation of directionally correlated spatial observability distribution 203.

[0103] Step S3: Establish association mapping Numbered by target surface area and projection direction number For the joint index, establish the association mapping between the target surface region and the projection direction 301.

[0104] The associated mapping 301 includes regional direction evaluation records and projection direction configuration records. The regional direction evaluation record records the... The target surface region in the first Under each projection direction , , , The number of effective structured light response pixels and the number of projected coverage pixels, among which and Determined by the same-frequency evaluation phase shift subsequence; the projection direction configuration record is numbered according to the projection direction. For the index, set the projection direction selection result field, the low observability area pixel ratio field, and the encoding configuration result field. In step S3, all three fields are yet to be determined and are in a pending write state.

[0105] Step S4: Determine the combination of projection directions like Figure 5 As shown, the three projection directions are respectively denoted as... , and Enumeration by , and The set of seven non-empty candidate projection directions .

[0106] The stripe modulation determined by the same-frequency evaluation phase shift subsequence Not less than 0.10 and fitted DC component Pixels located between 205 and 3890 are identified as effective structured light response pixels.

[0107] The same pixel in the set of candidate projection directions When two or more projection directions are valid, it is counted only once.

[0108] For the set of candidate projection directions In the The ratio of the number of effective structured light response pixels within a target surface region to the number of mask pixels in that region.

[0109] The set of candidate projection directions is determined according to equation (3). Projection direction selection index : Equation (3) in, For the set of candidate projection directions Number of projection directions included.

[0110] Determine the set of candidate projection directions according to equation (4). Does the measurement coverage constraint meet? Equation (4) In the candidate set that satisfies equation (4), The largest set is determined by the combination of projection directions. Combine the projection directions Determined as the current measurement projection direction combination .

[0111] Two candidate sets The difference does not exceed When the number of projection directions is small, select the set with fewer projection directions; when the number of projection directions is the same, select the set according to... , , The numbering sequence determines the unique After completing the above comparisons, determine whether each projection direction belongs to a combination of projection directions. The result is written to the projection direction selection result field. When there is no set of candidate projection directions that satisfy equation (4), , and The complete set of projection directions is determined as the backup projection direction combination, and the backup projection direction combination is determined as the current measurement projection direction combination. The result of selecting all three projection directions is written into the projection direction selection result field, and a measurement coverage insufficiency flag is output; the alternative projection direction combination is not considered a projection direction combination that satisfies equation (4). .

[0112] Step S5: Configure encoding parameters and acquire structured light images When the current measurement projection direction combination The projection direction combination At that time, for the current measurement projection direction combination For each projection direction, count the number of projection directions within the target object's projection coverage area. The target surface area pixel percentage is calculated and written into the low observable area pixel percentage field.

[0113] When the pixel percentage is less than 20%, a low-redundancy coding sequence 303 is configured; when the pixel percentage is not less than 20%, an enhanced coding sequence 304 is configured. When the current measurement projection direction combination... When combining the alternative projection directions, the pixel ratio of the low observability area is not calculated, and the "undetermined" flag is written into the low observability area pixel ratio field. Enhanced coding sequence 304 is configured for all three projection directions, and the corresponding coding configuration results are written into the coding configuration result field.

[0114] Complete the current measurement projection direction combination After configuring the encoding parameters for each projection direction, the configuration results of the low-redundancy encoding sequence 303 or the enhanced encoding sequence 304 corresponding to each projection direction are written into the encoding configuration result field.

[0115] The low-redundancy coding sequence 303 consists of six Gray codes and three phase-shifted fringes, for a total of nine images. The phases of the three phase shifts are 0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 10, 11, 19, 10, and .

[0116] The enhanced coding sequence 304 consists of six Gray codes and five phase-shifting fringes, for a total of 11 images. The phases of the five phase shifts are 0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 10, 11, 19, 10, 1 , , and .

[0117] Both coded sequences have a phase-shift fringe period of 16 projector pixels. Within an effective projection width of 1024 pixels, 64 complete periods are formed, with each period numbered from 0 to 63 identified by a six-bit Gray code.

[0118] For each projection direction, six Gray code images are first projected from the most significant bit to the least significant bit, followed by phase-shifted fringes. Current measurement projection direction combination. The projector in , , The numbered images are projected sequentially, and the structured light image 305 is simultaneously acquired by camera 10.

[0119] Combine the current measurement projection direction The number of projection directions of the low-redundancy coding sequence is denoted as . The number of projection directions of the configuration enhancement coding sequence is denoted as The number of images measured in the formal measurements is: .

[0120] After the initial pre-scan, projection direction combination determination, and encoding parameter configuration of the same workpiece, in subsequent measurement cycles where the workpiece and its measurement posture remain unchanged, the orientation-related spatial observability distribution 203, association mapping 301, and current measurement projection direction combination obtained in the initial measurement cycle are reused. The encoding configuration results are used, and structured light projection and image acquisition are performed directly based on the reuse results; when the target object or its measured posture changes, steps S2 to S5 are re-executed.

[0121] Step S6: Perform phase unfolding, 3D reconstruction and fusion. like Figure 6 As shown, based on the current measurement projection direction combination The structured light image 305 in each projection direction calculates the wrapping phase 401, and the wrapping phase range is... .

[0122] A region adjacency graph is constructed based on the four-neighbor relationships between target surface regions. Two adjacent regions are determined to have a propagable adjacency relationship 407 when they satisfy the following conditions: The shared boundary length is no less than 4 camera pixels; The proportion of pixels with effective wrapping phase on both sides of the shared boundary is no less than 80%; The wrapping phase difference of corresponding pixel pairs with effective wrapping phase on both sides of the shared boundary. The median of the absolute values ​​of the phase differences obtained after equivalent conversion is no greater than .

[0123] Will The target surface area with a value of not less than 0.75 and an effective pixel ratio of not less than 80% for six-bit Gray code is determined as the phase unwrapping starting region 402, and the number of integer cycles within the starting region is determined based on the Gray code decoding result.

[0124] Phase unrolling employs a confidence-priority queue. Each time, it selects from the unrolled regions that form a propagable adjacency relationship 407 with the unrolled regions. The largest region is designated as the current region. If the numbers are the same, select the region with the smaller region number.

[0125] Select from the expanded adjacent regions of the current region The largest area is used as the reference area.

[0126] For integer period numbers in the range of 0 to 63 Calculate the median of the following values ​​at each shared boundary: .

[0127] Take the one with the smallest median The integer period number is used as the current region. The above process is repeated until the confidence priority queue is empty. Regions not connected to any phase unrolling starting region 402 do not output absolute phase 404.

[0128] like Figure 7 As shown, the absolute phase 404 is converted into column coordinates within the effective projection area of ​​the projector, and these column coordinates are used to determine the projected equal-phase light plane in the camera coordinate system. The intersection points of the camera pixel space ray and the projected equal-phase light plane are then obtained to acquire the three-dimensional points.

[0129] The 3D points are reprojected onto the camera's imaging plane and the projector's phase coordinates. Any 3D point with a reprojection error greater than 0.50 pixels, a fringe modulation degree less than 0.10, or a DC component exceeding 205 to 3890 is discarded. The remaining data constitutes the valid 3D data 405 for the corresponding projection direction.

[0130] If a camera pixel has only one valid 3D point in a single projection direction, that 3D point is retained.

[0131] The same camera pixel has valid 3D points in two or more projection directions, and the corresponding valid projection directions are... When the sum is greater than zero, the corresponding effective projection directions are... Divided by the above The sum of these values ​​yields the normalized fusion weights, which are then used to perform weighted fusion on the corresponding 3D coordinates to obtain the initial 3D topography 406.

[0132] When the effective projection directions corresponding to the same camera pixel When the sum is zero, the effective 3D points corresponding to the camera pixel are not written into the initial 3D shape 406, and the camera pixel is marked as a pixel that has not obtained effective 3D points.

[0133] Step S7: Compensate for unobservable areas like Figure 8 As shown, the four-connected region within the target object mask where no effective three-dimensional points are obtained is defined as the unobservable region 501.

[0134] For each unobservable region, an annular neighborhood with a width of 8 camera pixels is extracted outward to obtain the effective 3D point cloud 502 of the neighborhood.

[0135] For each 3D point in the neighborhood, principal component analysis is performed on the 30 3D points with the closest spatial distance to determine the local surface unit normal vector; then a quadratic surface is fitted to determine the two principal curvatures at that 3D point.

[0136] The neighborhood average absolute principal curvature is the arithmetic mean of the absolute values ​​of the two principal curvatures at all neighborhood 3D points.

[0137] The number of pixels in the unobservable region is less than 400 and the average absolute principal curvature of the neighborhood is less than In this case, the first compensation parameter group 507 is used. The second compensation parameter group 508 is used for the remaining unobservable areas.

[0138] The first compensation parameter group 507 is fixed as a quadratic tensor product B-spline, an 8-pixel control point spacing, and 20 Gaussian-Newton iterations.

[0139] The second compensation parameter set 508 is fixed at cubic tensor product B-splines, 4-pixel control point spacing, and 80 Gaussian-Newton iterations. The first compensation parameter set 507 is suitable for unobservable regions with a small number of pixels and a low neighborhood mean absolute principal curvature. The lower B-spline surface degree, larger control point spacing, and fewer iterations are used to reduce computation while meeting the fitting requirements for low curvature regions. The second compensation parameter set 508 is suitable for unobservable regions with a large number of pixels or a high neighborhood mean absolute principal curvature. The higher B-spline surface degree, smaller control point spacing, and more iterations are used to improve the fitting freedom and convergence accuracy of the compensation surface for larger regions or higher curvature surfaces.

[0140] Based on the three-dimensional spatial distance between the point to be compensated and the effective three-dimensional points in the neighborhood, and the consistency between the estimated normal vector of the point to be compensated and the local surface unit normal vector of the effective three-dimensional points in the neighborhood, a surface geometric continuity constraint 503 is formed.

[0141] For each unobservable region 501, the effective absolute phase data of different projection directions obtainable at its boundary are statistically analyzed. First, starting from the current measurement projection direction combination... Extract the valid formal measurement absolute phase obtained in step S6; when the number of different projection directions corresponding to the valid formal measurement absolute phase is not less than two, directly use the valid formal measurement absolute phase; when the number of different projection directions corresponding to the valid formal measurement absolute phase is less than two, use the observability confidence recorded by association mapping 301. From high to low, the effective pre-scan absolute phases of the remaining projection directions obtained from the low-frequency geometric subsequence decoding in step S2 are used for supplementation. After completing the above selection and necessary supplementation, when at least two different projection directions' effective absolute phase data have been obtained, they are converted into three-dimensional boundary points in the camera coordinate system according to the fringe period and spatial parameter calibration result 101 corresponding to each effective absolute phase. When participating in fusion... When the sum is greater than zero, the corresponding Divide by the number of participants in the integration The sum of these values ​​is used to obtain the corresponding normalized weights, and the three-dimensional boundary points are then weighted and fused according to these normalized weights; when the points participating in the fusion... When the sum is zero, equal-weight fusion is performed on the three-dimensional boundary points. The fused three-dimensional boundary points are used to form the multi-projection phase derived boundary constraint 504. The above equal-weight fusion only considers that the data participating in the fusion are all valid absolute phase data, but their corresponding... Degeneracy handling when the sum is zero and normalized weights cannot be calculated, and invalid absolute phase data is not included in the fusion.

[0142] If, after completing the above selection and necessary supplements, effective absolute phase data for at least two different projection directions still cannot be obtained, the corresponding unobservable region 501 is marked as the region to be remeasured. The B-spline surface compensation model for this region is not solved, nor is the compensation point cloud 505 generated for this region.

[0143] The compensation model uses camera image coordinates Mapped to 3D points Its objective function is fixed as: .

[0144] in, It is the sum of the weighted points to the squared surfaces of the local tangent planes from the point to be compensated to the effective 3D points in the neighborhood; The sum of squared dot products between the estimated normal vector of the point to be compensated and the unit normal vector of the local surface of the effective 3D points in the neighborhood is calculated as 1 minus the squared dot product. This is the sum of squared distances between the compensation point at the boundary of the unobservable region and the phase-derived three-dimensional boundary point.

[0145] The estimated normal vector of the point to be compensated is obtained by the B-spline surface at the camera image coordinates corresponding to the point to be compensated, along... direction and The cross product of the first-order partial guide vectors in the direction is obtained by normalization.

[0146] For the unobservable region that has already formed a multi-projection phase derived boundary constraint 504, under the surface geometric continuity constraint 503 and the multi-projection phase derived boundary constraint 504, the B-spline surface compensation model is solved according to the compensation parameter set corresponding to the unobservable region to generate the compensation point cloud 505.

[0147] Step S8: Fuse and output the 3D shape of the target object The compensation points in the compensation point cloud 505 are written into the positions of the initial 3D topography 406 where no valid 3D points have been obtained, according to the corresponding camera pixel positions; for the pixel positions of the initial 3D topography 406 where valid initial 3D points already exist, no compensation points are written.

[0148] The written compensated point cloud 505 is fused with the initial 3D topography 406 to obtain the 3D topography 506 of the target object. Each 3D point in the 3D topography 506 of the target object is marked with a data source label as either a "direct measurement point" or a "compensation point" to distinguish between directly measured 3D data and compensated data generated based on boundary and neighborhood constraints.

[0149] Application test case: Technical effect comparison experiment: I. Experimental conditions and test specimens The experiment used the apparatus described in Example 1 and the method described in Example 2. The experiment was conducted under the following conditions: ambient light intensity not exceeding 200 lx and both the device and the target object being fixed.

[0150] Spatial parameter calibration is completed before the start of each group's timing. All groups share the same spatial parameter calibration result 101. The spatial parameter calibration time is not included in the measurement time of each group.

[0151] The experiment used a double-sphere ceramic ball-and-stick and a complex curved surface.

[0152] The calibrated diameters and center-to-center distance of the two balls in a double-ball ceramic bat are provided by a qualified metrology institution. First sphere calibration diameter ; Second ball calibration diameter ; Calibration ball center distance ; Expanded uncertainty Including factors .

[0153] Place the bat at seven positions in the measurement space: center, front left, front right, back left, back right, top, and bottom. Repeat the measurement 10 times at each position.

[0154] The complex curved test specimen has external dimensions of 120 mm × 90 mm × 40 mm. Its surface is provided with blind holes with a diameter of 20 mm and a depth of 20 mm, a V-shaped groove with an included angle of 60° and a depth of 12 mm, and two steps with heights of 15 mm and 25 mm respectively. The surface of the specimen is treated with matte white finish.

[0155] A reference 3D point cloud of a complex curved surface was obtained using a calibrated industrial X-ray CT system. The voxel spacing of the reference point cloud was 0.05 mm, and the expanded uncertainty was... Including factors .

[0156] II. Control Group Control group A was a single-projection fixed-coding group, using only the first projector 21 to project a fixed six-bit Gray code and five-step phase-shifting fringes, for a total of 11 images. Control group A did not perform projection direction selection, confidence-guided region propagation, multi-directional weighted fusion, or unobservable region compensation.

[0157] Control group B is a three-projection fixed-coding group, with three projectors projecting six-bit Gray codes and five-step phase-shifting fringes, totaling 33 images. For cases where a single camera pixel has a valid 3D point in only one projection direction, that valid 3D point is retained; for cases where a single camera pixel has a valid 3D point in multiple projection directions, the valid 3D points in each projection direction are fused with equal weights. Missing regions are treated using cubic tensor product B-splines, 4-pixel control point spacing, and 80 Gaussian-Newton iterations, employing only geometric distance and normal consistency terms, without using multi-projection phase-derived boundary constraints. The projection directions and coding parameters of both control groups A and B are pre-fixed values ​​and do not depend on observability evaluation results; therefore, the pre-scanning step S2 is not performed in either group.

[0158] Both control group A and control group B directly determine the integer number of phase expansion cycles of the corresponding pixel based on the pixel-by-pixel decoding result of the six-bit Gray code, and combine the integer number of phase expansion cycles with the effective wrapping phase obtained by the phase shift fringe to obtain the absolute phase of the corresponding pixel, without performing region propagation based on observability confidence; pixels that cannot be effectively decoded by the six-bit Gray code or the wrapping phase do not output the absolute phase.

[0159] Experimental group C is the group of this invention. It uses the spatial parameter calibration result 101 obtained before the start of this experiment according to step S1 and shared by all groups. In each independent repeated experiment, steps S2 to S8 of Example 2 are executed. In the 30 independent repeated experiments of this experiment, the projection direction combinations determined by experimental group C all include two projection directions, and the proportion of pixels in the low observability region of both projection directions is less than 20%. Therefore, low-redundancy coding sequences are configured for each direction. Each projection direction projects a six-bit Gray code and a three-step phase-shifting fringe, resulting in a total of nine images. Experimental group C acquired a total of 18 formal measurement images per formal measurement cycle.

[0160] In the complex curved surface experiment, control group A, control group B, and experimental group C conducted 30 independent repeated trials in the order of A, B, C. Before each independent repeated trial in experimental group C, the orientation-related spatial observability distribution, correlation mapping, projection direction combination, and coding configuration results formed by the previous independent repeated trial were cleared, and the first measurement cycle was executed first. After the first measurement cycle was completed, subsequent measurement cycles were executed directly while keeping the target object and its measurement posture unchanged. The projection direction and coding parameters of control groups A and B were pre-fixed values, and a formal measurement was performed once for each independent repeated trial. Thus, experimental group C obtained 30 initial measurement times and 30 subsequent measurement times, while control groups A and B each obtained 30 formal measurement times under fixed configurations.

[0161] III. Evaluation Indicators Ten acquisitions were performed on the three projection directions using six-bit Gray code and twelve-step phase-shifting fringes, and the averages were calculated to obtain the reference absolute phase corresponding to each projection direction. The twelve-step phase-shifting fringes were projected by the same projector corresponding to the projection direction and had the same fringe spatial frequency, projection direction, and phase zero as the phase-shifting fringes used in the formal measurement of that projection direction. The six-bit Gray code had the same encoding range and encoding order as the six-bit Gray code used in the formal measurement.

[0162] Before calculating the phase unfolding error rate, the test absolute phase and the reference absolute phase under the same projection direction are registered at the phase zero point according to the preset reference area.

[0163] During testing, the test absolute phase obtained in each projection direction is compared with the reference absolute phase corresponding to the same projection direction; for camera pixels where both the test absolute phase and the reference absolute phase are valid, the absolute value of the difference between the two is greater than [missing value]. The number of pixels with phase unfolding errors in each projection direction is counted. The sum of the number of phase unfolding error pixels in each projection direction involved in the formal measurement is divided by the sum of the number of pixels in the corresponding projection direction where both the test absolute phase and the reference absolute phase are valid to obtain the phase unfolding error rate. When the same camera pixel has a phase unfolding error in different projection directions, it is counted separately according to different projection directions.

[0164] Point cloud completeness is the ratio of the number of effective 3D points obtained within the target object mask to the total number of pixels in the target object mask. The nearest distance from the test point to the reference point cloud is used to calculate the Chamfer distance, compensate for the root mean square error of the boundary, and other 3D accuracy indicators, but is not used as a criterion for counting point cloud completeness.

[0165] The unobservable region ratio is the ratio of the number of pixels within the target object mask that did not obtain valid 3D points to the total number of pixels in the target object mask. Under the same statistical mask and valid 3D point determination conditions, the sum of point cloud integrity and the unobservable region ratio is 100%.

[0166] The root mean square error of the compensation boundary is the root mean square of the shortest distance from the boundary point of the compensation point cloud to the reference point cloud.

[0167] The Chamfer distance is the sum of the average nearest neighbor distances from the test point cloud to the reference point cloud and the average nearest neighbor distances from the reference point cloud to the test point cloud.

[0168] The detection shape error is the difference between the maximum and minimum radial residuals at each measurement point after fitting the sphere.

[0169] IV. Results of the Data Acquisition Efficiency Experiment The theoretical acquisition time is calculated based on the 120 Hz projection pattern frequency, excluding projector switching time and calculation time.

[0170]

[0171] The measurement time for each group starts from the measurement command that triggers the corresponding measurement cycle and ends with the completion of the 3D measurement result output. Spatial parameter calibration time is not included in the measurement time. Equipment startup, initial loading of structured light patterns, initial allocation of acquisition buffers, and initialization of processing programs are one-time initialization processes, recorded separately and not included in the measurement time in the table. The projection direction and encoding parameters of control groups A and B are fixed in advance, and there is no first measurement cycle generated by pre-scanning and dynamic configuration. The table records the single formal measurement time under the fixed configuration. The total first measurement time of experimental group C includes the projection and acquisition of 42 pre-scanned images, pre-scanned image processing, formation of orientation-related spatial observability distribution, establishment of association mapping, comparison of candidate projection direction sets, encoding parameter configuration, projection and acquisition of formal measurement images, as well as phase decoding, 3D reconstruction, fusion, and compensation processing. The single formal measurement time after configuration is determined corresponds to the reuse of the orientation-related spatial observability distribution 203, association mapping 301, and current measurement projection direction combination obtained in the first measurement cycle, under the condition that the target object and its measurement posture remain unchanged. The time required to complete a formal measurement after the coding configuration results are obtained.

[0172] V. Bat Measurement Results The maximum absolute error and repeatability standard deviation are calculated from ten measurements taken at seven measurement positions. The repeatability standard deviation is the maximum value among the ten repeatability standard deviations of the first ball diameter, second ball diameter, and ball center distance at each of the seven measurement positions. For experimental group C, after each time the stick is adjusted to a new measurement position, due to the change in the stick's measurement posture, an initial measurement cycle is performed, including pre-scanning, projection direction combination determination, and encoding parameter configuration. Subsequently, nine subsequent measurement cycles with reused configurations are performed continuously while the stick and its measurement posture remain unchanged. The maximum absolute error and repeatability standard deviation at this measurement position are calculated based on the ten sets of three-dimensional measurement results output from the initial measurement cycle and the nine subsequent measurement cycles.

[0173]

[0174] VI. Measurement Results of Complex Curved Surface Test Items The measurement quality index for complex curved test specimens was statistically analyzed using the three-dimensional results output by experimental group C under configuration reuse conditions in each independent repeated experiment, resulting in 30 sets of results. The output of the first measurement cycle was only used for the total time statistics of the first measurement and was not included in the measurement quality index statistics in this section. Control groups A and B used the single formal measurement results under fixed configuration in each independent repeated experiment. The data in the table are the average ± standard deviation of the 30 measurement results. The "Formal Measurement Effective Response Pixel Coverage" in the table is the ratio of the number of camera pixels in each projection direction participating in the formal measurement with a fringe modulation degree of not less than 0.10, a fitted DC component in the range of 205 to 3890, and passing the phase validity judgment to the total number of target object mask pixels. The phase validity judgment refers to the ability of the corresponding camera pixels to output an effective absolute phase according to the absolute phase acquisition process of their respective experimental groups. For control groups A and B, when the corresponding camera pixels can determine the integer number of phase unfolding cycles based on the effective pixel-by-pixel decoding result of the six-bit Gray code, and combine it with the effective wrapping phase to obtain the absolute phase, they pass the phase validity determination. For experimental group C, when the corresponding camera pixels can obtain the effective wrapping phase, and the target surface area where the camera pixel is located is the phase unfolding starting area, or is connected to any phase unfolding starting area through a propagable adjacency relationship, thereby obtaining an effective absolute phase, they pass the phase validity determination. Pixels that do not output an effective absolute phase do not pass the phase validity determination. The formal measurement of the effective response pixel coverage is statistically analyzed before 3D reconstruction, reprojection error screening, and multi-directional 3D data fusion; the same camera pixel is counted only once when it meets the above conditions in multiple projection directions. The measurement coverage constraint in step S4 is statistically analyzed according to the union of the effective structured light response pixels in each candidate projection direction in the pre-scanned image; the statistical stages and uses of the two are different.

[0175]

[0176] The above experimental results show that, under the experimental conditions, the overall technical solution adopted in this invention, which combines orientation-related spatial observability evaluation, projection direction and coding parameter configuration, observability confidence-guided phase unfolding and 3D data fusion, and compensation processing that combines multi-projection phase-derived boundary constraints and surface geometric continuity constraints, can reduce the amount of formal measurement image acquisition and subsequent measurement time, reduce the phase unfolding error rate, and improve the integrity of 3D point clouds and the continuity of compensation boundaries.

[0177] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A structured light rapid three-dimensional imaging and measurement device, characterized in that, include: The system comprises a camera, multiple projectors, and a computer device. The multiple projectors are respectively positioned at different locations relative to the camera. The projection optical axes of the multiple projectors have different directions relative to the observation optical axis of the camera. The projection areas of the multiple projectors at least partially overlap with the imaging field of view of the camera. The camera and the multiple projectors are all communicatively connected to the computer device. The computer device includes the following modules: a calibration module, used to calibrate the spatial parameters of the camera and the plurality of projectors; An observability evaluation module is used to control the multiple projectors to sequentially project a pre-scanning structured light sequence onto the target object, and to control the camera to synchronously acquire pre-scanning image sequences corresponding to each projection direction. The module divides the target surface corresponding to the pre-scanning image sequence into multiple target surface regions. The pre-scanning structured light sequence includes a first phase-shifted fringe sub-sequence, a first Gray code sub-sequence corresponding to the first phase-shifted fringe sub-sequence, and a second phase-shifted fringe sub-sequence. The fringe spatial frequency of the first phase-shifted fringe sub-sequence is lower than that of the second phase-shifted fringe sub-sequence, and the fringe spatial frequency of the second phase-shifted fringe sub-sequence is the same as that of the phase-shifted fringe spatial frequency in the structured light coding sequence used for formal measurement. Based on the fringe modulation and grayscale response stability of each target surface region under each projection direction determined by the second phase-shifted fringe sub-sequence, and the directional consistency determined by the first phase-shifted fringe sub-sequence, the first Gray code sub-sequence, and the spatial parameter calibration results, the corresponding observability confidence is determined, and a direction-related spatial observability distribution is formed from the observability confidence of each target surface region under each projection direction. The dynamic scheduling and acquisition module is used to establish an association mapping based on the direction-related spatial observability distribution. This association mapping uses the target surface region number and projection direction number as a joint index to record the corresponding fringe modulation, grayscale response stability, direction consistency, observability confidence, effective structured light response pixel information, and projection coverage information. It also sets fields for projection direction selection results, low observability region pixel proportion, and encoding configuration results. The effective structured light response pixels are determined based on whether the pixel fringe modulation determined by the second phase-shifted fringe subsequence meets a preset modulation condition, and whether the DC component fitted by the second phase-shifted fringe subsequence meets a preset unsaturation condition. For a non-empty candidate projection direction set, the module determines the effective structured light response pixel coverage rate of the candidate projection direction set in each target surface region, and the coverage rate of each target surface region. The maximum observability confidence of the surface region in the candidate projection direction set and the number of projection directions in the candidate projection direction set are used to weight the effective structured light response pixel coverage and the maximum observability confidence according to the proportion of the number of pixels in each target surface region to the total number of pixels in all target surface regions. This weights the weighted effective structured light response pixel coverage and the maximum observability confidence, and constructs a projection direction selection index that increases with the weighted effective structured light response pixel coverage and the maximum observability confidence, and decreases with the proportion of the number of projection directions to the total number of projection directions. The candidate projection direction set that meets the measurement coverage constraint that the weighted effective structured light response pixel coverage is not lower than a preset coverage threshold and has the largest projection direction selection index is determined as the projection direction combination. The projection direction selection result, which characterizes whether each projection direction belongs to the projection direction combination, is written into the projection direction selection result field. Configure corresponding structured light coding parameters according to the association mapping and the observability confidence distribution corresponding to each projection direction in the projection direction combination, and write the configuration result of the structured light coding parameters into the coding configuration result field; control the projector corresponding to the projection direction combination to sequentially project their respective structured light coding sequences, and control the camera to synchronously acquire the corresponding structured light images; the pre-scan, the determination of the projection direction combination, and the configuration of the structured light coding parameters are performed in the first measurement cycle after the target object enters the measurement space or the measurement posture of the target object changes; in subsequent measurement cycles where the target object and its measurement posture remain unchanged, reuse the direction-related spatial observability distribution, the association mapping, the projection direction combination, and the configuration result of the structured light coding parameters obtained in the first measurement cycle, and control the corresponding projector and the camera to complete the subsequent formal measurement according to the reuse result; The phase decoding and reconstruction module is used to calculate the wrapping phase based on the structured light image corresponding to each projection direction in the projection direction combination, determine the phase unfolding starting region and phase unfolding path of the projection direction based on the observability confidence of each target surface region under the same projection direction, so as to obtain the absolute phase of the projection direction; reconstruct the effective three-dimensional data under the corresponding projection direction based on the absolute phase of each projection direction in the projection direction combination and the result of the spatial parameter calibration, and fuse the effective three-dimensional data according to the corresponding observability confidence to obtain the initial three-dimensional morphology; The compensation and fusion module is used to determine the connected regions formed by camera pixels that have not obtained effective three-dimensional points in the initial three-dimensional topography as unobservable regions; For unobservable regions at the boundary where effective absolute phase data in at least two different projection directions can be obtained, the boundary constraints are derived by deriving the multi-projection phase based on the neighborhood 3D point cloud, surface geometric continuity constraints, and the results of selecting and fusing the effective absolute phase data according to the corresponding observability confidence of the associated mapping record, combined with the spatial parameter calibration. The B-spline surface compensation model is then solved to generate a compensation point cloud. For unobservable regions at the boundary where effective absolute phase data in at least two different projection directions cannot be obtained, they are marked as regions to be retested. The initial 3D shape is fused with the compensated point cloud to obtain the 3D shape of the target object.

2. The apparatus according to claim 1, characterized in that, The observability evaluation module is used to: determine the pre-scan absolute phase of each projection direction based on the first phase shift fringe subsequence and the first Gray code subsequence; obtain effective pre-scan three-dimensional data based on the pre-scan absolute phase and the spatial parameter calibration results; and determine the local surface unit normal vector of each target surface region based on the effective pre-scan three-dimensional data. The phase-shifting fringe subsequences corresponding to each projection direction are fitted with a phase-shifting fringe grayscale model. The fringe modulation degree of each target surface region under each projection direction is determined according to the ratio of the modulation amplitude to the DC component obtained by fitting. The fringe modulation degree is then normalized to obtain the normalized fringe modulation degree. The normalized residual between the actual grayscale and the fitted grayscale is used as the grayscale response fluctuation, and the normalized grayscale response stability is determined based on the grayscale response fluctuation. The smaller the grayscale response fluctuation, the greater the normalized grayscale response stability. A direction consistency index is determined based on the product of a first non-negative direction cosine and a second non-negative direction cosine. The first non-negative direction cosine is the larger of the sum of the dot product of the first unit vector pointing from the target surface region to the corresponding projector and the local surface unit normal vector of the target surface region, and zero. The second non-negative direction cosine is the larger of the sum of the dot product of the second unit vector pointing from the target surface region to the camera and the local surface unit normal vector, and zero. The direction of the local surface unit normal vector is chosen such that its dot product with the second unit vector is not less than zero. Preset positive weighting coefficients are used to weight the normalized fringe modulation, the normalized grayscale response stability, and the direction consistency index, respectively. An observability confidence level for each target surface region under each projection direction is obtained through a monotonically increasing normalization function. For target surface regions whose local surface unit normal vectors cannot be determined from the effective pre-scanned 3D data, the corresponding orientation consistency index is set to zero.

3. The apparatus according to claim 1, characterized in that, The dynamic scheduling and acquisition module is used to: when the same pixel is covered by multiple projection directions from the same candidate projection direction set, count only once when determining the effective structured light response pixel coverage rate; for each projection direction in the projection direction combination, determine the pixel proportion of the low observability area within the projection coverage range of the projector corresponding to the projection direction on the target surface, write the pixel proportion into the low observability area pixel proportion field, configure a low redundancy coding sequence when the pixel proportion is lower than a preset proportion threshold, and configure an enhanced coding sequence when the pixel proportion is not lower than the preset proportion threshold; the structured light coding parameters include at least one of phase shift steps and Gray code bits, the low redundancy coding sequence and the enhanced coding sequence are both used to cooperate with the phase unfolding processing to determine the absolute phase of the target surface area that meets the phase unfolding condition in their respective effective coverage areas, at least one structured light coding parameter of the low redundancy coding sequence is less than the same type of structured light coding parameter of the enhanced coding sequence, and the remaining structured light coding parameters are not greater than the same type of structured light coding parameter of the enhanced coding sequence; The low observability region is the target surface region where the observability confidence under the projection direction is lower than a preset confidence threshold.

4. The apparatus according to claim 1, characterized in that, The phase decoding and reconstruction module is used to: for each projection direction in the projection direction combination, establish a regional adjacency relationship based on the spatial adjacency relationship between the target surface regions; determine the adjacency relationship between two adjacent target surface regions where the proportion of corresponding pixel pairs with effective wrapping phases on both sides of the shared boundary is not less than a preset effective proportion threshold, and the median of the absolute value of the wrapping phase difference obtained after equivalent conversion of the corresponding pixel pairs with effective wrapping phases on both sides according to a complete phase period is not greater than a preset boundary phase threshold; determine the target surface regions with observability confidence not less than a preset unfolding threshold under the projection direction as the phase unfolding starting region set; and determine the integer number of wrapping phase periods in the phase unfolding starting region set based on the decoding result of the structured light coding sequence corresponding to the projection direction, so as to obtain the absolute phase of the phase unfolding starting region set; After obtaining the absolute phase of the set of phase unfolding starting regions, repeat the following process until there are no un-unfolded target surface regions that form a propagable adjacent relationship with the target surface regions that have completed phase unfolding: from the un-unfolded target surface regions that form a propagable adjacent relationship with the target surface regions that have completed phase unfolding, select the target surface region with the highest observability confidence under the projection direction as the current target surface region. From the target surface regions that form a propagable adjacency relationship with the current target surface region and have completed phase unfolding, select the target surface region with the highest observability confidence under the projection direction as the reference region; Based on the absolute phase continuity at the shared boundary between the reference region and the current target surface region, the integer number of phase unfolding cycles for the current target surface region is determined. The encapsulated phase within the current target surface region is compensated for by an integer multiple of the complete phase cycle according to the integer number of phase unfolding cycles to complete the phase unfolding of the current target surface region. If only one projection direction provides valid 3D data for the same camera pixel, this valid 3D data is retained as a valid 3D point in the initial 3D topography. If multiple projection directions provide valid 3D data for the same camera pixel, when the sum of the observability confidence scores for the target surface region corresponding to the multiple projection directions is greater than zero, the observability confidence scores for each projection direction are divided by the sum of the observability confidence scores to obtain corresponding normalized fusion weights. These normalized fusion weights are then used to perform weighted fusion of the valid 3D data corresponding to the same camera pixel. When the sum of the observability confidence scores is zero, the valid 3D data corresponding to the camera pixel is not written into the initial 3D topography, and the camera pixel is marked as a pixel that has not obtained valid 3D points.

5. The apparatus according to claim 1, characterized in that, The compensation and fusion module is used to: extract the effective three-dimensional point cloud adjacent to the unobservable region, and determine the local surface unit normal vector and the neighborhood average absolute principal curvature based on the effective three-dimensional point cloud, wherein the neighborhood average absolute principal curvature is the average of the absolute values ​​of the two principal curvatures at each three-dimensional point in the effective three-dimensional point cloud; The number of pixels in the unobservable region on the camera's imaging plane is determined. A first compensation parameter set is used when the number of pixels in the region is less than a preset pixel threshold and the average absolute principal curvature of the neighborhood is less than a preset curvature threshold; otherwise, a second compensation parameter set is used. In the second compensation parameter set, at least one of the B-spline surface order and the number of iterations is greater than a similar parameter in the first compensation parameter set, and the remaining similar parameters are not less than similar parameters in the first compensation parameter set. The surface geometric continuity constraint is formed based on the spatial distance between the 3D points in the effective 3D point cloud and the point to be compensated, and the consistency between the local surface unit normal vector and the estimated normal vector of the point to be compensated. For the unobservable region, the effective absolute phase data of different projection directions obtainable at its boundary are statistically analyzed. When the effective formally measured absolute phase data in the projection direction combination comes from at least two different projection directions, the effective formally measured absolute phase data is directly used. When the number of different projection directions corresponding to the effective formally measured absolute phase data is less than two, the effective pre-scan absolute phase data of the remaining projection directions are supplemented according to the observability confidence of the associated mapping record from high to low. After completing the above selection and necessary supplementation, when effective absolute phase data of at least two different projection directions have been obtained, each effective absolute phase data is converted into a three-dimensional boundary point in the camera coordinate system according to the spatial parameter calibration result. When the observability confidence of each three-dimensional boundary point participating in the fusion is... When the sum of the observability confidence scores is greater than zero, each observability confidence score is divided by the sum of the observability confidence scores to obtain the corresponding normalized weights, and the three-dimensional boundary points are weighted and fused according to the normalized weights; when the sum of the observability confidence scores is zero, the three-dimensional boundary points are fused with equal weights; the fused three-dimensional boundary points are used to form the multi-projection phase derived boundary constraints; after completing the above selection and necessary supplementation, if at least two different projection directions of effective absolute phase data cannot be obtained, the unobservable region is marked as a region to be retested, and no compensation point cloud is generated for the unobservable region; for unobservable regions where at least two different projection directions of effective absolute phase data have been obtained, under the surface geometric continuity constraints and the multi-projection phase derived boundary constraints, the B-spline surface compensation model is solved according to the first compensation parameter set or the second compensation parameter set to generate the compensation point cloud.

6. A measurement method using the structured light rapid three-dimensional imaging and measurement device according to any one of claims 1-5, characterized in that, include: S1: Perform spatial parameter calibration on the camera and the plurality of projectors; S2: Control the multiple projectors to sequentially project a pre-scanned structured light sequence onto the target object, and the camera synchronously acquires pre-scanned image sequences corresponding to each projection direction; the pre-scanned structured light sequence includes a first phase-shifted fringe sub-sequence, a first Gray code sub-sequence corresponding to the first phase-shifted fringe sub-sequence, and a second phase-shifted fringe sub-sequence, wherein the fringe spatial frequency of the first phase-shifted fringe sub-sequence is lower than that of the second phase-shifted fringe sub-sequence, and the fringe spatial frequency of the second phase-shifted fringe sub-sequence is the same as that of the phase-shifted fringe spatial frequency in the structured light coding sequence used for formal measurement; divide the target surface corresponding to the pre-scanned image sequence into multiple target surface regions, and determine the corresponding observability confidence based on the fringe modulation and grayscale response stability of each target surface region under each projection direction determined by the second phase-shifted fringe sub-sequence, as well as the directional consistency determined by the first phase-shifted fringe sub-sequence, the first Gray code sub-sequence, and the spatial parameter calibration results, and form a direction-related spatial observability distribution based on the observability confidence of each target surface region under each projection direction; S3: Establish an association mapping based on the direction-related spatial observability distribution. The association mapping uses the target surface region number and the projection direction number as a joint index to record the corresponding fringe modulation degree, grayscale response stability, direction consistency, observability confidence, effective structured light response pixel information, and projection coverage information. It also sets a projection direction selection result field, a low observability region pixel ratio field, and an encoding configuration result field. The effective structured light response pixels are determined based on whether the pixel fringe modulation degree determined by the second phase-shifted fringe subsequence meets a preset modulation degree condition, and whether the DC component fitted by the second phase-shifted fringe subsequence meets a preset unsaturation condition. S4: For a non-empty candidate projection direction set, determine the effective structured light response pixel coverage rate of the candidate projection direction set in each target surface region, the maximum observability confidence of each target surface region in the candidate projection direction set, and the number of projection directions in the candidate projection direction set. Weight the effective structured light response pixel coverage rate and the maximum observability confidence according to the proportion of pixels in each target surface region to the total number of pixels in all target surface regions. Construct a projection direction selection index that increases with the weighted effective structured light response pixel coverage rate and the maximum observability confidence, and decreases with the proportion of the number of projection directions to the total number of projection directions. Determine the candidate projection direction set that satisfies the measurement coverage constraint that the weighted effective structured light response pixel coverage rate is not lower than a preset coverage threshold and has the largest projection direction selection index as the projection direction combination. Write the projection direction selection result, which characterizes whether each projection direction belongs to the projection direction combination, into the projection direction selection result field. S5: Configure corresponding structured light coding parameters according to the correlation mapping and the observability confidence distribution corresponding to each projection direction in the projection direction combination; write the pixel ratio of low observability region in each projection direction into the low observability region pixel ratio field; and write the configuration result of the structured light coding parameters into the coding configuration result field; control the projector corresponding to the projection direction combination to sequentially project their respective structured light coding sequences, and the camera synchronously acquires the corresponding structured light images; The structured light coding parameters configured in steps S2, S3, S4 and S5 are executed in the first measurement cycle after the target object enters the measurement space or the measurement posture of the target object changes; in subsequent measurement cycles where the target object and its measurement posture remain unchanged, reuse the configuration results of the direction-related spatial observability distribution, the correlation mapping, the projection direction combination and the structured light coding parameters obtained in the first measurement cycle, and directly execute the structured light projection and image acquisition in step S5 according to the reuse results; S6: Calculate the wrapping phase based on the structured light image corresponding to each projection direction in the projection direction combination. Determine the phase unfolding starting region and phase unfolding path of the projection direction based on the observability confidence of each target surface region under the same projection direction to obtain the absolute phase of the projection direction. Reconstruct the effective three-dimensional data under the corresponding projection direction based on the absolute phase of each projection direction in the projection direction combination and the result of the spatial parameter calibration. Fuse the effective three-dimensional data based on the corresponding observability confidence to obtain the initial three-dimensional morphology. S7: The connected regions formed by camera pixels that have not obtained effective three-dimensional points in the initial three-dimensional topography are defined as unobservable regions; For unobservable regions at the boundary where effective absolute phase data in at least two different projection directions can be obtained, the boundary constraints are derived by deriving the multi-projection phase based on the neighborhood 3D point cloud, surface geometric continuity constraints, and the results of selecting and fusing the effective absolute phase data according to the corresponding observability confidence of the associated mapping record, combined with the spatial parameter calibration. The B-spline surface compensation model is then solved to generate a compensation point cloud. For unobservable regions at the boundary where effective absolute phase data in at least two different projection directions cannot be obtained, they are marked as regions to be retested. S8: Fuse the initial three-dimensional shape with the compensated point cloud to obtain the three-dimensional shape of the target object.

7. The method according to claim 6, characterized in that, Step S2 includes: The pre-scan absolute phase of each projection direction is determined based on the first phase-shifting fringe subsequence and the first Gray code subsequence. Effective pre-scan three-dimensional data is obtained based on the pre-scan absolute phase and the spatial parameter calibration results. The local surface unit normal vector of each target surface region is determined based on the effective pre-scan three-dimensional data. Determine the first according to formula (1) The target surface region in the first Observability confidence under each projection direction : Equation (1) The first is determined according to formula (2). The target surface region is in the first Orientation consistency index under each projection direction : Equation (2) in, Represents a monotonically increasing normalized function. This means taking the maximum value among them. Represents the dot product of two vectors; The number of target surface regions. The number of all projection directions. , ; For the first The target surface region is in the first Normalized fringe modulation in a projection direction, the fringe modulation being based on the first projection direction. The ratio of the modulation amplitude to the DC component is determined by fitting the phase-shifted fringe grayscale model to the second phase-shifted fringe subsequence corresponding to each projection direction. For the first The target surface region is in the first The normalized grayscale response stability under each projection direction is determined based on the normalized residual between the actual grayscale and the fitted grayscale of the second phase-shifted fringe subsequence. A smaller normalized residual indicates a more stable response. The larger; For the first The target surface region points to the first The unit vector of the projector corresponding to each projection direction. For the first A unit vector pointing from the target surface region to the camera. The local surface unit normal vector is determined from the effective pre-scan three-dimensional data obtained based on the pre-scan absolute phase determined by the first phase-shifting fringe subsequence and the first Gray code subsequence, as well as the spatial parameter calibration results. The direction is taken to make The direction; for target surface regions where the local surface unit normal vector cannot be determined from the effective pre-scanned 3D data, the corresponding Set to zero; , , The preset positive weighting coefficients are given, and the sum of the three is 1. Step S4 includes: determining the set of non-empty candidate projection directions according to equation (3). Projection direction selection index : Equation (3) In equation (3), the summation symbol represents the summation of the sums ... From 1 to Sum of corresponding terms, Indicates that within the set of candidate projection directions Take from each projection direction The maximum value; It is the set of candidate projection directions consisting of at least one projection direction from all projection directions. For the first The proportion of pixels in one target surface region to the total number of pixels in all target surface regions. The set of candidate projection directions In the first Effective structured light response pixel coverage in each target surface region The set of candidate projection directions The number of projection directions in the image. , , The preset positive weighting coefficients are used; the effective structured light response pixels are pixels whose pixel fringe modulation degree, determined according to the second phase-shifted fringe sub-sequence, is not lower than a preset modulation degree threshold and whose fitted DC component is within a preset unsaturated range. The same pixel is represented by the candidate projection direction set. When multiple projection directions are covered, only one count is made; The candidate projection direction set is determined according to equation (4). Does the measurement coverage constraint meet? Equation (4) The projection direction selection index satisfies equation (4) and Largest set of candidate projection directions Determined as the combination of projection directions ,in, The preset coverage threshold is defined as follows.

8. The method according to claim 6, characterized in that, Step S5 includes: for each projection direction in the projection direction combination, determining the target surface area with an observability confidence level not lower than a preset confidence threshold under that projection direction as a high observability area, and determining the target surface area with an observability confidence level lower than the preset confidence threshold under that projection direction as a low observability area; determining the pixel percentage of the low observability area within the projection coverage range of the projector corresponding to that projection direction on the target surface, writing the pixel percentage into the low observability area pixel percentage field, configuring a low redundancy coding sequence when the pixel percentage is lower than a preset ratio threshold, and configuring an enhanced coding sequence when the pixel percentage is not lower than the preset ratio threshold; the structured light coding parameters include at least one of phase shift steps and Gray code bits, the low redundancy coding sequence and the enhanced coding sequence are both used to cooperate with the phase unfolding processing in step S6 to determine the absolute phase of the target surface area that meets the phase unfolding condition in their respective effective coverage areas, at least one structured light coding parameter of the low redundancy coding sequence is less than the same type of structured light coding parameter of the enhanced coding sequence, and the remaining structured light coding parameters are not greater than the same type of structured light coding parameter of the enhanced coding sequence.

9. The method according to claim 6, characterized in that, Step S6 includes: For each projection direction in the projection direction combination, a regional adjacency relationship is established based on the spatial adjacency relationship between the target surface regions. The adjacency relationship between two adjacent target surface regions is determined as a propagable adjacency relationship, where the proportion of corresponding pixel pairs with effective wrapping phases on both sides of the shared boundary to all corresponding pixel pairs at the shared boundary is not less than a preset effective proportion threshold, and the median of the absolute value of the wrapping phase difference of the corresponding pixel pairs with effective wrapping phases on both sides after equivalent conversion according to a complete phase period is not greater than a preset boundary phase threshold. The target surface regions with observability confidence not less than a preset unfolding threshold under the projection direction are determined as the phase unfolding starting region set. The integer number of wrapping phase cycles in the phase unfolding starting region set is determined based on the decoding result of the structured light coding sequence corresponding to the projection direction, so as to obtain the absolute phase of the phase unfolding starting region set. After obtaining the absolute phase of the set of phase unfolding starting regions, the following process is repeated until there are no un-unfolded target surface regions that form a propagable adjacency with the target surface regions that have completed phase unfolding: from the un-unfolded target surface regions that form a propagable adjacency with the target surface regions that have completed phase unfolding, the target surface region with the highest observability confidence under the projection direction is selected as the current target surface region; from the target surface regions that form a propagable adjacency with the current target surface region and have completed phase unfolding, the target surface region with the highest observability confidence under the projection direction is selected as the reference region; based on the absolute phase continuity at the shared boundary between the reference region and the current target surface region, the integer number of phase unfolding cycles of the current target surface region is determined, and the wrapped phase in the current target surface region is compensated for an integer multiple of the complete phase cycle according to the integer number of phase unfolding cycles to complete the phase unfolding of the current target surface region; For a single camera pixel with only one valid 3D data point in a single projection direction, this valid 3D data point is retained as a valid 3D point in the initial 3D topography. For a single camera pixel with valid 3D data points from multiple projection directions, when the sum of the observability confidence scores for the target surface region where the camera pixel is located for the multiple projection directions is greater than zero, the observability confidence scores for each projection direction are divided by the sum of the observability confidence scores to obtain corresponding normalized fusion weights. These normalized fusion weights are then used to perform weighted fusion on each valid 3D data point corresponding to the same camera pixel. When the sum of the observability confidence scores is zero, the valid 3D data points corresponding to the camera pixel are not written into the initial 3D topography, and the camera pixel is marked as a pixel that has not obtained a valid 3D point.

10. The method according to claim 6, characterized in that, Step S7 includes: Extract the effective 3D point cloud adjacent to the unobservable region, and determine the local surface unit normal vector and the neighborhood average absolute principal curvature based on the effective 3D point cloud. The neighborhood average absolute principal curvature is the average of the absolute values ​​of the two principal curvatures at each 3D point in the effective 3D point cloud. Determine the number of region pixels of the unobservable region in the imaging plane of the camera. If the number of region pixels is less than a preset pixel number threshold and the neighborhood average absolute principal curvature is less than a preset curvature threshold, a first compensation parameter group is used; otherwise, a second compensation parameter group is used. At least one of the B-spline surface number and the number of iterations in the second compensation parameter group is greater than the same type of parameter in the first compensation parameter group, and the remaining same type of parameters are not less than the same type of parameter in the first compensation parameter group. Form the surface geometric continuity constraint based on the spatial distance between the 3D points in the effective 3D point cloud and the point to be compensated, and the consistency between the local surface unit normal vector and the estimated normal vector of the point to be compensated. For the unobservable region, the effective absolute phase data of different projection directions obtainable at its boundary are statistically analyzed. When the effective formally measured absolute phase data in the projection direction combination comes from at least two different projection directions, the effective formally measured absolute phase data is directly used. When the number of different projection directions corresponding to the effective formally measured absolute phase data is less than two, the effective pre-scan absolute phase data of the remaining projection directions are supplemented according to the observability confidence of the associated mapping record from high to low. After completing the above selection and necessary supplementation, when effective absolute phase data of at least two different projection directions have been obtained, the results of the spatial parameter calibration are used. Each obtained effective absolute phase data is converted into a 3D boundary point in the camera coordinate system. When the sum of the observability confidence scores of the 3D boundary points involved in the fusion is greater than zero, each observability confidence score is divided by the sum of the observability confidence scores to obtain a corresponding normalized weight, and the 3D boundary points are weighted and fused according to the normalized weight. When the sum of the observability confidence scores is zero, the 3D boundary points are fused with equal weight. The fused 3D boundary points are used to form the multi-projection phase derived boundary constraint. If, after completing the above selection and necessary supplementation, effective absolute phase data of at least two different projection directions cannot be obtained, the unobservable region is marked as a region to be retested, and no compensation point cloud is generated for the unobservable region. For unobservable regions where effective absolute phase data of at least two different projection directions have been obtained, the B-spline surface compensation model is solved according to the first compensation parameter set or the second compensation parameter set under the surface geometric continuity constraint and the multi-projection phase derived boundary constraint to generate the compensation point cloud.

Images