Partitioned structured light reconstruction method, computer device and medium

Abstract

The application relates to the technical field of optical three-dimensional measurement and reconstruction, and provides a partitioned structured light reconstruction method, computer equipment and a medium. The method comprises the following steps: a mapping relationship between camera pixel coordinates and optical-mechanical pixel coordinates is established, and the mapping relationship is taken as a calculation basis; brightness response calibration is performed on a target object to obtain a brightness transfer function of the camera pixel corresponding to the optical-mechanical pixel; frequency response calibration is performed on the target object to obtain a frequency transfer function of the camera pixel corresponding to the optical-mechanical pixel; the camera pixel is clustered according to the frequency transfer function and the brightness transfer function, so that the camera pixel is divided into a target region comprising different regional characteristics; projection parameters of the target region are obtained according to the brightness transfer function, the frequency transfer function and the mapping relationship, the projection parameters comprising projection frequency and projection gray scale; and ROI adaptive projection is performed on the target region based on the projection parameters, so that the corresponding target region meets the requirements of a PSM algorithm, and the partitioned structured light reconstruction is performed.

Description

Technical Field

[0001] This application relates to the field of optical three-dimensional measurement and reconstruction technology, and in particular to a partitioned structured light reconstruction method, computer equipment and medium. Background Technology

[0002] In the field of optical 3D reconstruction, phase-shifting fringe structured light (PSM) technology is widely used in industrial inspection, medical imaging, and other scenarios due to its high precision. Traditional PSM technology projects a globally uniform sinusoidal fringe pattern through a projector, relying on the assumption that the target object's surface is a Lambertian body. After the camera acquires the image, it extracts phase information based on a fixed model.

[0003] The light emitted by the optical engine is reflected by the target object and then reaches the camera. In other words, the final image obtained by the camera is modulated by the target object. Mathematically, this can be expressed as... ; Where F is the modulation function. The intensity of the pattern received by the camera. The intensity of the pattern projected by the optical engine.

[0004] The PSM algorithm calculates the phase based on the fringe image received by the camera, and is based on the following assumptions. ; Here, A and B are constants independent of time t. That is, the PSM algorithm is based on the assumption that the pattern modulated by the target object still retains sinusoidal characteristics, but this assumption only holds if the target object surface is subjected to Lambertian scattering.

[0005] However, in practical applications, target objects often have non-Lambertian surfaces (such as metals, highly reflective coatings, and black light-absorbing materials), complex geometries (deep holes, steep slopes), and dynamic scene requirements, which renders the above assumptions invalid. Therefore, this global PSM algorithm has the following drawbacks: 1. Grayscale distortion caused by material differences: Non-uniform reflection characteristics cause overexposed or underexposed areas in camera images, phase extraction fails, and the missing rate of 3D point clouds can reach 15%-30%; 2. Depth of field limitation and high-frequency fringe blur: The modulation transfer function (MTF) of high-frequency fringe in the region outside the depth of field decreases by more than 50%, and the phase signal-to-noise ratio deteriorates; 3. Secondary reflection interference: Mutual reflection in highly reflective areas leads to phase information errors, significantly affecting the inspection accuracy of precision parts; 4. Insufficient timeliness of multi-optical-mechanical systems: Traditional solutions require serial mapping, which cannot meet real-time requirements in dynamic scenarios.

[0006] Therefore, a method is urgently needed to solve at least one of the above problems. Summary of the Invention

[0007] This application provides a partitioned structured light reconstruction method, computer equipment, and medium, aiming to solve the problems existing in the global PSM algorithm.

[0008] In a first aspect, embodiments of this application provide a method for partitioned structured light reconstruction, the method comprising: Establish a mapping relationship between camera pixel coordinates and optical engine pixel coordinates, and use the mapping relationship as the basis for calculation; The brightness response of the target object is calibrated to obtain the brightness transfer function of the optical engine corresponding to the camera pixel; Frequency response calibration of the target object is performed to obtain the frequency transfer function of the optical engine corresponding to the camera pixels; The camera pixels are clustered based on the frequency transfer function and the brightness transfer function to divide the camera pixels into target regions that include different regional features. The projection parameters of the target area are obtained based on the luminance transfer function, frequency transfer function and mapping relationship. The projection parameters include the projection frequency and projection grayscale. The target area is then subjected to ROI adaptive projection based on the projection parameters so that the corresponding target area meets the requirements of the PSM algorithm for partitioned structured light reconstruction.

[0009] In some embodiments, establishing the mapping relationship between camera pixel coordinates and optical-mechanical pixel coordinates includes: obtaining camera pixel coordinates and corresponding optical-mechanical pixel coordinates; establishing a sub-pixel-level mapping relationship between camera pixel coordinates and optical-mechanical pixel coordinates through bi-target calibration, while using bicubic interpolation to optimize the non-integer coordinate problem as the basis for calculating the corresponding luminance transfer function and frequency transfer function.

[0010] In some embodiments, the step of calibrating the brightness response of the target object to obtain the brightness transfer function of the optical engine corresponding to the camera pixel includes: controlling the optical engine to project a grayscale gradient pattern onto the target object and obtaining the average grayscale of the optical engine projection pattern; obtaining the grayscale of the camera pixel based on the grayscale gradient pattern received by the camera; obtaining the gain coefficient and the offset coefficient based on the average grayscale of the optical engine projection pattern and the grayscale of the camera pixel; and fitting a brightness response calibration curve based on the gain coefficient and the offset coefficient to calibrate the brightness response of the target object and obtain the brightness transfer function of the optical engine corresponding to the camera pixel.

[0011] In some embodiments, the step of calibrating the brightness response of the target object to obtain the brightness transfer function of the optical engine corresponding to the camera pixel further includes: obtaining a safe range of grayscale projected by the optical engine; setting a saturation threshold of the camera according to the safe range; and setting constraints on the optical engine according to the saturation threshold, so that the operation of the optical engine and the camera are both within the safe range.

[0012] In some embodiments, the step of calibrating the frequency response of the target object to obtain the frequency transfer function of the optical engine corresponding to the camera pixel includes: controlling the optical engine to project a standard pattern, the standard pattern including an input contrast; capturing the standard pattern with a camera to obtain an output contrast; and calculating the frequency transfer function of the optical engine corresponding to the camera pixel based on the input contrast and the output contrast.

[0013] In some embodiments, the step of calibrating the frequency response of the target object to obtain the frequency transfer function of the optical engine corresponding to the camera pixel further includes: obtaining the fringe frequency corresponding to each pixel according to the frequency transfer function; finding the optimal frequency based on the multiple fringe frequencies of the pixel, and the pixel having the highest sharpness when it is at the optimal frequency.

[0014] In some embodiments, clustering camera pixels according to a frequency transfer function and a luminance transfer function to divide the camera pixels into target regions including different regional features includes: constructing multiple pixel feature vectors according to the luminance transfer function and the frequency transfer function; clustering the constructed multiple pixel feature vectors to cluster the captured camera pixels; constructing a region mask according to the clustered camera pixels, and dividing the camera pixels into regions including different regional features through the region mask.

[0015] Secondly, this application provides a computer device including a processor, a memory, and a computer program stored in the memory and executable by the processor. The memory stores a strategy model, and the computer program, when executed by the processor, implements the method provided in any embodiment of this application.

[0016] Thirdly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the method provided in any embodiment of this application.

[0017] The method provided in this application has the following beneficial effects: 1. By eliminating the limitations of the Lambertian volume assumption through pixel-level response calibration and region adaptive projection, the point cloud integrity rate in complex scenes is >95%.

[0018] 2. By using a multi-optical-mechanical parallel mechanism, the projection time becomes independent of the number of optical-mechanical units, ensuring that the production line cycle time is <0.5s / frame.

[0019] 3. Its anti-reflective / light-absorbing interference capabilities make it suitable for cross-domain scenarios such as metal processing, semiconductor packaging, and biological tissues.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the structured light imaging system provided in one embodiment of this application; Figure 2 This is a schematic diagram of an existing global projection pattern; Figure 3 This is a schematic flowchart illustrating the steps of a partitioned structured light reconstruction method provided in an embodiment of this application; Figure 4 This is a schematic diagram comparing the grayscale distortion of target areas with different reflective properties under global fringe projection, provided in an embodiment of this application. Figure 5 This is a schematic diagram illustrating the effect of different regions after brightness response calibration and grayscale correction according to an embodiment of this application; Figure 6 This is a schematic diagram of the reconstruction effect of adaptive frequency-segmented stripe projection of a partitioned ROI provided in an embodiment of this application; Figure 7 This is a schematic block diagram of a partitioned structured light reconstruction system provided in one embodiment of this application; Figure 8 This is a schematic block diagram of the structure of a computer device provided in an embodiment of this application.

[0023] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0026] It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0027] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0028] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0029] In fields such as industrial manufacturing (e.g., high-precision industrial inspection, industrial measurement, robot vision guidance) and scientific research and medical care (e.g., dental scanning, cosmetic surgery, microscopic imaging), non-contact optical 3D (3D) reconstruction technology has gradually become an industry necessity. This is because, compared to 2D image data, 3D point cloud data can provide more dimensional information to meet the needs of high-precision inspection, measurement, and positioning.

[0030] Various technologies can achieve optical 3D reconstruction, such as stereovision, time-of-flight (ToF), and structured light. Among them, phase-shifted stripe method (PSM) is known for its high precision and robustness and is widely used in the aforementioned industrial manufacturing and scientific research and medical fields. This patent is a universal improvement on PSM technology.

[0031] Currently, 3D reconstruction schemes based on PSM technology mainly project multiple sets of sinusoidal fringe patterns with phase changes onto the target object through a projector (hereinafter referred to as an optical engine). The camera simultaneously acquires images, and then calculates phase information based on the acquired fringe images. Finally, the height information of the target object is obtained through a phase height mapping model or triangulation method. A typical system includes a single camera and a single projection device.

[0032] In the field of optical 3D reconstruction, phase-shifting fringe structured light technology (PSM) is widely used in industrial inspection, medical imaging, and other scenarios due to its high precision. For example... Figure 2As shown, traditional PSM technology projects a globally uniform sinusoidal fringe pattern (the rectangular and circular areas in the figure) through a projector, relying on the assumption that the surface of the target object is a Lambertian body. After the camera acquires the image, it extracts phase information based on a fixed model.

[0033] The light emitted by the optical engine is reflected by the target object and then reaches the camera. In other words, the final image obtained by the camera is modulated by the target object. Mathematically, this can be expressed as... ; Where F is the modulation function. The intensity of the pattern received by the camera. The intensity of the pattern projected by the optical engine.

[0034] The PSM algorithm calculates the phase based on the stripe image received by the camera, and is based on the following assumptions: ; Here, A and B are constants independent of time t. That is, the PSM algorithm is based on the assumption that the pattern modulated by the target object still retains sinusoidal characteristics, but this assumption only holds if the target object surface is subjected to Lambertian scattering.

[0035] However, in practical applications, target objects often have non-Lambertian surfaces (such as metals, highly reflective coatings, and black light-absorbing materials), complex geometries (deep holes, steep slopes), and dynamic scene requirements, which renders the above assumptions invalid. Therefore, this global PSM algorithm has the following drawbacks: 1. Grayscale distortion caused by material differences: Non-uniform reflection characteristics cause overexposed or underexposed areas in camera images, phase extraction fails, and the missing rate of 3D point clouds can reach 15%-30%; 2. Depth of field limitation and high-frequency fringe blur: The modulation transfer function (MTF) of high-frequency fringe in the region outside the depth of field decreases by more than 50%, and the phase signal-to-noise ratio deteriorates; 3. Secondary reflection interference: Mutual reflection in highly reflective areas leads to phase information errors, significantly affecting the inspection accuracy of precision parts; 4. Insufficient timeliness of multi-optical-mechanical systems: Traditional solutions require serial mapping, which cannot meet real-time requirements in dynamic scenarios.

[0036] Therefore, a method is urgently needed to solve at least one of the above problems.

[0037] To solve the above problem, please refer to Figure 3 This application provides a partitioned structured light reconstruction method applied to computer equipment. The computer equipment can be an industrial control computer, an embedded processing unit, a server cluster, or integrated into a handheld 3D scanning device, an industrial inspection robot, a medical imaging device, or other terminals.

[0038] The structured light imaging system upon which this method is based is, for example... Figure 1 As shown, the system includes an imaging acquisition unit (camera), a first projection unit (optical engine 1), a second projection unit (optical engine 2), and a computing and processing unit. Optical engine 1 and optical engine 2 are respectively arranged on the left and right sides of the camera. The light output port of the optical engine faces the target object and is used to project preset grayscale patterns, stripe patterns, and other structured light onto the surface of the target object. The imaging lens of the camera faces the target object and is used to acquire the structured light image reflected from the surface of the target object. The camera, optical engine 1, and optical engine 2 are all electrically connected to the computing and processing unit, which is used to output projection control commands, synchronously trigger acquisition, and perform calibration, clustering, and 3D reconstruction calculations on the acquired image data.

[0039] The provided partitioned structured light reconstruction method includes steps S101 to S105. Details are as follows: Step S101. Establish a mapping relationship between camera pixel coordinates and optical engine pixel coordinates, and use the mapping relationship as the basis for calculation.

[0040] Specifically, the core purpose of this step is to establish a pixel-level coordinate correspondence between the camera imaging plane and the optical engine projection plane, which serves as the spatial reference for all subsequent calibrations, region mappings, and adaptive projections.

[0041] In practical implementation, the basic calibration of the system is first completed: the camera's intrinsic parameters are calibrated to obtain the camera's focal length, principal point coordinates, radial distortion and tangential distortion coefficients; the optical engine's intrinsic parameters are calibrated, and the optical engine is used as a reverse imaging system. By projecting calibration patterns and combining them with the camera's acquisition results, the intrinsic and distortion parameters of the optical engine are solved.

[0042] Subsequently, a coordinate mapping was established using stereo calibration: a standard calibration board was placed in multiple different poses within the measurement space, and an coded calibration pattern was projected by the optomechanical system in each pose, while the camera simultaneously acquired images. Decoding was used to obtain the pixel coordinates of feature points on the calibration board in the camera images, as well as the corresponding pixel coordinates of these feature points in the optomechanical projection plane, resulting in multiple sets of one-to-one coordinate point pairs. Based on these coordinate point pairs, and combined with the intrinsic parameters of the camera and optomechanical system, the rotation matrix and translation vector between the camera and optomechanical system, i.e., the binocular extrinsic parameters, were calculated.

[0043] Based on the extrinsic and intrinsic parameter matrices, the homography mapping matrix from camera pixels to optical engine pixels is further derived. For any pixel coordinate in the camera plane, its corresponding projected coordinates in the optical engine plane can be calculated using the mapping matrix. Since the mapped coordinates are mostly sub-pixel level non-integer values, a bicubic interpolation algorithm is used to optimize the pixel values ​​of the optical engine projection, ensuring the accuracy of the coordinate mapping and eliminating discretization errors. The final sub-pixel level coordinate mapping relationship will serve as the unified calculation basis for subsequent calculations of the luminance transfer function, frequency transfer function, and ROI region projection transformation.

[0044] Step S102. Perform brightness response calibration on the target object to obtain the brightness transfer function of the optical engine corresponding to the camera pixel.

[0045] Specifically, the core purpose of this step is to quantify the reflection characteristics of each location on the surface of the target object, establish a functional relationship between the grayscale projected by the optomechanical system and the grayscale acquired by the camera, and provide a basis for subsequent adaptive grayscale adjustment.

[0046] In practice, the grayscale gradient pattern sequence is first generated: within the full grayscale range of the optomechanical system, multiple frames of uniform grayscale patterns are generated, with grayscale values ​​increasing linearly from the lowest to the highest, covering the complete grayscale output range; the number of patterns can be set according to the calibration accuracy requirements, and the grayscale step is uniform to ensure the calibration resolution.

[0047] Subsequently, synchronous acquisition is performed by controlling the optical engine to project each uniform grayscale pattern sequentially. After each frame is projected, the camera is triggered to expose synchronously, acquiring the reflection image of the target object at the corresponding grayscale level. Throughout the acquisition process, the camera's exposure time, gain, and the optical engine's output power, among other hardware parameters, are kept constant to avoid calibration errors introduced by fluctuations in hardware parameters.

[0048] Next, pixel-by-pixel function fitting is performed. For each pixel of the camera, its acquired grayscale value under all grayscale gradient patterns is extracted. With the optical engine projection grayscale as the independent variable and the camera acquired grayscale as the dependent variable, the least squares method is used to fit the function and obtain the luminance transfer function corresponding to that pixel. For Lambertian surfaces, a linear function is used to fit the function to obtain the gain coefficient and offset coefficient. For surfaces with nonlinear reflections, a second-order or third-order polynomial is used to fit the function to improve the matching accuracy.

[0049] By using the luminance transfer function, we can both forward predict the camera's captured grayscale under any projected grayscale and backward solve the projected grayscale required for the camera to work in the optimal grayscale range, providing a quantitative basis for solving overexposure and underexposure problems.

[0050] Step S103. Perform frequency response calibration on the target object to obtain the frequency transfer function of the optical engine corresponding to the camera pixels.

[0051] Specifically, the core purpose of this step is to quantify the modulation capability of the target object surface and system depth of field on different spatial frequency fringes, obtain the fringe frequency response characteristics at each pixel position, and provide a basis for the adaptive selection of fringe frequencies in the future.

[0052] In practical implementation, a multi-frequency standard stripe sequence is first generated by pre-setting multiple sets of standard sinusoidal stripe patterns with different spatial frequencies, covering the effective working range of the system from low to high frequencies. Each set of stripes has a fixed, known input contrast. The input contrast is defined as the difference between the maximum and minimum gray levels of the stripes, divided by the sum of the maximum and minimum gray levels, used to quantify the degree of difference in brightness between the stripes.

[0053] Subsequently, fringe image acquisition was performed by controlling the optical engine to project each set of sinusoidal fringes sequentially from low to high frequency. The camera simultaneously acquired the fringe images after reflection from the target object, resulting in multiple sets of fringe acquisition images at different frequencies. The hardware parameters during the acquisition process were kept consistent with those during brightness calibration to ensure uniformity of calibration conditions.

[0054] Next, the frequency response is calculated pixel by pixel. For each pixel of the camera, a local neighborhood window is taken around it, and the maximum and minimum gray levels of the stripes within that window are counted to calculate the output contrast at that frequency. The ratio of the output contrast to the input contrast is the modulation transfer coefficient at that frequency. After traversing all test frequencies, a continuous frequency transfer function, i.e., the modulation transfer function curve, is obtained by interpolation fitting with the stripe spatial frequency as the abscissa and the modulation transfer coefficient as the ordinate.

[0055] The frequency transfer function directly reflects the clarity of different frequency stripes at the pixel location. The higher the function value, the better the stripe contrast, the higher the signal-to-noise ratio of phase extraction, and the more guaranteed the reconstruction accuracy.

[0056] Step S104. Cluster the camera pixels according to the frequency transfer function and the brightness transfer function to divide the camera pixels into target regions that include different regional features.

[0057] Specifically, the core purpose of this step is to divide pixels with similar optical characteristics in the camera image into the same region based on the two-dimensional calibration results, thereby achieving automatic zoning of the target region and providing a foundation for subsequent regional adaptive projection.

[0058] In practical implementation, the pixel feature vector is first constructed. For each camera pixel, parameters such as gain coefficient, offset coefficient, and fitting linearity are extracted from the luminance transfer function, and parameters such as optimal operating frequency, cutoff frequency, and peak modulation coefficient are extracted from the frequency transfer function. Together, they form a multi-dimensional feature vector that comprehensively characterizes the optical response characteristics of the pixel.

[0059] Then, feature normalization is performed: Since the dimensions and numerical ranges of different feature parameters are different, the feature vectors of all pixels are standardized, and the feature values ​​of each dimension are converted into a uniform scale to eliminate the influence of the difference in dimensions on the clustering results.

[0060] Next, unsupervised clustering is performed by classifying the feature vectors of all pixels in the image using a clustering algorithm, grouping pixels with similar features into the same cluster. The clustering algorithm can be selected based on the complexity of the scene: for scenes with well-defined characteristics, the K-means algorithm is used with a preset number of clusters; for complex and unknown scenes, DBSCAN density clustering is used to automatically identify the categories while filtering out noisy pixels.

[0061] Finally, the generated region mask is produced by performing morphological erosion and dilation on each pixel class obtained from clustering to remove isolated noise points and merge connected regions of the same class, generating a binarized region mask. Each mask corresponds to a target region, and pixels within the same target region have similar brightness and frequency response characteristics, allowing for adaptation using the same set of projection parameters.

[0062] Step S105. Obtain the projection parameters of the target area based on the luminance transfer function, frequency transfer function and mapping relationship. The projection parameters include the projection frequency and projection grayscale. Perform ROI adaptive projection on the target area based on the projection parameters so that the corresponding target area meets the requirements of the PSM algorithm for partitioned structured light reconstruction.

[0063] Specifically, the core purpose of this step is to customize and adapt the phase-shift fringe parameters according to the optical characteristics of each target region, and to achieve partitioned projection by the region of interest projection method, so that the fringes of all regions satisfy the sinusoidal assumption of the phase-shift fringe method, and finally complete the high-precision three-dimensional reconstruction.

[0064] In practice, the optimal projection parameters for each region are first calculated. For each target region, based on the brightness transfer function of the pixels in the region, the average projected gray level and stripe contrast are calculated to ensure that the vast majority of pixels in the region are in the linear gray level range, thus avoiding overexposure and underexposure. Based on the frequency transfer function of the pixels in the region, the stripe spatial frequency with the best overall signal-to-noise ratio for the region is selected to ensure stripe clarity.

[0065] Subsequently, ROI pattern mapping and generation are performed. By converting the mask of each target region into an ROI region on the optomechanical projection plane through the coordinate mapping relationship in step S101, a multi-frame phase-shifted fringe pattern is generated within the corresponding region based on the projection parameters of that region. Typically, a four-step phase-shifting method is used to generate four frames of fringes with phase shifts of 90 degrees each. The fringe patterns of all regions are stitched together to form a complete projection image, and the fringe parameters of different regions are independent of each other.

[0066] Then, synchronous projection and acquisition are performed. By controlling the optical engine to sequentially project multiple frames of phase-shifted fringe patterns, the camera simultaneously exposes and acquires the images, resulting in the corresponding number of fringe images. For multi-optical-engine systems, different optical engines can project fringes into different areas, achieving parallel projection, improving acquisition speed, and adapting to dynamic scene requirements.

[0067] Finally, partition reconstruction and point cloud fusion are performed. By combining the masks of each region, PSM phase calculation is performed on each target region. The wrapping phase is calculated by the phase shift formula, and the absolute phase is obtained by unwrapping through multi-frequency heterodyne and other methods. Based on the pre-calibrated phase-height mapping relationship or the principle of triangulation, the phase is converted into three-dimensional coordinates to obtain the three-dimensional point cloud of each region. The point clouds of all regions are stitched and fused to finally obtain the complete three-dimensional reconstruction result of the target object.

[0068] For example, the optimal projection parameters for a target region are calculated by statistically analyzing the brightness and frequency characteristics of all pixels within each target region. Grayscale parameters: Take the intersection of the safe grayscale ranges of all pixels in the area to determine the upper and lower limits of the projected grayscale of the area, calculate the average grayscale and contrast of the stripes, and ensure that the grayscale of more than 95% of the pixels in the area is within the linear safe range, while the stripes have sufficient brightness difference. Frequency parameters: The median of the optimal operating frequencies of all pixels in the region is taken as the projection fringe frequency of the region, taking into account the sharpness requirements of most pixels in the region and ensuring the overall phase signal-to-noise ratio.

[0069] The binary mask of each target region is transformed pixel by pixel onto the optical-mechanical projection plane through the camera-optical-mechanical pixel mapping relationship established in step S101, resulting in the ROI region mask from the optical-mechanical perspective. For each ROI region, a four-step phase-shifting sinusoidal fringe pattern is generated based on the calculated average grayscale, contrast, and fringe frequency. The phases of the four frames are 0, π / 2, π, and 3π / 2, respectively.

[0070] The stripe patterns of all regions are stitched together using a mask to generate a complete optical-mechanical projection image: the areas within the mask are stripes with corresponding parameters, while the areas outside the mask are set as a low-grayscale background to avoid light interference from irrelevant areas. The stripe parameters of different ROI regions are independent of each other and do not affect each other.

[0071] The computational processing unit controls the optical engine to sequentially project four frames of phase-shifted fringe patterns. Each frame is projected, and a trigger signal is output to control the camera to synchronously expose and acquire the images, resulting in four frames of fringe images. For a dual-optical-engine system, different target areas can be assigned to two optical engines for projection. The two optical engines project images synchronously, and the camera can acquire fringe information for two areas in a single acquisition, reducing the number of projections by half, significantly improving acquisition efficiency, and adapting to the real-time requirements of dynamic scenes.

[0072] By combining the four frames of striped images acquired with the masks of each target region, PSM phase calculation is performed independently for each region: Wrapping phase calculation: The wrapping phase value of each pixel in the region is calculated using a four-step phase shift formula; Phase unwrapping: The absolute phase map is obtained by using the multi-frequency heterodyne method and combining the fringe phases of different frequencies. 3D coordinate transformation: Based on the pre-calibrated phase-height mapping relationship, or the camera-optical-mechanical triangulation principle, the absolute phase of each pixel is converted into 3D spatial coordinates to obtain the 3D point cloud of the region.

[0073] Finally, the 3D point clouds of all target areas are stitched together, deduplicated, and smoothed to merge them into a complete 3D point cloud of the target object, thus completing the entire partitioned structured light reconstruction process.

[0074] In some embodiments, establishing the mapping relationship between camera pixel coordinates and optical-mechanical pixel coordinates includes: obtaining camera pixel coordinates and corresponding optical-mechanical pixel coordinates; establishing a sub-pixel-level mapping relationship between camera pixel coordinates and optical-mechanical pixel coordinates through bi-target calibration, while using bicubic interpolation to optimize the non-integer coordinate problem as the basis for calculating the corresponding luminance transfer function and frequency transfer function.

[0075] First, the camera's intrinsic parameters are calibrated. Using a checkerboard calibration board, images of the calibration board are acquired by changing at least 20 different poses in the measurement space. Through corner point extraction and Zhang Zhengyou's calibration algorithm, the camera's intrinsic parameter matrix is ​​obtained, which includes the equivalent focal length in the x-axis and y-axis directions, the coordinates of the principal imaging point, as well as the radial distortion coefficient and the tangential distortion coefficient, thus completing the calibration of the camera's intrinsic parameters and distortion.

[0076] Subsequently, the intrinsic parameters of the optical engine were calibrated. By treating the optical engine as a reverse imaging system, a checkerboard pattern was projected from the optical engine onto the calibration board, and the camera captured the reflected pattern. By extracting the coordinates of the corner points of the projected checkerboard in the camera image, the intrinsic parameters and distortion parameters of the optical engine were solved in reverse, and the intrinsic parameter matrix and distortion coefficients of the optical engine were obtained, thus completing the intrinsic parameter calibration of the optical engine.

[0077] A checkerboard calibration board is placed within the measurement working distance range, and at least 15 different poses are sequentially changed. In each pose, the optomechanical system projects a coded calibration pattern combining Gray code and phase-shifting fringes, while the camera simultaneously acquires images. Through Gray code decoding and phase calculation, the sub-pixel coordinates of each feature point on the calibration board plane in the camera image are obtained. Simultaneously, the corresponding coordinates of the feature point in the optomechanical projection plane are decoded, ultimately obtaining at least hundreds of one-to-one camera-optomechanical pixel coordinate pairs.

[0078] Based on the acquired coordinate pairs and the intrinsic parameter matrices of the camera and optical engine, the rotation matrix R and translation vector T from the camera coordinate system to the optical engine coordinate system are obtained through a binocular stereo calibration algorithm; these are the extrinsic parameters of the camera and optical engine. Based on the intrinsic and extrinsic parameters, a mapping matrix from the camera pixel coordinate system to the optical engine pixel coordinate system is established through homography transformation. For any pixel coordinate (u, v) in the camera plane, its corresponding projected coordinates (x, y) in the optical engine plane can be obtained through matrix operations.

[0079] Since the optical-mechanical coordinates obtained from mapping are usually non-integer sub-pixel values, direct rounding would introduce coordinate errors. Therefore, a bicubic interpolation algorithm is used to interpolate the pixel values ​​of the optical-mechanical projection pattern. Bicubic interpolation, by considering the grayscale values ​​of the 16 neighboring pixels around the target point for weighted calculation, can preserve the details and smoothness of the pattern, ultimately achieving sub-pixel level accuracy in coordinate mapping, providing a high-precision coordinate reference for subsequent calibration and projection.

[0080] In some embodiments, the step of calibrating the brightness response of the target object to obtain the brightness transfer function of the optical engine corresponding to the camera pixel includes: controlling the optical engine to project a grayscale gradient pattern onto the target object and obtaining the average grayscale of the optical engine projection pattern; obtaining the grayscale of the camera pixel based on the grayscale gradient pattern received by the camera; obtaining the gain coefficient and the offset coefficient based on the average grayscale of the optical engine projection pattern and the grayscale of the camera pixel; and fitting a brightness response calibration curve based on the gain coefficient and the offset coefficient to calibrate the brightness response of the target object and obtain the brightness transfer function of the optical engine corresponding to the camera pixel.

[0081] By generating 32 uniform grayscale projection patterns, the grayscale values ​​are linearly and uniformly distributed from 0 to 255, with a grayscale step of 8 between adjacent frames, covering the complete 8-bit grayscale output range of the optical engine. All patterns are uniformly filled across the entire screen without any texture or stripes, ensuring that the grayscale of the incident light received at every position on the target object's surface is uniform and consistent.

[0082] By placing the target object at the measurement station, the computational processing unit controls the optomechanic to sequentially project 32 uniform grayscale patterns in ascending order of grayscale. After each pattern is projected, a trigger signal is output to control the camera to perform synchronous exposure and acquisition, resulting in 32 frames of target object reflection images corresponding to different projected grayscale levels. Throughout the acquisition process, the camera's exposure time and gain parameters remain fixed, the optomechanic's output power remains fixed, and the ambient lighting conditions remain stable to avoid external factors interfering with the calibration results.

[0083] For each pixel on the camera's imaging plane, its grayscale value in 32 acquired images is extracted in frame order to form a grayscale response dataset for that pixel: the independent variable is the optical engine projection grayscale Ip=[0, 8, 16, ..., 255], and the dependent variable is the camera-acquired grayscale Ic of that pixel in the corresponding frame.

[0084] The luminance transfer function is fitted by performing a linear fit using the least squares method on the grayscale response dataset for each pixel, resulting in a linear luminance transfer function: Ic = k × Ip + b; Where k is the gain coefficient corresponding to the pixel, the larger the value, the higher the reflectivity of the target object at that position; b is the offset coefficient corresponding to the pixel, reflecting the basic grayscale shift caused by ambient light and camera dark current.

[0085] For materials with nonlinear surface reflections, such as polished metals and gradient coatings, linear fitting has a large error. In this case, a second-order polynomial is used for fitting. Ic = a × Ip 2 +k×Ip+b; Where 'a' is the coefficient of the quadratic term, used to characterize the nonlinearity of reflection. Polynomial fitting can significantly improve the accuracy of the brightness transfer function's adaptation to complex reflective surfaces, ensuring the accuracy of grayscale prediction.

[0086] By following the steps above, the independent luminance transfer function for each pixel in the entire camera frame can be obtained, enabling precise quantification of the point-by-point luminance response characteristics of the target object's surface.

[0087] In some embodiments, the step of calibrating the brightness response of the target object to obtain the brightness transfer function of the optical engine corresponding to the camera pixel further includes: obtaining a safe range of grayscale projected by the optical engine; setting a saturation threshold of the camera according to the safe range; and setting constraints on the optical engine according to the saturation threshold, so that the operation of the optical engine and the camera are both within the safe range.

[0088] The safe grayscale range setting for the optical engine's projection is determined by setting the effective working grayscale range of the optical engine based on its hardware output characteristics. When the optical engine operates near its extreme grayscale values ​​(0 or 255), problems such as output nonlinearity and accelerated brightness decay will occur. Therefore, the safe projection grayscale range of the optical engine is set to [10, 245], which covers most grayscale ranges while ensuring that the optical engine operates in a linear and stable output range.

[0089] The camera grayscale threshold setting is based on the camera's photosensitive characteristics, setting the camera's saturation threshold and underexposure threshold: setting the saturation threshold to 240 means that when the camera pixel grayscale is higher than this value, the photosensitive element enters the saturation region, and the grayscale is no longer linearly related to the incident light intensity, resulting in overexposure; setting the underexposure threshold to 20 means that when the pixel grayscale is lower than this value, the signal is overwhelmed by noise, the signal-to-noise ratio is too low, and the phase extraction requirements cannot be met.

[0090] The derivation of the per-pixel safe projection range is achieved by calculating the corresponding optical engine projection grayscale range when the camera grayscale is in the range of [20, 240] for each camera pixel based on its luminance transfer function. Then, the intersection of this range with the safe projection range of the optical engine [10, 245] is taken to obtain the effective working grayscale range of the pixel, which is the optical engine projection grayscale constraint condition corresponding to the pixel.

[0091] Constraint integration integrates the safe grayscale range of each pixel into the parameters of the luminance transfer function. When calculating the projection parameters of the region in the subsequent calculation, the projection grayscale is set with the safe range as the boundary, ensuring that the projection grayscale of all regions meets the working requirements of the camera and optical engine, thereby avoiding overexposure and underexposure problems from the root, and ensuring the working stability and service life of the optical engine.

[0092] In some embodiments, the step of calibrating the frequency response of the target object to obtain the frequency transfer function of the optical engine corresponding to the camera pixel includes: controlling the optical engine to project a standard pattern, the standard pattern including an input contrast; capturing the standard pattern with a camera to obtain an output contrast; and calculating the frequency transfer function of the optical engine corresponding to the camera pixel based on the input contrast and the output contrast.

[0093] Multi-frequency standard stripe generation uses 10 preset sinusoidal stripe patterns with different spatial frequencies. The stripe frequencies are distributed logarithmically, ranging from 2 cycles / frame to 128 cycles / frame, covering the entire working range from low to high frequencies. Each stripe group has a standard sinusoidal distribution, with an average grayscale set to 128 and a uniform input contrast ratio of 0.8. This means the maximum grayscale value of the stripes is 230.4 and the minimum grayscale value is 25.6, ensuring that the input stripes have standard sinusoidal characteristics and sufficient contrast.

[0094] The fringe image synchronous acquisition involves controlling the optical engine to project 10 sets of sinusoidal fringe patterns sequentially from low to high frequency. The camera simultaneously acquires the fringe images reflected from the target object's surface, resulting in 10 fringe acquisition images at different frequencies. During the acquisition process, all hardware parameters of the camera and optical engine are completely consistent with those in the brightness calibration stage, ensuring uniformity between the two calibrations and improving the accuracy of subsequent parameter matching.

[0095] Pixel-by-pixel output contrast calculation involves selecting a 7×7 local neighborhood window around each pixel in the stripe image for each frequency group, statistically analyzing the grayscale values ​​within the window to obtain the maximum grayscale value Icmax and the minimum grayscale value Icmin, and then calculating the output contrast at that pixel location at that frequency. C_out=(Icmax-Icmin) / (Icmax+Icmin); Using a local window calculation method can avoid noise interference from individual pixels and improve the stability of contrast calculation.

[0096] For each pixel, the spatial frequency of 10 sets of stripes is used as the horizontal axis, and the modulation transfer factor MTF=C_out / C_in at the corresponding frequency is used as the vertical axis to obtain 10 discrete frequency response points. The discrete points are fitted by a cubic spline interpolation algorithm to obtain a continuous frequency transfer function curve, that is, the modulation transfer function (MTF) curve of the pixel.

[0097] This frequency transfer function can intuitively reflect the degree of contrast retention of stripes of different spatial frequencies at the pixel position. The higher the MTF value, the clearer the stripes and the higher the accuracy of phase calculation.

[0098] In some embodiments, the step of calibrating the frequency response of the target object to obtain the frequency transfer function of the optical engine corresponding to the camera pixel further includes: obtaining the fringe frequency corresponding to each pixel according to the frequency transfer function; finding the optimal frequency based on the multiple fringe frequencies of the pixel, and the pixel having the highest sharpness when it is at the optimal frequency.

[0099] Based on the frequency transfer function of each pixel, the MTF values ​​of that pixel at all test fringe frequencies are extracted to form the MTF value sequence of that pixel. The MTF value directly corresponds to the image sharpness and phase signal-to-noise ratio of the fringe, and is a core indicator that determines the accuracy of phase extraction.

[0100] For each pixel, within the effective range of its frequency transfer function, the fringe spatial frequency corresponding to the peak value of the MTF is selected as the optimal operating frequency for that pixel. When the fringe frequency equals the optimal operating frequency, the fringe sharpness at that pixel location is the highest, the signal-to-noise ratio of phase extraction is optimal, and the reconstruction error is minimal.

[0101] For pixels located at the edge of depth of field, steep slopes, or deep holes, the MTF of their high-frequency fringes will decrease sharply, with the peak MTF appearing in the low-frequency range. Therefore, their optimal operating frequency is a lower frequency to ensure that the fringes have sufficient contrast. For pixels located at the optimal working distance and with flat surfaces, their high-frequency MTF remains at a high level, and the optimal operating frequency can correspond to a higher fringe frequency to obtain higher phase resolution and reconstruction accuracy.

[0102] The optimal operating frequency corresponding to each pixel is stored as part of the pixel feature vector for subsequent region clustering. It also serves as the core basis for selecting frequency parameters in subsequent adaptive projection, ensuring that the fringe frequency projected in each region is highly adapted to the optical characteristics of that region.

[0103] In some embodiments, clustering camera pixels according to a frequency transfer function and a luminance transfer function to divide the camera pixels into target regions including different regional features includes: constructing multiple pixel feature vectors according to the luminance transfer function and the frequency transfer function; clustering the constructed multiple pixel feature vectors to cluster the captured camera pixels; constructing a region mask according to the clustered camera pixels, and dividing the camera pixels into regions including different regional features through the region mask.

[0104] For each pixel of the camera, extract 6-dimensional feature parameters to construct a pixel feature vector: Luminance dimension: gain coefficient k, offset coefficient b, and linear fitting determination coefficient R² of the luminance transfer function; Frequency dimension: optimal operating frequency f_opt, cutoff frequency f_cut (the frequency corresponding to when MTF drops to 0.1), peak MTF value MTF_peak.

[0105] The above six parameters together form a 6-dimensional feature vector V=[k, b, R] 2 [f_opt, f_cut, MTF_peak] comprehensively characterizes the brightness and frequency response characteristics of the pixel location.

[0106] Because the dimensions and numerical ranges of different features vary significantly—for example, gain coefficients are mostly between 0 and 2, while the optimal operating frequency can reach hundreds—direct clustering would result in excessively high weights for dimensions with large numerical values. Therefore, the Z-score normalization method is used to standardize each feature dimension of all pixels separately, transforming each feature into a distribution with a mean of 0 and a standard deviation of 1, thus eliminating the influence of dimensional differences.

[0107] The K-means clustering algorithm is used to cluster the standardized feature vectors of all pixels in the image. First, the number of clusters K is preset based on the surface complexity of the target object. For example, for a scene with four types of regions simultaneously existing—high reflectivity, light absorption, sharpness, and blurriness—K=4 is set. The algorithm randomly selects four initial cluster centers. By iteratively calculating the Euclidean distance from each pixel to the cluster center, the pixel is assigned to the nearest cluster. The cluster center for each cluster is then recalculated until the cluster centers no longer change, completing the clustering process and obtaining the category label for each pixel.

[0108] For complex scenarios with unknown surface characteristics, the DBSCAN density clustering algorithm is adopted. It does not require a preset number of categories. By setting the neighborhood radius and minimum number of samples, it automatically identifies dense pixel clusters and marks isolated noise pixels as outliers, thereby improving the adaptability of partitioning.

[0109] For each pixel cluster, morphological processing is performed: first, an erosion operation is performed using a 3×3 structuring element to remove isolated noise pixels from the cluster; then, a dilation operation is performed to fill small holes within the region; subsequently, connected component analysis is performed to remove connected components with excessively small areas, retaining the main connected regions. Finally, a binary region mask is generated for each pixel cluster, where white areas represent the range of the corresponding target region, and black areas represent areas outside the target region.

[0110] Through the above process, the entire camera frame is divided into several non-overlapping target regions. The pixels in each region have highly similar optical response characteristics, providing accurate region boundaries for subsequent partitioned adaptive projection.

[0111] In some embodiments, the drawbacks of conventional global projection include... Figure 4 As shown, the target object's surface contains two regions with significantly different characteristics: a rectangular region representing a highly reflective, bright surface, and a circular region representing a low-reflectivity, dark surface. When using sinusoidal fringe projection with globally uniform parameters, if the parameters are set according to the dark area, the rectangular highly reflective area in the left image will be severely overexposed, with the fringe grayscale approaching saturation, thus destroying the sinusoidal characteristics. Conversely, if the parameters are set according to the bright area, the circular dark area in the right image will be severely underexposed, with extremely low fringe contrast, making it impossible to extract effective phase. Since neither type of region can simultaneously meet the requirements of the PSM algorithm using the same set of global fringes, one type of region will inevitably fail to be reconstructed, resulting in missing point clouds.

[0112] To address this issue, luminance response calibration was first performed to obtain the luminance transfer functions for the rectangular and circular regions. The gain coefficient of the rectangular region was significantly larger than that of the circular region, indicating that, for the same projected grayscale, the acquired grayscale of the rectangular region was much higher than that of the circular region. Subsequently, frequency response calibration was performed to obtain the frequency transfer functions for the two regions, thus determining their respective optimal operating frequencies. Based on the calibration results, pixel clustering was performed, automatically dividing the rectangular and circular regions into two independent target regions, and generating corresponding region masks for each.

[0113] Then, adaptive grayscale correction was performed, and the result was as follows. Figure 5 As shown: For the highly reflective rectangular area on the left, the projected average gray level and fringe contrast are reduced according to its luminance transfer function, causing the camera-captured gray level in this area to fall back from saturation to the linear range, and the fringes to restore a standard sinusoidal distribution. For the low-reflectivity circular area on the right, the projected average gray level and fringe contrast are increased according to its luminance transfer function, causing the captured gray level in this area to rise from underexposed to the effective range, and the fringe contrast to meet the phase extraction requirements. Through regional adaptive gray level adjustment, the fringes in both areas simultaneously possess good sinusoidal characteristics, satisfying the premise assumptions of the PSM algorithm.

[0114] Based on this, further frequency-adaptive projection of different regions is performed, with the following effect: Figure 6 As shown. The dashed box on the left represents the two target regions obtained from clustering. Based on the frequency characteristics of the two regions, appropriate fringe frequencies are selected respectively: for regions with flat surfaces and good high-frequency response, high-frequency fringes are used to improve reconstruction accuracy; for regions with blurred edges and poor high-frequency response, low-frequency fringes are used to ensure phase stability.

[0115] The right side shows the effect after frequency-band ROI projection. The rectangular area is further divided into two sub-regions, 1 and 2, which project stripes of different frequencies. The circular area is also divided into two sub-regions, 1 and 2, each using its own adapted frequency parameters. Through adaptive projection by frequency band, the stripes in each sub-region operate in an optimal state, balancing reconstruction accuracy and robustness.

[0116] Finally, PSM phase calculation is performed independently for each partition. The phase results of each region have a high signal-to-noise ratio. After stitching and fusing the 3D point clouds of each region, a complete and high-precision 3D reconstruction result of the target object can be obtained. This scheme effectively solves the problem that different characteristic regions in traditional global projection cannot simultaneously meet the PSM requirements. It can significantly reduce the point cloud missing rate of complex surface targets, improve reconstruction accuracy and system adaptability, and can be widely used in various scenarios such as industrial inspection and medical imaging.

[0117] Please see Figure 7 As shown, Figure 7This is a schematic diagram of the partitioned structured light reconstruction system 200 provided in this application embodiment. The partitioned structured light reconstruction system 200 is used to execute the steps of the partitioned structured light reconstruction method shown in the above embodiments. The partitioned structured light reconstruction system 200 can be a single server or a server cluster, or it can be a terminal, such as a handheld terminal, a laptop computer, a wearable device, or a robot.

[0118] like Figure 7 As shown, the partitioned structured light reconstruction system 200 includes: The relationship establishment unit 201 is used to establish a mapping relationship between camera pixel coordinates and optical engine pixel coordinates, and to use the mapping relationship as the basis for calculation. The first acquisition unit 202 is used to perform brightness response calibration on the target object and obtain the brightness transfer function of the optical engine corresponding to the camera pixel; The second acquisition unit 203 is used to perform frequency response calibration on the target object in order to obtain the frequency transfer function of the optical engine corresponding to the camera pixel. The region segmentation unit 204 is used to cluster camera pixels according to the frequency transfer function and the brightness transfer function, so as to divide the camera pixels into target regions including different regional features. The partition reconstruction unit 205 is used to obtain the projection parameters of the target area according to the luminance transfer function, frequency transfer function and mapping relationship. The projection parameters include the projection frequency and the projection gray level. Based on the projection parameters, the target area is subjected to ROI adaptive projection so that the corresponding target area meets the requirements of the PSM algorithm for partition structured light reconstruction.

[0119] In some embodiments, establishing the mapping relationship between camera pixel coordinates and optical-mechanical pixel coordinates includes: obtaining camera pixel coordinates and corresponding optical-mechanical pixel coordinates; establishing a sub-pixel-level mapping relationship between camera pixel coordinates and optical-mechanical pixel coordinates through bi-target calibration, while using bicubic interpolation to optimize the non-integer coordinate problem as the basis for calculating the corresponding luminance transfer function and frequency transfer function.

[0120] In some embodiments, the step of calibrating the brightness response of the target object to obtain the brightness transfer function of the optical engine corresponding to the camera pixel includes: controlling the optical engine to project a grayscale gradient pattern onto the target object and obtaining the average grayscale of the optical engine projection pattern; obtaining the grayscale of the camera pixel based on the grayscale gradient pattern received by the camera; obtaining the gain coefficient and the offset coefficient based on the average grayscale of the optical engine projection pattern and the grayscale of the camera pixel; and fitting a brightness response calibration curve based on the gain coefficient and the offset coefficient to calibrate the brightness response of the target object and obtain the brightness transfer function of the optical engine corresponding to the camera pixel.

[0121] In some embodiments, the step of calibrating the brightness response of the target object to obtain the brightness transfer function of the optical engine corresponding to the camera pixel further includes: obtaining a safe range of grayscale projected by the optical engine; setting a saturation threshold of the camera according to the safe range; and setting constraints on the optical engine according to the saturation threshold, so that the operation of the optical engine and the camera are both within the safe range.

[0122] In some embodiments, the step of calibrating the frequency response of the target object to obtain the frequency transfer function of the optical engine corresponding to the camera pixel includes: controlling the optical engine to project a standard pattern, the standard pattern including an input contrast; capturing the standard pattern with a camera to obtain an output contrast; and calculating the frequency transfer function of the optical engine corresponding to the camera pixel based on the input contrast and the output contrast.

[0123] In some embodiments, the step of calibrating the frequency response of the target object to obtain the frequency transfer function of the optical engine corresponding to the camera pixel further includes: obtaining the fringe frequency corresponding to each pixel according to the frequency transfer function; finding the optimal frequency based on the multiple fringe frequencies of the pixel, and the pixel having the highest sharpness when it is at the optimal frequency.

[0124] In some embodiments, clustering camera pixels according to a frequency transfer function and a luminance transfer function to divide the camera pixels into target regions including different regional features includes: constructing multiple pixel feature vectors according to the luminance transfer function and the frequency transfer function; clustering the constructed multiple pixel feature vectors to cluster the captured camera pixels; constructing a region mask according to the clustered camera pixels, and dividing the camera pixels into regions including different regional features through the region mask.

[0125] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the partitioned structured light reconstruction system and its modules described above can be referred to the corresponding content in the various embodiments of the partitioned structured light reconstruction method, and will not be repeated here.

[0126] The aforementioned partitioned structured light reconstruction method can be implemented as a computer program, which can be used in, for example... Figure 7 It runs on the device shown.

[0127] Please see Figure 8 , Figure 8 This is a schematic block diagram of the structure of a computer device provided in an embodiment of this application. The computer device includes a processor, a memory, and a network interface connected via a device bus, wherein the memory may include a storage medium and internal memory.

[0128] The storage medium can store operating devices and computer programs. The computer program includes program instructions that, when executed, cause the processor to perform any partitioned structured light reconstruction method.

[0129] The processor provides computing and control capabilities, supporting the operation of the entire computer device.

[0130] Internal memory provides an environment for the execution of computer programs in non-volatile storage media. When the computer program is executed by the processor, it enables the processor to perform any partitioned structured light reconstruction method.

[0131] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the terminal to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0132] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0133] In one embodiment, the processor is configured to run a computer program stored in memory to perform the following steps: Establish a mapping relationship between camera pixel coordinates and optical engine pixel coordinates, and use the mapping relationship as the basis for calculation; The brightness response of the target object is calibrated to obtain the brightness transfer function of the optical engine corresponding to the camera pixel; Frequency response calibration of the target object is performed to obtain the frequency transfer function of the optical engine corresponding to the camera pixels; The camera pixels are clustered based on the frequency transfer function and the brightness transfer function to divide the camera pixels into target regions that include different regional features. The projection parameters of the target area are obtained based on the luminance transfer function, frequency transfer function and mapping relationship. The projection parameters include the projection frequency and projection grayscale. The target area is then subjected to ROI adaptive projection based on the projection parameters so that the corresponding target area meets the requirements of the PSM algorithm for partitioned structured light reconstruction.

[0134] In some embodiments, establishing the mapping relationship between camera pixel coordinates and optical-mechanical pixel coordinates includes: obtaining camera pixel coordinates and corresponding optical-mechanical pixel coordinates; establishing a sub-pixel-level mapping relationship between camera pixel coordinates and optical-mechanical pixel coordinates through bi-target calibration, while using bicubic interpolation to optimize the non-integer coordinate problem as the basis for calculating the corresponding luminance transfer function and frequency transfer function.

[0135] In some embodiments, the step of calibrating the brightness response of the target object to obtain the brightness transfer function of the optical engine corresponding to the camera pixel includes: controlling the optical engine to project a grayscale gradient pattern onto the target object and obtaining the average grayscale of the optical engine projection pattern; obtaining the grayscale of the camera pixel based on the grayscale gradient pattern received by the camera; obtaining the gain coefficient and the offset coefficient based on the average grayscale of the optical engine projection pattern and the grayscale of the camera pixel; and fitting a brightness response calibration curve based on the gain coefficient and the offset coefficient to calibrate the brightness response of the target object and obtain the brightness transfer function of the optical engine corresponding to the camera pixel.

[0136] In some embodiments, the step of calibrating the brightness response of the target object to obtain the brightness transfer function of the optical engine corresponding to the camera pixel further includes: obtaining a safe range of grayscale projected by the optical engine; setting a saturation threshold of the camera according to the safe range; and setting constraints on the optical engine according to the saturation threshold, so that the operation of the optical engine and the camera are both within the safe range.

[0137] In some embodiments, the step of calibrating the frequency response of the target object to obtain the frequency transfer function of the optical engine corresponding to the camera pixel includes: controlling the optical engine to project a standard pattern, the standard pattern including an input contrast; capturing the standard pattern with a camera to obtain an output contrast; and calculating the frequency transfer function of the optical engine corresponding to the camera pixel based on the input contrast and the output contrast.

[0138] In some embodiments, the step of calibrating the frequency response of the target object to obtain the frequency transfer function of the optical engine corresponding to the camera pixel further includes: obtaining the fringe frequency corresponding to each pixel according to the frequency transfer function; finding the optimal frequency based on the multiple fringe frequencies of the pixel, and the pixel having the highest sharpness when it is at the optimal frequency.

[0139] In some embodiments, clustering camera pixels according to a frequency transfer function and a luminance transfer function to divide the camera pixels into target regions including different regional features includes: constructing multiple pixel feature vectors according to the luminance transfer function and the frequency transfer function; clustering the constructed multiple pixel feature vectors to cluster the captured camera pixels; constructing a region mask according to the clustered camera pixels, and dividing the camera pixels into regions including different regional features through the region mask.

[0140] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the steps of the partitioned structured light reconstruction method provided in any embodiment of this application.

[0141] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiments, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, SmartMediaCard (SMC), SecureDigital (SD) card, or FlashCard equipped on the computer device.

[0142] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for reconstructing structured light in a partitioned manner, characterized in that, include: Establish a mapping relationship between camera pixel coordinates and optical engine pixel coordinates, and use the mapping relationship as the basis for calculation; The brightness response calibration of the target object is performed to obtain the brightness transfer function of the optical engine corresponding to the camera pixel. This includes: controlling the optical engine to project a grayscale gradient pattern onto the target object and obtaining the average grayscale of the projected pattern; obtaining the grayscale of the camera pixel based on the grayscale gradient pattern received by the camera; obtaining the gain coefficient and offset coefficient based on the average grayscale of the projected pattern and the grayscale of the camera pixel; and fitting a brightness response calibration curve based on the gain coefficient and offset coefficient to perform brightness response calibration on the target object and obtain the brightness transfer function of the optical engine corresponding to the camera pixel. The frequency response calibration of the target object is performed to obtain the frequency transfer function of the optical engine corresponding to the camera pixel, including: controlling the optical engine to project a standard pattern, the standard pattern including the input contrast; taking pictures of the standard pattern with the camera to obtain the output contrast; and calculating the frequency transfer function of the optical engine corresponding to the camera pixel based on the input contrast and the output contrast. The camera pixels are clustered based on the frequency transfer function and the brightness transfer function to divide the camera pixels into target regions that include different regional features. The projection parameters of the target area are obtained based on the luminance transfer function, frequency transfer function and mapping relationship. The projection parameters include the projection frequency and projection grayscale. The target area is then subjected to ROI adaptive projection based on the projection parameters so that the corresponding target area meets the requirements of the PSM algorithm for partitioned structured light reconstruction.

2. The method according to claim 1, characterized in that, The process of establishing the mapping relationship between camera pixel coordinates and optical engine pixel coordinates includes: Obtain the camera pixel coordinates and the corresponding optical-mechanical pixel coordinates; A sub-pixel-level mapping relationship between camera pixel coordinates and optical engine pixel coordinates is established through bi-target calibration. At the same time, bicubic interpolation is used to optimize the non-integer coordinate problem, which serves as the basis for the calculation of the corresponding luminance transfer function and frequency transfer function.

3. The method according to claim 1, characterized in that, The step of calibrating the brightness response of the target object to obtain the brightness transfer function of the optical engine corresponding to the camera pixels also includes: Obtain the safe range of grayscale projected by the optical engine; Set the camera's saturation threshold according to the safe range, and set the optical engine's constraints according to the saturation threshold, so that the operation of both the optical engine and the camera is within the safe range.

4. The method according to claim 1, characterized in that, The step of calibrating the frequency response of the target object to obtain the frequency transfer function of the optical engine corresponding to the camera pixels also includes: The stripe frequency corresponding to each pixel is obtained based on the frequency transfer function; The optimal frequency is determined by identifying the multiple stripe frequencies of the pixel. The pixel has the highest sharpness when it is at the optimal frequency.

5. The method according to claim 1, characterized in that, The step of clustering camera pixels based on the frequency transfer function and the luminance transfer function to divide the camera pixels into target regions including different regional features includes: Multiple pixel feature vectors are constructed based on the luminance transfer function and the frequency transfer function; Cluster the constructed pixel feature vectors to cluster the captured camera pixels; A region mask is constructed based on the clustered camera pixels, and the camera pixels are divided into regions that include different regional features using the region mask.

6. A computer device, characterized in that, The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program and, in executing the computer program, implement the method as described in any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to implement the method as described in any one of claims 1 to 5.

Images