Complex reflective object surface reconstruction method and system based on multi-system point cloud fusion

Abstract

The invention relates to the technical field of optical measurement, in particular to a complex reflective object surface reconstruction method and system based on multi-system point cloud fusion, and the method comprises the steps: carrying out the joint calibration of the spatial relation among a camera, a projector and an LCD screen; projecting to a target object based on a projection grating phase method and a phase deflection operation path, and collecting structured light patterns of the two through a camera; phase solving is carried out on the structured light pattern collected by the camera, and a three-dimensional reconstruction result is obtained in combination with the joint calibration information; and world coordinate system matching and point cloud fusion are carried out on the three-dimensional reconstruction result based on the joint calibration information, and the three-dimensional morphology of the target object is generated. According to the method, the problems that in diffuse reflection reconstruction, patterns projected by a projector cannot be comprehensively collected due to exposure and height and gradient ambiguity cannot be solved by a monocular phase deflection technology in high reflection can be solved complementarily, high reflection and diffuse reflection objects can be solved, a complete reconstructed object can be obtained, and the reconstruction precision is improved.

Description

Technical Field

[0001] This application relates to the field of optical measurement technology, and in particular to a method and system for reconstructing the surface of complex reflective objects based on multi-system point cloud fusion. Background Technology

[0002] With the rapid development of precision manufacturing technology and the increasing demands for product surface quality, highly reflective objects, mirror-like surfaces, and transparent glass are increasingly used in industrial production, optics, medicine, and astronomy. Examples include precision-machined ground metal parts, high-precision mirrors in optical systems, eyeglasses for nearsightedness and farsightedness, polished car body surfaces and automotive glass in the automotive industry, large-aperture aspherical mirrors used in astronomical optical systems, and protective glass for mobile phone touchscreens and computer displays. The measurement of these products places higher demands on optical 3D reconstruction technology. Real-world needs drive the development of the manufacturing industry, while innovations in measurement technology provide more precise quantitative standards for these needs. Therefore, technical research on 3D reconstruction of highly reflective surfaces has significant theoretical and practical implications. To achieve non-contact, high-efficiency, and high-precision surface measurement, structured light-based 3D reconstruction technology has been widely adopted and is gradually becoming the mainstream optical measurement method.

[0003] Existing structured light 3D reconstruction methods primarily rely on fringe projection and phase decoding mechanisms. By projecting specific grating patterns onto the object's surface and combining this with camera-acquired deformed patterns, the 3D topography of the object's surface is reconstructed. However, these methods are susceptible to problems such as reflection saturation and pattern distortion when dealing with highly reflective objects, making it difficult to obtain stable and reliable image information. This leads to decreased phase calculation accuracy and poor 3D reconstruction results. To address this, some studies have introduced phase deflection techniques, calculating the object's surface topography through the refraction path of a reference plane, improving measurement capabilities in complex reflective scenes. However, these methods suffer from limitations in practical applications, including reliance on a reference plane, complex deployment, and inflexible measurement processes, making it difficult to meet the industrial demands for efficient and stable measurement. Furthermore, most current commercial systems focus on defect detection, relying solely on absolute phase maps to determine object conformity. This makes it difficult to accurately identify the specific type, location, or size of defects, limiting their applicability in high-precision measurement fields.

[0004] The above analysis reveals that existing 3D reconstruction techniques for highly reflective objects still face the following core technical challenges: Firstly, monocular reconstruction based on phase deflection suffers from height and gradient ambiguity, making it unsuitable for complex surface structures. Secondly, traditional structured light reconstruction methods for diffuse reflective objects are prone to pattern loss on highly reflective surfaces, resulting in unstable reconstruction outcomes. Therefore, addressing the issues of incomplete pattern acquisition due to exposure problems in diffuse reflective reconstruction and the inability of monocular phase deflection techniques to resolve height and gradient ambiguity in highly reflective surfaces remains a significant challenge. Summary of the Invention

[0005] This application provides a method and system for reconstructing the surface of complex reflective objects based on multi-system point cloud fusion. It offers complementary solutions to the problems of incomplete acquisition of projected patterns due to exposure issues in diffuse reflection reconstruction and the inability of monocular phase deflection techniques to resolve height and gradient ambiguities in highly reflective objects. This method can solve for both highly reflective and diffuse reflective objects while obtaining a complete reconstructed object, thus improving reconstruction accuracy. The technical solution provided in this application is as follows:

[0006] In a first aspect, this application provides a method for reconstructing the surface of complex reflective objects based on multi-system point cloud fusion, the method comprising:

[0007] A camera, projector, and LCD screen are deployed around the target object, and the spatial relationship between the camera, projector, and LCD screen is jointly calibrated.

[0008] The structured light patterns projected onto the target object are respectively projected onto a projector and an LCD screen using the projection grating phase method and the phase deflection technique, and the projected structured light patterns are captured by a camera.

[0009] The absolute phase of the structured light pattern acquired by the camera is solved, and the three-dimensional reconstruction results based on the projection grating phase method and the phase deflection method are obtained by combining the joint calibration information.

[0010] Based on the joint calibration information, the 3D reconstruction results obtained by the projection grating phase method and phase deflection method are matched with the world coordinate system and fused with point clouds to generate the 3D shape of the target object.

[0011] In one specific implementation scheme, the joint calibration of the spatial relationship between the camera, projector, and LCD screen includes:

[0012] A structured light pattern-assisted method is used to capture the coded patterns projected by the projector and the LCD screen respectively using a camera, and the imaging correspondence of the three in their respective coordinate systems is extracted.

[0013] For the calibration between the camera and the projector, the calibration matrix between the camera and the projector is calculated by combining the phase encoding method; the virtual image coordinate system method is introduced, and the LCD screen is regarded as a back projection model. The structured light pattern is projected onto the LCD screen by the camera to form an image, thereby realizing the point-to-point correspondence between the camera and the LCD screen.

[0014] In one specific implementation, the step of projecting structured light patterns onto the target object using a projector and an LCD screen respectively based on the projection grating phase method and phase deflection method, and then capturing the structured light patterns projected by both using a camera, includes:

[0015] Turn on the projector and project structured light patterns onto the object surface in a set order. The patterns adopt a typical phase encoding scheme, and the camera simultaneously captures the deformation image sequence of the patterns on the object surface.

[0016] In one specific implementation, the step of projecting structured light patterns onto the target object using a projector and an LCD screen respectively based on the projection grating phase method and phase deflection method, and capturing the projected structured light patterns with a camera, further includes:

[0017] Turn off the projector and turn on the LCD screen. Continue to project another set of structured light patterns onto the target object. The patterns also adopt the phase-encoded form. At the same time, an extra all-white pattern is inserted in the entire projection sequence to mark and identify highly reflective areas. The camera also synchronously acquires the patterns projected on the LCD screen.

[0018] In a specific feasible implementation, the step of performing absolute phase calculation on the structured light pattern acquired by the camera and combining it with joint calibration information to obtain 3D reconstruction results based on the projection grating phase method and phase deflection method respectively includes:

[0019] For the structured light pattern projected by the projector, the wrapping phase is calculated using a multi-step phase-shifting phase solution method. Combined with phase unfolding technology, a high-precision absolute phase map is obtained. By combining the joint spatial calibration parameters of the camera and the projector, the two-dimensional absolute phase data is mapped to three-dimensional space, and the point cloud data of the diffuse reflection area is calculated as the three-dimensional reconstruction result of the diffuse reflection area.

[0020] For structured light patterns projected from an LCD screen, a phase calculation process is applied to handle phase distortion in highly reflective areas. A phase deflection calculation model is used to correct phase anomalies caused by specular reflection, and an absolute phase map of the area is obtained. Combined with the joint calibration data of the camera and the LCD screen, the two-dimensional absolute phase of the highly reflective area is mapped to three-dimensional space to generate corresponding point cloud data, which serves as the three-dimensional reconstruction result of the highly reflective area.

[0021] In one specific implementation scheme, the step of performing world coordinate system matching and point cloud fusion on the 3D reconstruction results obtained by the projection grating phase method and phase deflection technique based on joint calibration information to generate the 3D shape of the target object includes:

[0022] Based on the established joint calibration relationship between the camera, projector, and LCD screen, the two sets of point cloud data are uniformly transformed into the same world coordinate system.

[0023] In one specific implementation scheme, the step of performing world coordinate system matching and point cloud fusion on the 3D reconstruction results obtained by the projection grating phase method and phase deflection technique based on joint calibration information to generate the 3D shape of the target object further includes:

[0024] Based on the all-white pattern image projected from the LCD screen, the highly reflective areas on the surface of the target object are identified, and a region of interest mask is generated accordingly.

[0025] The two sets of point cloud data are compared point by point and analyzed in a unified coordinate system: In non-high reflectivity areas, the marked areas are excluded according to the mask image, and the point cloud data generated by the projection grating phase method is retained; In high reflectivity areas, if the point cloud generated by the projection grating phase method is found to have quality problems in the high reflectivity areas marked by the mask, the point cloud data generated by the phase deflection method at the corresponding position is called to replace or complete it.

[0026] Output the fused complete 3D topographic data.

[0027] Secondly, this application provides a complex reflective object surface reconstruction system based on multi-system point cloud fusion, employing the following technical solution:

[0028] A complex reflective object surface reconstruction system based on multi-system point cloud fusion includes:

[0029] Thirdly, this application provides an electronic device, the device including a processor and a memory; the memory stores a program, the program being loaded and executed by the processor to implement a method for reconstructing the surface of a complex reflective object based on multi-system point cloud fusion as described in the first aspect.

[0030] Fourthly, this application provides a computer-readable storage medium storing a program that, when executed by a processor, is used to implement a method for reconstructing the surface of a complex reflective object based on multi-system point cloud fusion as described in the first aspect.

[0031] In summary, the beneficial effects of this application include at least the following:

[0032] The 3D reconstruction method provided in this application overcomes the problems of decreased accuracy and data loss in 3D imaging of complex curved surfaces or highly reflective areas by fusing two reconstruction techniques: projection grating phase method and phase deflection. This significantly improves the overall reconstruction quality and completeness. Specifically, this scheme first establishes a joint calibration relationship between the camera, projector, and LCD screen, and introduces a virtual image coordinate system to perform spatial geometric calibration of the LCD screen, achieving data correspondence of multi-device imaging paths under a unified world coordinate system. Based on this, structured light pattern projection is performed through two light source paths. The projection grating phase method is used for a first reconstruction of the diffuse reflection area of ​​the object, and then the phase deflection technique is combined to perform a second reconstruction of the highly reflective area, thereby obtaining the highest quality point cloud data for different areas. Furthermore, this application utilizes the full-white pattern image during the LCD screen projection process to accurately identify highly reflective areas, and replaces the defective reconstruction data of the projection grating phase method in the same area with the high-quality point cloud data obtained by the phase deflection technique in that area, achieving targeted fusion at the point cloud level. Ultimately, the system outputs a three-dimensional topography model with high precision, high density, and high integrity across the entire field of view. This model realistically reflects the surface structure of the target object under complex reflection conditions, providing a reliable foundation for subsequent measurement, identification, and analysis. Compared with existing technologies, this application significantly improves the three-dimensional reconstruction capability of highly reflective objects and irregular curved surface regions, expanding the application boundaries of structured light three-dimensional measurement technology in fields such as industrial inspection and high-precision modeling.

[0033] By collaboratively deploying a camera, projector, and LCD screen around the target object, a dual structured light path is constructed, and joint calibration is performed using structured light coded patterns to achieve accurate mapping of the three components in a unified world coordinate system. Subsequently, the object is projected with structured light patterns using both the projector and the LCD screen, while the camera simultaneously acquires images, obtaining dual-path image data covering diffuse and highly reflective areas. After solving for the absolute phase and completing independent 3D reconstructions, the highly reflective areas are identified by combining a full-white image, achieving point cloud-level adaptive replacement and fusion, ultimately generating a 3D topography model with both completeness and accuracy. This method effectively bridges the technical gap between the lack of reconstruction of highly reflective areas using traditional structured light and the monocular ambiguity of phase deflection techniques, significantly improving the applicability and accuracy of 3D reconstruction of complex surfaces.

[0034] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0035] Figure 1 This is a flowchart illustrating the method for reconstructing the surface of complex reflective objects based on multi-system point cloud fusion in this application embodiment.

[0036] Figure 2 This is a schematic diagram of a use case of a method for reconstructing the surface of a complex reflective object based on multi-system point cloud fusion in an embodiment of this application.

[0037] Figure 3 This is a structural block diagram of a complex reflective object surface reconstruction system based on multi-system point cloud fusion in an embodiment of this application.

[0038] Figure 4 This is a block diagram of an electronic device for reconstructing the surface of a complex reflective object based on multi-system point cloud fusion, as described in this application. Detailed Implementation

[0039] The specific embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate this application, but are not intended to limit the scope of this application.

[0040] Optionally, this application uses the method for reconstructing the surface of complex reflective objects based on multi-system point cloud fusion provided in various embodiments as an example in an electronic device. The electronic device is a terminal or a server. The terminal can be a computer, tablet computer, etc. This embodiment does not limit the type of electronic device.

[0041] Reference Figure 1 This is a flowchart illustrating a method for reconstructing the surface of a complex reflective object based on multi-system point cloud fusion, provided in an embodiment of this application. The method includes at least the following steps:

[0042] Step S101: Deploy a camera, projector, and LCD screen around the target object, and jointly calibrate the spatial relationship between the camera, projector, and LCD screen.

[0043] In step S101, the spatial deployment of three core devices—a camera, a projector, and an LCD screen—is first required. These three components together form a dual-path reconstruction structure. The projector and camera constitute a traditional structured light reconstruction system used to acquire information about the diffuse reflection region of the object; while the LCD screen and camera constitute a phase-deflection-based reconstruction system specifically designed to acquire topographic data of highly reflective areas. Therefore, the spatial configuration of these three components must be arranged around the target object, ensuring that the camera can clearly capture all pattern changes projected onto the object's surface by the projector and LCD screen.

[0044] In terms of specific deployment, the projector and LCD screen are placed on the left and right sides of the object, forming a symmetrical distribution to avoid interference between the two types of structured light paths. The camera is positioned above or directly in front of the center, with an appropriate tilt angle selected to effectively cover the key areas of the object. Throughout the deployment, occlusion between the three components should be minimized, and a large overlap of the field of view between the camera and each projection component should be maintained to facilitate pattern alignment and phase matching during calibration. After the equipment deployment is completed, the geometric relationship between the three components needs to be jointly calibrated. The calibration uses a structured light pattern-assisted method, using the camera to capture the coded patterns projected by the projector and LCD screen respectively, and then extracting the imaging correspondence of the three components in their respective coordinate systems. For the calibration between the camera and the projector, a standard checkerboard pattern can be used, combined with the phase encoding method to solve the calibration matrix between the camera and the projector; however, since the LCD screen is a virtual image projection device, it cannot be calibrated using a physical reference object. Therefore, a virtual image coordinate system method is introduced, treating the LCD as a back-projection model, and using the camera to image the structured light pattern projected onto the LCD screen to establish a point-to-point correspondence between the camera and the LCD screen.

[0045] Through the above calibration process, the intrinsic and extrinsic parameter matrices of the camera, projector, and LCD screen, as well as their spatial transformation relationships, are finally obtained. Specifically, in this scheme, all coordinate systems are ultimately unified to the world coordinate system to ensure that the point cloud data obtained from the two systems are consistent, alignable, and fusionable within the same spatial framework. The core objective of joint calibration is to provide a consistent geometric benchmark for the two subsequent reconstruction methods, enabling seamless stitching of point cloud data from different paths in three-dimensional space and maximizing the overall reconstruction accuracy of highly reflective and diffuse reflective areas.

[0046] Step S102: Project the structured light patterns onto the target object using a projector and an LCD screen respectively, based on the projection grating phase method and phase deflection method, and capture the structured light patterns projected by both using a camera.

[0047] In step S102, after completing the spatial calibration of the camera, projector and LCD screen and establishing a unified world coordinate system, the next step is to project a structured light pattern onto the target object and have the camera acquire the corresponding deformed pattern image. This pattern acquisition process is divided into two independent imaging paths, corresponding to two different three-dimensional reconstruction strategies: the projection grating phase method and the phase deflection method.

[0048] Specifically, the projector is first activated, projecting structured light patterns onto the object's surface in a pre-defined sequence. The patterns employ a typical phase-encoding scheme, such as a four-step or eight-step phase-shifting fringe pattern. This process corresponds to the projection grating phase method, which utilizes the phase distortion created by structured light on the object's surface to recover the 3D contour, making it particularly suitable for diffuse surfaces. The camera simultaneously acquires a sequence of deformed images of the patterns on the object's surface, providing data for subsequent phase calculations and 3D point cloud generation. The projector's imaging features a large field of view and high adaptability, making it suitable for acquiring the overall shape of an object, especially exhibiting high reconstruction stability for diffuse areas. After completing the pattern projection and image acquisition along the projector path, the projector is turned off and the LCD screen is activated to project another set of structured light patterns onto the target object. This set of patterns also uses phase encoding, and an additional all-white pattern is inserted into the entire projection sequence to calibrate and identify highly reflective areas. The camera simultaneously acquires the patterns projected onto the LCD screen, a process corresponding to the phase deflection method, focusing particularly on pattern deformation in specular or strongly reflective areas to reconstruct highly reflective regions on the target object's surface. Adding a pure white pattern during LCD screen projection is primarily to accurately identify highly reflective areas on the target object's surface, thus providing precise region segmentation for the subsequent point cloud fusion stage. The pure white pattern has a uniform and bright light intensity distribution. When it is projected onto the object's surface, the highly reflective areas in the image captured by the camera exhibit a strong, saturated bright response due to specular reflection, clearly distinguishing them from the normal grayscale response of diffuse reflection areas.

[0049] In practice, the entire pattern projection and image acquisition process was performed independently in two steps to ensure that pattern information under different projection paths would not overlap or interfere with each other, thereby guaranteeing the accuracy and spatial consistency of phase information calculation. Through this step, image datasets based on the projection grating phase method and phase deflection technique were established, covering the imaging response of the target object in diffuse and highly reflective regions, laying a solid image foundation for subsequent multi-source point cloud reconstruction and fusion.

[0050] Step S103: Perform absolute phase calculation on the structured light pattern acquired by the camera, and combine the joint calibration information to obtain the three-dimensional reconstruction results based on the projection grating phase method and the phase deflection method, respectively.

[0051] In step S103, the absolute phase of the two sets of structured light image data acquired by the camera is calculated separately. First, for the structured light pattern projected by the projector, a multi-step phase-shifting phase calculation method is used to calculate the wrapping phase, and a high-precision absolute phase map is obtained by combining phase unwrapping technology. This absolute phase map accurately reflects the surface morphology information of the diffuse reflection area of ​​the object. Subsequently, by combining the joint spatial calibration parameters of the camera and the projector, the two-dimensional absolute phase data is mapped to three-dimensional space, and the point cloud data of the diffuse reflection area is calculated. This point cloud serves as the three-dimensional reconstruction result of the diffuse reflection area. Second, for the structured light pattern projected by the LCD screen, a similar phase calculation process is applied, focusing on addressing the phase distortion phenomenon in the highly reflective area. A phase deflection-based calculation model is used to correct the phase anomaly caused by specular reflection, obtaining the absolute phase map of this area. By combining the joint calibration data of the camera and the LCD screen, the two-dimensional absolute phase of the highly reflective area is mapped to three-dimensional space, generating the corresponding point cloud data, which serves as the three-dimensional reconstruction result of the highly reflective area.

[0052] This step yields two sets of 3D point cloud data for the diffuse and highly reflective regions, respectively. Both sets of point clouds are located in a unified world coordinate system, ensuring smooth subsequent point cloud matching and fusion processing. Finally, the output of step S103 consists of two sets of high-precision 3D point cloud reconstruction results corresponding to the diffuse and highly reflective regions, providing a crucial data foundation for constructing the complete 3D shape of the target object.

[0053] Step S104: Based on the joint calibration information, perform world coordinate system matching and point cloud fusion on the three-dimensional reconstruction results obtained by the projection grating phase method and phase deflection method to generate the three-dimensional shape of the target object.

[0054] In step S104, the reconstruction results based on the projection grating phase method (projection path) and the phase deflection method (phase deflection method) based on the LCD screen path have been obtained, both representing 3D point cloud data from their respective viewpoints. Since the two types of data are acquired from different projection paths, their original coordinate systems belong to the projector and the LCD screen, respectively. To achieve effective integration, the two sets of point cloud data must first be uniformly transformed to the same world coordinate system based on the joint calibration relationship between the camera, projector, and LCD screen established in step S101, ensuring consistency in spatial location.

[0055] After coordinate unification, the highly reflective areas on the target object's surface are identified based on the all-white pattern image projected onto the LCD screen. A region-of-interest (ROI) mask is then generated, serving as spatial guidance information for subsequent point cloud fusion, clearly indicating the target areas that need replacement or correction. Within highly reflective areas, point clouds generated using the projection grating phase method often exhibit saturation, breaks, or missing data due to specular reflection. Therefore, a fusion mechanism is needed to correct these issues. To this end, the two sets of point cloud data are compared point-by-point and analyzed in a unified coordinate system: In non-highly reflective areas, the marked areas are excluded based on the mask, retaining the projection grating phase method point cloud data due to its higher measurement stability and field-of-view coverage; in highly reflective areas, if holes, distortions, or breaks are detected in the point cloud generated by the projection grating phase method within the masked highly reflective areas, the corresponding phase-deflection point cloud data is automatically used for replacement or completion. During data fusion, spatial registration algorithms can be used to interpolate and smooth boundary transition areas, avoiding abrupt changes or reconstruction errors. Finally, a set of fused, complete 3D topographic data is output. This data comprehensively reflects the true surface structure of the target object in both diffuse and highly reflective regions, possessing both high accuracy and high completeness, significantly outperforming the results obtained from a single reconstruction path. This 3D topographic model can be directly used in subsequent applications such as detection, identification, or surface quality analysis.

[0056] In summary, by collaboratively deploying a camera, projector, and LCD screen around the target object, a dual structured light path is constructed, and joint calibration is performed using structured light coded patterns, achieving accurate mapping of the three components in a unified world coordinate system. Subsequently, the object is projected with structured light patterns using both the projector and the LCD screen, while the camera simultaneously acquires images, obtaining dual-path image data covering diffuse and highly reflective areas. After solving for the absolute phase and completing independent 3D reconstruction, the highly reflective areas are identified by combining a full-white image, achieving point cloud-level adaptive replacement and fusion, ultimately generating a 3D topography model with both completeness and accuracy. This method effectively bridges the technical gap between the lack of reconstruction of highly reflective areas using traditional structured light and the monocular ambiguity of phase deflection techniques, significantly improving the applicability and accuracy of 3D reconstruction of complex surfaces.

[0057] Figure 3 This is a structural block diagram of a complex reflective object surface reconstruction system based on multi-system point cloud fusion provided in one embodiment of this application. The system includes at least the following modules:

[0058] The joint calibration module is used to deploy cameras, projectors, and LCD screens around the target object and perform joint calibration of the spatial relationship between the cameras, projectors, and LCD screens.

[0059] The projection module is used to project onto the target object using a projector and an LCD screen respectively, based on the projection grating phase method and the phase deflection technique, and to capture the structured light patterns projected by both using a camera.

[0060] The 3D reconstruction module is used to solve the absolute phase of the structured light pattern acquired by the camera, and to obtain 3D reconstruction results based on the projection grating phase method and the phase deflection method by combining the joint calibration information.

[0061] The point cloud fusion module is used to perform world coordinate system matching and point cloud fusion on the 3D reconstruction results obtained by the projection grating phase method and phase deflection method based on joint calibration information, so as to generate the 3D shape of the target object.

[0062] For relevant details, please refer to the above method implementation examples.

[0063] Figure 4 This is a block diagram of an electronic device provided in one embodiment of this application. The device includes at least a processor 401 and a memory 402.

[0064] Processor 401 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 401 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 401 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 401 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 401 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0065] The memory 402 may include one or more computer-readable storage media, which may be non-transitory. The memory 402 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 402 is used to store at least one instruction, which is executed by the processor 401 to implement the method for reconstructing complex reflective object surfaces based on multi-system point cloud fusion provided in the method embodiments of this application.

[0066] In some embodiments, the electronic device may also optionally include: a peripheral device interface and at least one peripheral device. The processor 401, memory 402, and peripheral device interface can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface via a bus, signal line, or circuit board. Indicatively, peripheral devices include, but are not limited to: radio frequency circuits, touch displays, audio circuits, and power supplies.

[0067] Of course, electronic devices may also include fewer or more components, and this embodiment does not limit this.

[0068] Optionally, this application also provides a computer-readable storage medium storing a program that is loaded and executed by a processor to implement the method for reconstructing complex reflective object surfaces based on multi-system point cloud fusion in the above-described method embodiments.

[0069] Optionally, this application also provides a computer product including a computer-readable storage medium storing a program, which is loaded and executed by a processor to implement the method for reconstructing complex reflective object surfaces based on multi-system point cloud fusion as described in the above method embodiments.

[0070] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0071] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for reconstructing the surface of complex reflective objects based on multi-system point cloud fusion, characterized in that, The method includes: A camera, projector, and LCD screen are deployed around the target object, and the spatial relationship between the camera, projector, and LCD screen is jointly calibrated. The structured light patterns projected onto the target object are respectively projected onto a projector and an LCD screen using the projection grating phase method and the phase deflection technique, and the projected structured light patterns are captured by a camera. The absolute phase of the structured light pattern acquired by the camera is solved, and the three-dimensional reconstruction results based on the projection grating phase method and the phase deflection method are obtained by combining the joint calibration information. Based on the joint calibration information, the 3D reconstruction results obtained by the projection grating phase method and phase deflection method are matched with the world coordinate system and fused with point clouds to generate the 3D shape of the target object.

2. The method for reconstructing the surface of complex reflective objects based on multi-system point cloud fusion according to claim 1, characterized in that, The joint calibration of the spatial relationship between the camera, projector, and LCD screen includes: A structured light pattern-assisted method is used to capture the coded patterns projected by the projector and the LCD screen respectively using a camera, and the imaging correspondence of the three in their respective coordinate systems is extracted. For the calibration between the camera and the projector, the calibration matrix between the camera and the projector is calculated by combining the phase encoding method; the virtual image coordinate system method is introduced, and the LCD screen is regarded as a back projection model. The structured light pattern is projected onto the LCD screen by the camera to form an image, thereby realizing the point-to-point correspondence between the camera and the LCD screen.

3. The method for reconstructing the surface of complex reflective objects based on multi-system point cloud fusion according to claim 1, characterized in that, The process of projecting structured light patterns onto the target object using a projector and an LCD screen respectively based on the projection grating phase method and phase deflection technique, and then capturing the projected structured light patterns with a camera, includes: Turn on the projector and project structured light patterns onto the object surface in a set order. The patterns adopt a typical phase encoding scheme, and the camera simultaneously captures the deformation image sequence of the patterns on the object surface.

4. The method for reconstructing the surface of complex reflective objects based on multi-system point cloud fusion according to claim 3, characterized in that, The process of projecting structured light patterns onto the target object using a projector and an LCD screen respectively based on the projection grating phase method and phase deflection technique, and then capturing the projected structured light patterns with a camera, further includes: Turn off the projector and turn on the LCD screen. Continue to project another set of structured light patterns onto the target object. The patterns also adopt the phase-encoded form. At the same time, an extra all-white pattern is inserted in the entire projection sequence to mark and identify highly reflective areas. The camera also synchronously acquires the patterns projected on the LCD screen.

5. The method for reconstructing the surface of complex reflective objects based on multi-system point cloud fusion according to claim 4, characterized in that, The process of performing absolute phase calculation on the structured light pattern acquired by the camera and combining it with joint calibration information to obtain 3D reconstruction results based on the projection grating phase method and phase deflection method includes: For the structured light pattern projected by the projector, the wrapping phase is calculated using a multi-step phase-shifting phase solution method. Combined with phase unfolding technology, a high-precision absolute phase map is obtained. By combining the joint spatial calibration parameters of the camera and the projector, the two-dimensional absolute phase data is mapped to three-dimensional space, and the point cloud data of the diffuse reflection area is calculated as the three-dimensional reconstruction result of the diffuse reflection area. For structured light patterns projected from an LCD screen, a phase calculation process is applied to handle phase distortion in highly reflective areas. A phase deflection calculation model is used to correct phase anomalies caused by specular reflection, and an absolute phase map of the area is obtained. Combined with the joint calibration data of the camera and the LCD screen, the two-dimensional absolute phase of the highly reflective area is mapped to three-dimensional space to generate corresponding point cloud data, which serves as the three-dimensional reconstruction result of the highly reflective area.

6. The method for reconstructing the surface of complex reflective objects based on multi-system point cloud fusion according to claim 5, characterized in that, The process of matching the world coordinate system and fusing point clouds of the 3D reconstruction results obtained by the projection grating phase method and phase deflection technique based on joint calibration information to generate the 3D shape of the target object includes: Based on the established joint calibration relationship between the camera, projector, and LCD screen, the two sets of point cloud data are uniformly transformed into the same world coordinate system.

7. The method for reconstructing the surface of complex reflective objects based on multi-system point cloud fusion according to claim 6, characterized in that, The process of matching the world coordinate system and fusing point clouds of the 3D reconstruction results obtained by the projection grating phase method and phase deflection technique based on joint calibration information to generate the 3D shape of the target object also includes: Based on the all-white pattern image projected from the LCD screen, the highly reflective areas on the surface of the target object are identified, and a region of interest mask is generated accordingly. The two sets of point cloud data are compared point by point and analyzed in a unified coordinate system: In non-high reflectivity areas, the marked areas are excluded according to the mask image, and the point cloud data generated by the projection grating phase method is retained; In high reflectivity areas, if the point cloud generated by the projection grating phase method is found to have quality problems in the high reflectivity areas marked by the mask, the point cloud data generated by the phase deflection method at the corresponding position is called to replace or complete it. Output the fused complete 3D topographic data.

8. A complex reflective object surface reconstruction system based on multi-system point cloud fusion, characterized in that, include: The joint calibration module is used to deploy cameras, projectors, and LCD screens around the target object and perform joint calibration of the spatial relationship between the cameras, projectors, and LCD screens. The projection module is used to project onto the target object using a projector and an LCD screen respectively, based on the projection grating phase method and the phase deflection technique, and to capture the structured light patterns projected by both using a camera. The 3D reconstruction module is used to solve the absolute phase of the structured light pattern acquired by the camera, and to obtain 3D reconstruction results based on the projection grating phase method and the phase deflection method by combining the joint calibration information. The point cloud fusion module is used to perform world coordinate system matching and point cloud fusion on the 3D reconstruction results obtained by the projection grating phase method and phase deflection method based on joint calibration information, so as to generate the 3D shape of the target object.

9. An electronic device, characterized in that, The device includes a processor and a memory; the memory stores a program, which is loaded and executed by the processor to implement a method for reconstructing the surface of a complex reflective object based on multi-system point cloud fusion as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a program that, when executed by a processor, is used to implement a method for reconstructing the surface of a complex reflective object based on multi-system point cloud fusion as described in any one of claims 1 to 7.

Images