Highly reflective object surface reconstruction method, device, system and storage medium

Abstract

The present application discloses a method, device, system and storage medium for reconstructing the surface of a highly reflective object, and relates to the field of optical measurement technology. The method includes acquiring raw data, projecting a structured light pattern onto the surface of a highly reflective object, processing the structured light pattern, acquiring structural information, and performing reconstruction and linear fitting to obtain the three-dimensional coordinates of the highly reflective object; reflecting a sinusoidal light pattern onto the surface of a highly reflective object, solving and phase matching the sinusoidal light pattern, and obtaining depth or three-dimensional shape information; calculating slope information based on the three-dimensional coordinates of the highly reflective object and the coordinate data of the three, and reconstructing the surface to be measured to obtain reconstruction data; iteratively reconstructing the reconstruction data until the difference between two consecutive reconstruction results is less than a preset value. Based on phase deflectometry, the present application solves the problem that the projected pattern of the projector cannot be fully collected due to exposure problems in diffuse reflection reconstruction, and that the monocular phase deflectometry in high reflectivity has height and gradient ambiguity.

Description

Technical Field

[0001] The present application belongs to the field of optical measurement technology, and in particular, relates to a method, device, system and storage medium for reconstructing the surface of a highly reflective object. Background Art

[0002] Highly reflective surfaces usually refer to those surfaces that produce strong reflections when exposed to light, such as mirrors, metal surfaces or other highly smooth surfaces. The measurement of such surfaces may face interference from reflected light, which requires special technology to deal with. Existing technologies usually use projection suppression technology, phase shift technology, polarization technology, and control of ambient light to overcome the interference caused by reflected light. Different scenarios may require different methods or a combination of multiple technologies for measurement and processing.

[0003] At present, the measurement technology of high-reflective surfaces can be roughly divided into two categories: contact measurement and non-contact measurement. Contact measurement is a mechanical measurement method. This technology is widely used in three-dimensional coordinate measuring machines. However, in the measurement process, the probe is in direct contact with the object surface and there is a certain force. There is a risk of scratching the measured surface, especially for sensitive objects with high-gloss surfaces. In addition, the three-dimensional coordinate measuring machine equipment is large in size, the instrument structure is precise, and the cost is expensive. Under the constraints of these comprehensive factors, the application of contact three-dimensional profile measurement technology on high-gloss surfaces is limited. Non-contact methods, such as powder spraying, interferometry, and phase deflectometry, try to solve the problems of contact measurement. However, these methods also have their own limitations. Powder spraying method: Change the surface properties by spraying special fuel, but it may be very difficult to clean the sprayed pigment, especially in the measurement of some fine surfaces, which may cause damage to the object. Interference method: The interference method uses the principle of light wave interference, but it is greatly affected by the wavelength of the interference light, and the measurement of non-planar or spherical reflective surfaces is limited. Summary of the invention

[0004] The purpose of this application is to provide a method, device, system and storage medium for reconstructing the surface of a highly reflective object, based on phase deflectometry to solve the problem that the projected pattern of the projector cannot be fully collected due to exposure problems in diffuse reflection reconstruction and the problem that monocular phase deflectometry has height and gradient ambiguity in high reflectivity.

[0005] In order to achieve the above objectives, the solution of this application is:

[0006] In a first aspect, an embodiment of the present application provides a method for reconstructing the surface of a highly reflective object, comprising:

[0007] Obtaining raw data, including coordinate calibration information of the camera, projector, and LCD screen;

[0008] Projecting a structured light pattern onto the surface of a highly reflective object, capturing the structured light pattern on the surface of the highly reflective object, processing the structured light pattern on the surface of the highly reflective object, obtaining structural information on the surface of the highly reflective object, reconstructing and linearly fitting the structural information on the surface of the highly reflective object, and obtaining the three-dimensional coordinates of the highly reflective object;

[0009] Reflecting the sinusoidal light pattern onto the surface of a highly reflective object, collecting the sinusoidal light pattern reflected from the surface of the highly reflective object, solving and phase matching the sinusoidal light pattern reflected from the surface of the highly reflective object, and obtaining depth or three-dimensional shape information of the surface of the highly reflective object;

[0010] The slope information of the surface of the highly reflective object is calculated based on the three-dimensional coordinates of the highly reflective object, the camera coordinates and the LCD screen coordinates, and the surface to be measured is reconstructed based on the slope information of the surface of the highly reflective object to obtain reconstructed data; the reconstructed data is iteratively reconstructed until the difference between two consecutive reconstruction results is less than a preset value.

[0011] According to the above method of the embodiment of the present application, the following additional technical features may also be provided:

[0012] Furthermore, the structured light pattern on the surface of the highly reflective object is processed, including:

[0013] The structured light pattern on the surface of highly reflective objects is processed by phase recovery methods, which include phase difference measurement, phase shift method and triangulation measurement.

[0014] Furthermore, the structural information of the surface of the highly reflective object includes the incomplete region structural information and the effective structural information. The surface structural information of the highly reflective object is reconstructed and linearly fitted to obtain the three-dimensional coordinates of the highly reflective object, including:

[0015] The incomplete area in the surface structural information of the highly reflective object is identified, the boundary line between the valid data and the incomplete part in the incomplete area is obtained, the incomplete area is processed by linear fitting, the incomplete area structural information is linearly fitted based on the valid structural information, the incomplete area structural information is filled with the fitting curve obtained by linear fitting, the filled incomplete area structural information and the valid structural information are merged to obtain the three-dimensional coordinates of the highly reflective object.

[0016] Furthermore, the original data is obtained, which includes the coordinate calibration information of the camera, projector and LCD screen, including:

[0017] The optical device includes a camera, a projector and an LCD screen. The camera coordinates, the projector coordinates and the LCD screen coordinates are obtained by calibrating the camera, the projector coordinates and the LCD screen coordinates respectively.

[0018] Furthermore, the camera is calibrated, including internal parameter calibration and external parameter calibration of the camera, wherein:

[0019] The camera is calibrated by obtaining the internal parameters of the camera, including focal length, principal point position, radial and tangential distortion;

[0020] The camera's external parameters are calibrated by photographing an object or template at a known position and calculating the camera's external parameters, including the camera's position and orientation in the world coordinate system, based on the camera's internal parameters.

[0021] Furthermore, the projector is calibrated, including:

[0022] Calibrate the internal parameters of the projector, including focal length, principal point position, radial and tangential distortion;

[0023] To calibrate the external parameters of the projector, the camera is used to collect the calibration pattern projected by the projector, and the external parameters of the projector are calculated by combining the internal and external parameters of the camera, including the position and orientation of the projector in the world coordinate system.

[0024] Furthermore, the LCD screen is calibrated, including:

[0025] Determine the imaging model of the LCD screen, including screen size, resolution and external parameter matrix; display a specific geometric pattern on the LCD screen, shoot the specific geometric pattern on the LCD screen with a camera, obtain image data of the specific geometric pattern, process the image data, obtain feature points of the specific geometric pattern on the LCD screen, and calculate the internal and external parameters of the LCD screen based on the calibration algorithm in combination with the virtual image coordinate system model.

[0026] Furthermore, the surface to be measured is reconstructed based on the slope information of the surface of the highly reflective object to obtain reconstructed data, including:

[0027] Based on the improved Southwell algorithm, a cost function or optimization model is established. The cost function or optimization model is used to describe the characteristics and local information of the surface to be measured. The surface to be measured is estimated and optimized according to the cost function or optimization model in an iterative manner until the difference between two consecutive reconstruction results is less than a preset value.

[0028] In a second aspect, an embodiment of the present application provides a highly reflective object surface reconstruction device, comprising:

[0029] A data acquisition module is configured to acquire raw data, the raw data including coordinate calibration information of a camera, a projector, and an LCD screen;

[0030] An optical processing module is configured to project a structured light pattern onto a surface of a highly reflective object, capture the structured light pattern on the surface of the highly reflective object, reflect the sinusoidal light pattern onto the surface of the highly reflective object, and collect the sinusoidal light pattern reflected from the surface of the highly reflective object;

[0031] A data processing module is configured to process the structured light pattern on the surface of a highly reflective object, obtain structural information on the surface of the highly reflective object, reconstruct and linearly fit the structural information on the surface of the highly reflective object, obtain the three-dimensional coordinates of the highly reflective object, and solve and phase match the sinusoidal light pattern reflected from the surface of the highly reflective object to obtain the depth or three-dimensional shape information of the surface of the highly reflective object;

[0032] The surface reconstruction module is configured to calculate the slope information of the surface of the highly reflective object based on the three-dimensional coordinates of the highly reflective object, the camera coordinates and the LCD screen coordinates, reconstruct the surface to be measured based on the slope information of the surface of the highly reflective object to obtain reconstructed data; and iteratively reconstruct the reconstructed data until the difference between two consecutive reconstruction results is less than a preset value.

[0033] In a third aspect, an embodiment of the present application provides a system for reconstructing the surface of a highly reflective object. The system includes a processor and a memory. The memory stores a computer program, which is loaded and executed by the processor to implement a method for reconstructing the surface of a highly reflective object as provided in the first aspect of the embodiment of the present application.

[0034] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, it is used to implement the method for reconstructing the surface of a highly reflective object as described in the first aspect of the embodiment of the present application.

[0035] Compared with the prior art, the method for reconstructing the surface of a highly reflective object provided by the embodiment of the present application has the following beneficial technical effects:

[0036] The embodiments of the present application can perform three-dimensional reconstruction of highly reflective objects through the projection of structured light and sinusoidal light and the acquisition of coordinate data, and can obtain the shape and depth information of the object surface; by processing the structured light pattern, the structural information of the surface of the highly reflective object can be obtained, and linear fitting of this information helps to obtain more accurate three-dimensional coordinate data; through the reflection and phase matching of sinusoidal light, the depth or three-dimensional shape information of the surface of the highly reflective object can be obtained, which can improve the accuracy of surface measurement; based on the three-dimensional coordinates of the highly reflective object, the slope information of the object surface is calculated through the camera and LCD screen coordinates, and the slope information is used to reconstruct the surface to be measured, providing information about the surface inclination and curvature, and improving the calculation accuracy of the shape and characteristics of the surface of the object to be measured; an iterative method is used to optimize the reconstruction data to ensure that the difference between two consecutive reconstruction results is less than a preset value, thereby improving the stability and accuracy of the reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A schematic diagram showing a process flow of a method for reconstructing a surface of a highly reflective object according to an embodiment of the present application is shown;

[0038] Figure 2 A structural block diagram of a highly reflective object surface reconstruction device according to an embodiment of the present application is shown;

[0039] Figure 3 A structural block diagram of a computer device according to an embodiment of the present application is shown;

[0040] Figure 4 A structural block diagram of an optical system according to an embodiment of the present application is shown.

[0041] Explanation of the accompanying figures: 1. LCD screen; 2. Projector; 3. Camera; 4. Computer; 5. Highly reflective surface. DETAILED DESCRIPTION

[0042] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. It is to be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only some structures related to the present application are shown in the accompanying drawings, rather than all structures. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0043] The terms "including" and "having" and any variations thereof in this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0044] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0045] like Figure 1 As shown, the embodiment of the present application provides a method for reconstructing the surface of a highly reflective object, comprising the following steps:

[0046] Step 101, obtaining raw data, where the raw data includes coordinate calibration information of a camera, a projector, and an LCD screen.

[0047] like Figure 4 As shown, the optical device used in the embodiment of the present application includes an LCD screen 1, a projector 2 and a camera 3. The camera 3 is connected to a computer 4. The camera 3 collects information on the surface 5 of a highly reflective object. In this embodiment, the camera 3 and the LCD screen 1 are calibrated to obtain the LCD screen coordinates in the camera coordinate system, wherein the camera 3 is calibrated, including the internal parameter calibration and the external parameter calibration of the camera 3 to obtain the camera image coordinates, and the projector 2 is calibrated by the camera 3 to obtain the projector coordinates in the camera coordinate system:

[0048] The intrinsic calibration of camera 3 is performed by obtaining the internal parameters of camera 3, which include focal length, principal point position, radial and tangential distortions; the extrinsic calibration of camera 3 is performed by photographing an object or template at a known position and calculating the external parameters of camera 3 in combination with the internal parameters of camera 3, including the position and orientation of camera 3 in the world coordinate system.

[0049] The internal parameters of projector 2 are calibrated, and the internal parameters of projector 2 include focal length, principal point position, radial and tangential distortion; the external parameters of projector 2 are calibrated by collecting the calibration pattern projected by the projector through the camera, and combining the internal and external parameters of the camera 3 to calculate the internal and external parameters of the projector, including the position and orientation of the projector in the world coordinate system.

[0050] The LCD screen 1 is calibrated, including: determining an imaging model of the LCD screen 1, including screen size, resolution, and external parameter matrix; displaying a specific geometric pattern on the LCD screen 1, photographing the specific geometric pattern on the LCD screen 1 with a camera 3 to obtain image data of the specific geometric pattern, processing the image data to obtain feature points of the specific geometric pattern on the LCD screen 1, and calculating internal and external parameters of the LCD screen 1 based on a calibration algorithm in combination with a virtual image coordinate system model.

[0051] It should be noted that the embodiment of the present application does not limit the calibration method of the optical device. The above embodiment is only used to illustrate the method of calibrating the optical device in this embodiment, and is not used to limit the protection scope of the embodiment of the present application.

[0052] Step 102, projecting a structured light pattern onto the surface 5 of a highly reflective object, capturing the structured light pattern on the surface 5 of the highly reflective object, processing the structured light pattern on the surface 5 of the highly reflective object, obtaining structural information on the surface 5 of the highly reflective object, reconstructing and linearly fitting the structural information on the surface 5 of the highly reflective object, and obtaining the three-dimensional coordinates of the highly reflective object.

[0053] In the embodiment of the present application, a structured light pattern is first projected onto the surface 5 of a highly reflective object by a projector 2, and then the structured light pattern of the surface 5 of the highly reflective object is captured by a camera 3. The structured light pattern of the surface 5 of the highly reflective object is processed in a computer 4 to obtain structural information of the surface 5 of the highly reflective object. The structural information of the surface 5 of the highly reflective object is reconstructed and linearly fitted to obtain the three-dimensional coordinates of the highly reflective object.

[0054] Specifically, in this embodiment, the structured light pattern on the surface 5 of the highly reflective object is processed by a phase recovery method, and the phase recovery method includes phase difference measurement, phase shift method and triangulation measurement. It should be noted that the embodiment of the present application does not limit the method for processing the structured light pattern on the surface 5 of the highly reflective object. The processing by the phase recovery method is only used to illustrate the processing method in the embodiment of the present application, and is not used to limit the protection scope of the embodiment of the present application.

[0055] The incomplete area in the structural information of the surface of the highly reflective object is identified, the boundary line between the valid data and the incomplete part in the incomplete area is obtained, the incomplete area is processed by linear fitting, the structural information of the incomplete area is linearly fitted based on the valid structural information, the structural information of the incomplete area is filled with the fitting curve obtained by linear fitting, the filled structural information of the incomplete area and the valid structural information are merged to obtain the three-dimensional coordinates of the highly reflective object.

[0056] Specifically, in the embodiment of the present application, the incomplete areas in the structural information of the surface 5 of a highly reflective object are identified based on computer vision technology, including image processing and pattern recognition algorithms, which can identify abnormal or missing areas in the image, and then complete or reconstruct them. It should be noted that the embodiment of the present application does not limit the method for identifying the incomplete areas. In addition to the computer vision technology method provided in the embodiment of the present application, the incomplete areas can also be identified by a three-dimensional scanner, a phase measurement 3D scanner or texture analysis software. The identification of the incomplete areas by computer vision technology methods is only used to illustrate the identification method of the embodiment of the present application, and is not used to limit the protection scope of the embodiment of the present application.

[0057] Step 103, reflecting the sinusoidal light pattern to the highly reflective object surface 5, collecting the sinusoidal light pattern reflected by the highly reflective object surface 5, solving and phase matching the sinusoidal light pattern reflected by the highly reflective object surface 5, and obtaining the depth or three-dimensional shape information of the highly reflective object surface 5.

[0058] In the embodiment of the present application, the projector 2 is turned off and a sinusoidal light pattern is displayed on the LCD screen 1. The sinusoidal light pattern reflected by the surface 5 of the highly reflective object is collected by the camera 3. By maintaining a fixed state between the surface 5 of the highly reflective object and the camera 3, the phase of the structured light pattern reflected by the LCD screen 1 is solved and matched.

[0059] Specifically, in the above-described solution, the purpose of turning off the projector 2 and allowing the LCD screen 1 to display the sinusoidal light pattern is to obtain information about the surface 5 of the highly reflective object by using the way the LCD screen 1 reflects light. The principle in optics is that the LCD screen 1 acts as a reflective surface, and when it displays a specific pattern, such as a sinusoidal light pattern, the light reflected from the surface 5 of the highly reflective object will also carry the characteristics of this pattern. This specific pattern can be used to perform phase resolution and matching in the image captured by the camera 3 to obtain information about the surface of the object.

[0060] The sinusoidal light pattern has certain structural characteristics, including frequency and phase changes. By displaying this specific pattern on the LCD screen 1 and using the camera 3 to capture the pattern reflected from the surface of the object, the phase can be solved, and the depth or shape information of the surface of the object can be derived. This method is used to solve and match the phase of the structured light pattern reflected by the LCD screen 1, thereby obtaining the structural information of the surface of the object.

[0061] After the structured light pattern reflected by the LCD screen 1 is captured, it undergoes phase solving and matching, mainly to obtain the depth or three-dimensional shape information of the object surface.

[0062] Specifically, phase solving and matching is the processing of a specific structured light pattern reflected by the LCD screen 1. By analyzing the pattern reflected by the LCD screen 1, especially the phase information change in the pattern, the phase information of the pattern is inferred using a phase recovery technique or a phase demodulation algorithm. The phase information reflects the phase change of the light wave when it is reflected or scattered on the surface of an object, and the depth or distance of different parts of the surface of the object can be derived using this information.

[0063] The pattern reflected by the LCD screen 1 is compared with the original projected pattern by matching or decoding the phase information. From this comparison, the depth information or distance data of different areas on the surface of the object can be obtained. Specifically, the phase difference can be converted into information about the surface height or three-dimensional structure.

[0064] Therefore, after phase resolution and matching, the structured light pattern reflected by the LCD screen 1 can provide information about the geometric shape, height or depth of the object surface, thereby realizing three-dimensional reconstruction or shape measurement of the object surface.

[0065] Step 104, using the three-dimensional coordinates of the highly reflective object, based on the camera coordinates and the LCD screen coordinates, calculate the slope information of the surface 5 of the highly reflective object, reconstruct the surface to be measured based on the slope information of the surface 5 of the highly reflective object, and obtain reconstructed data; iteratively reconstruct the reconstructed data until the difference between two consecutive reconstruction results is less than a preset value.

[0066] The embodiment of the present application provides two methods for calculating the slope information of the surface 5 of a highly reflective object based on the camera coordinates and the LCD screen coordinates through the three-dimensional coordinates of the highly reflective object. One of the methods utilizes the principle of differential geometry and calculates the slope information by taking the partial derivatives of the surface in different directions. Since the three-dimensional coordinates (x, y, z), the camera coordinates (u, v) and the LCD screen coordinates (x', y') of the highly reflective object are obtained, the coordinates of the object in the camera coordinate system are calculated according to the relationship between the coordinates (u, v) in the camera coordinate system and the three-dimensional coordinates (x, y, z), and the partial derivatives in the u and v directions in the camera coordinate system are calculated. The slope of the surface in the camera coordinate system can be estimated by the obtained partial derivatives. The LCD screen coordinates are used to calibrate or correct the slope information in the camera coordinate system, and need to be specifically confirmed based on the position and direction of the LCD screen 1 in the three-dimensional space and its mapping relationship in the camera coordinate system.

[0067] Another method is to calculate the slope in the world coordinate system and convert the slope in the camera coordinate system to the slope in the world coordinate system. This requires considering the transformation matrix from the camera to the world coordinate system and using the camera calibration information to complete the conversion between coordinate systems.

[0068] The specific slope calculation method may be adjusted according to the actual situation and specific application. It should be noted that the present application embodiment does not limit the slope calculation method, and the above method is only used to illustrate the slope calculation method in the present application embodiment, and is not used to limit the protection scope of the present application embodiment.

[0069] The embodiment of the present application is based on an improved Southwell algorithm to establish a cost function or an optimization model, which is used to describe the characteristics and local information of the surface to be measured. The surface to be measured is estimated and optimized according to the cost function or the optimization model in an iterative manner until the difference between two consecutive reconstruction results is less than a preset value.

[0070] Among them, the Southwell algorithm is used for surface reconstruction, and its cost function or optimization model is based on the topological structure and constraint conditions of the local surface. In this embodiment, the Southwell algorithm is implemented based on the cost function, and the formula is as follows:

[0071] E(S)=λ1·Smoothness(S)+λ2·DataFit(S)

[0072] Among them, E(S) represents the overall cost function; Smoothness(S) represents the smoothness term; DataFit(S) represents the data fitting term; λ1 and λ2 represent weight parameters.

[0073] In this embodiment, the cost function may be adjusted and expanded according to specific scenarios and requirements to accommodate more constraints and information. The actual application of the algorithm requires determining an appropriate cost function or optimization model based on the specific problem domain and data conditions.

[0074] The reconstruction data is obtained by the Southwell algorithm provided in the above embodiment, and the reconstruction data is iteratively reconstructed until the difference between two consecutive reconstruction results is less than a preset value, thereby completing the reconstruction of the highly reflective object surface 5.

[0075] like Figure 2 As shown, a highly reflective object surface reconstruction device provided in an embodiment of the present application includes a data acquisition module 201, an optical processing module 202, a data processing module 203 and a surface reconstruction module 204, wherein:

[0076] The data acquisition module 201 is configured to acquire raw data, the raw data including coordinate calibration information of the camera, the projector and the LCD screen;

[0077] The optical processing module 202 is configured to project the structured light pattern onto the highly reflective object surface 5, capture the structured light pattern on the highly reflective object surface 5, reflect the sinusoidal light pattern onto the highly reflective object surface 5, and collect the sinusoidal light pattern reflected by the highly reflective object surface 5;

[0078] The data processing module 203 is configured to process the structured light pattern of the highly reflective object surface 5, obtain the structural information of the highly reflective object surface 5, reconstruct and linearly fit the structural information of the highly reflective object surface 5, obtain the three-dimensional coordinates of the highly reflective object, and solve and phase match the sinusoidal light pattern reflected by the highly reflective object surface 5 to obtain the depth or three-dimensional shape information of the highly reflective object surface 5;

[0079] The surface reconstruction module 204 is configured to calculate the slope information of the surface 5 of the highly reflective object based on the three-dimensional coordinates of the highly reflective object, based on the camera coordinates and the LCD screen coordinates, and reconstruct the surface to be measured based on the slope information of the surface 5 of the highly reflective object to obtain reconstructed data; and iteratively reconstruct the reconstructed data until the difference between two consecutive reconstruction results is less than a preset value.

[0080] The highly reflective object surface reconstruction device in the embodiment of the present application can be a computer device, or a component in the computer device, such as an integrated circuit or a chip. The computer device can be a terminal, or other devices other than a terminal. Exemplarily, the computer device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted computer device, a mobile Internet device (Mobile Internet Device, MID), an ultra-mobile personal computer (Ultra-Mobile Personal Computer, UMPC), a netbook or a personal digital assistant (Personal Digital Assistant, PDA), etc. It can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (Personal Computer, PC), etc., which is not specifically limited in the embodiment of the present application.

[0081] The device for reconstructing the surface of a highly reflective object provided in the embodiment of the present application can achieve Figure 1 To avoid repetition, the various processes implemented in the method embodiment of highly reflective object surface reconstruction are not described here.

[0082] The present application also provides a computer device, such as Figure 3As shown, the computer device includes a processor 301 and a memory 302. The memory 302 stores programs or instructions that can be run on the processor 301. When the program or instructions are executed by the processor 301, the various steps of the above-mentioned high-reflective object surface reconstruction method are implemented, and the same technical effect can be achieved. To avoid repetition, they are not repeated here.

[0083] It should be noted that the computer device in the embodiment of the present application includes the above-mentioned mobile computer device and non-mobile computer device.

[0084] The memory 302 can be used to store software programs and various data. The memory 302 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 302 may include a volatile memory or a non-volatile memory, or the memory 302 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 302 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0085] The processor 301 may include one or more processing units; optionally, the processor 301 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 301.

[0086] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned embodiment of the method for reconstructing the surface of a highly reflective object is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0087] An embodiment of the present application also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned embodiment of the method for reconstructing the surface of a highly reflective object, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0088] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0089] An embodiment of the present application also provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned highly reflective object surface reconstruction method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0090] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises one..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be noted that the scope of the method and device in the embodiment of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0091] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for reconstructing the surface of a highly reflective object, characterized in that: The method comprises: Acquire raw data, wherein the raw data includes coordinate calibration information of a camera, a projector, and an LCD screen; Projecting a structured light pattern onto the surface of a highly reflective object, capturing the structured light pattern on the surface of the highly reflective object, processing the structured light pattern on the surface of the highly reflective object, obtaining structural information on the surface of the highly reflective object, reconstructing and linearly fitting the structural information on the surface of the highly reflective object, and obtaining the three-dimensional coordinates of the highly reflective object; By turning off the projector and displaying a sinusoidal light pattern on an LCD screen, the sinusoidal light pattern is reflected onto the surface of a highly reflective object, the sinusoidal light pattern reflected from the surface of the highly reflective object is collected by a camera, and the sinusoidal light pattern reflected from the surface of the highly reflective object is solved and phase-matched to obtain depth or three-dimensional shape information of the surface of the highly reflective object; By using the three-dimensional coordinates of the highly reflective object, based on the camera coordinates and the LCD screen coordinates, the slope information of the surface of the highly reflective object is calculated, and the surface to be measured is reconstructed based on the slope information of the surface of the highly reflective object to obtain reconstruction data; the reconstructed data is iteratively reconstructed until the difference between two consecutive reconstruction results is less than a preset value; The processing of the structured light pattern on the surface of the highly reflective object includes: Processing the structured light pattern on the surface of the highly reflective object by a phase recovery method, wherein the phase recovery method includes phase difference measurement, phase shift method and triangulation measurement; The structural information of the surface of the highly reflective object includes incomplete region structural information and effective structural information, and the reconstruction and linear fitting of the structural information of the surface of the highly reflective object to obtain the three-dimensional coordinates of the highly reflective object includes: The incomplete area in the structural information of the surface of the highly reflective object is identified, the boundary line between the valid data and the incomplete part in the incomplete area is obtained, the incomplete area is processed by linear fitting, the structural information of the incomplete area is linearly fitted based on the valid structural information, the structural information of the incomplete area is filled with the fitting curve obtained by the linear fitting, the filled structural information of the incomplete area and the valid structural information are merged to obtain the three-dimensional coordinates of the highly reflective object.

2. The method for reconstructing the surface of a highly reflective object according to claim 1, wherein: The obtaining of raw data, wherein the raw data includes coordinate calibration information of the camera, the projector and the LCD screen, includes: An optical device is calibrated, wherein the optical device includes a camera, a projector and an LCD screen. The camera, the projector and the LCD screen are calibrated respectively to obtain camera coordinates, projector coordinates and LCD screen coordinates.

3. The method for reconstructing the surface of a highly reflective object according to claim 2, wherein: The calibrating the camera includes performing internal parameter calibration and external parameter calibration on the camera, wherein: The camera is calibrated by obtaining internal parameters of the camera, wherein the internal parameters of the camera include focal length, principal point position, radial distortion and tangential distortion; The camera is calibrated by photographing an object or template at a known position and calculating the camera's external parameters in combination with the camera's internal parameters, including the camera's position and orientation in a world coordinate system. The calibrating the projector includes: Calibrate the internal parameters of the projector, wherein the internal parameters of the projector include focal length, principal point position, radial distortion and tangential distortion; The projector is calibrated with external parameters by collecting the calibration pattern projected by the projector through the camera, and calculating the external parameters of the projector, including the position and orientation of the projector in the world coordinate system, by combining the internal parameters and external parameters of the camera.

4. The method for reconstructing the surface of a highly reflective object according to claim 2, wherein: The calibrating the LCD screen includes: Determine the imaging model of the LCD screen, including screen size, resolution and external parameter matrix; display a specific geometric pattern on the LCD screen, shoot the specific geometric pattern on the LCD screen with the camera to obtain image data of the specific geometric pattern, process the image data to obtain feature points of the specific geometric pattern on the LCD screen, and calculate the internal and external parameters of the LCD screen based on the calibration algorithm in combination with the virtual image coordinate system model.

5. The method for reconstructing the surface of a highly reflective object according to claim 1, wherein: The step of reconstructing the surface to be measured based on the slope information of the surface of the highly reflective object to obtain the reconstructed data includes: Based on the improved Southwell algorithm, a cost function or an optimization model is established, wherein the cost function or the optimization model is used to describe the characteristics and local information of the surface to be measured. The surface to be measured is estimated and optimized according to the cost function or the optimization model in an iterative manner until the difference between two consecutive reconstruction results is less than a preset value.

6. A device for reconstructing the surface of a highly reflective object using the method according to any one of claims 1 to 5, characterized in that: The device comprises: A data acquisition module is configured to acquire raw data, wherein the raw data includes coordinate calibration information of a camera, a projector, and an LCD screen; An optical processing module is configured to project a structured light pattern onto a surface of a highly reflective object, capture the structured light pattern on the surface of the highly reflective object, reflect a sinusoidal light pattern onto the surface of the highly reflective object, and collect the sinusoidal light pattern reflected from the surface of the highly reflective object; A data processing module is configured to process the structured light pattern on the surface of the highly reflective object, obtain the structural information of the surface of the highly reflective object, reconstruct and linearly fit the structural information on the surface of the highly reflective object, obtain the three-dimensional coordinates of the highly reflective object, and solve and phase match the sinusoidal light pattern reflected from the surface of the highly reflective object to obtain the depth or three-dimensional shape information of the surface of the highly reflective object; The surface reconstruction module is configured to calculate the slope information of the surface of the highly reflective object based on the three-dimensional coordinates of the highly reflective object, the camera coordinates and the LCD screen coordinates, reconstruct the surface to be measured based on the slope information of the surface of the highly reflective object to obtain reconstructed data; and iteratively reconstruct the reconstructed data until the difference between two consecutive reconstruction results is less than a preset value.

7. A highly reflective object surface reconstruction system, the system comprising a processor and a memory, wherein the memory stores a computer program, characterized in that: The computer program is loaded and executed by the processor to implement the method for reconstructing the surface of a highly reflective object as described in any one of claims 1 to 5.

8. A computer-readable storage medium, wherein a computer program is stored in the storage medium, characterized in that: When the computer program is executed by a processor, it is used to implement the method for reconstructing the surface of a highly reflective object as described in any one of claims 1 to 5.

Images