Three-dimensional reconstruction method and device, computer equipment and storage medium

Abstract

The invention relates to a three-dimensional reconstruction method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining a phase diagram corresponding to a structured light image collected by a camera, and carrying out the preprocessing of the phase diagram based on a pre-constructed mapping relation; the mapping relation indicates a first target position of each pixel point in the phase diagram after preprocessing; and performing three-dimensional reconstruction based on the preprocessed phase diagram. Through the method and the device, the problem of low three-dimensional reconstruction efficiency caused by the fact that an existing reconstruction strategy needs a large amount of real-time calculation is solved, the situation that a large amount of real-time calculation is carried out on each pixel point in the phase diagram is avoided while accurate preprocessing of the phase diagram is ensured, and the three-dimensional reconstruction efficiency is effectively improved.

Description

Technical Field

[0001] This application relates to the field of three-dimensional scanning technology, and particularly to a three-dimensional reconstruction method, apparatus, computer device, and storage medium. Background Art

[0002] Structured light three-dimensional reconstruction technology is a method for three-dimensionally reconstructing an object using optical principles. This technology projects a specific pattern of light, such as a line, grid, or point cloud pattern, onto the object, and uses a camera to capture the deformation of the pattern on the object's surface to calculate the depth information of each point of the object, and then reconstructs a complete three-dimensional model of the object. However, in existing structured light reconstruction strategies, a large amount of real-time calculations are usually required for the phase map corresponding to the image collected by the camera to ensure the accuracy of subsequent reconstruction, resulting in low three-dimensional reconstruction efficiency.

[0003] Regarding the problem in the related art that existing reconstruction strategies require a large amount of real-time calculations, resulting in low three-dimensional reconstruction efficiency, no effective solution has been proposed yet. Summary of the Invention

[0004] In this embodiment, a three-dimensional reconstruction method, apparatus, computer device, and storage medium are provided to solve the problem in the related art that existing reconstruction strategies require a large amount of real-time calculations, resulting in low three-dimensional reconstruction efficiency.

[0005] In a first aspect, in this embodiment, a three-dimensional reconstruction method is provided. The method includes:

[0006] Obtain a phase map corresponding to the structured light image collected by the camera, and preprocess the phase map based on a pre-constructed mapping relationship; the mapping relationship indicates the first target position of each pixel point in the phase map after preprocessing;

[0007] Perform three-dimensional reconstruction based on the preprocessed phase map.

[0008] In some embodiments, the camera is a monocular camera; constructing the mapping relationship for preprocessing the phase map includes:

[0009] Based on the internal parameters and distortion parameters of the camera, construct the mapping relationship corresponding to the phase map; the mapping relationship is used to undistort the phase map.

[0010] In some embodiments, the camera is a binocular camera or a multiocular camera; constructing the mapping relationship for preprocessing the phase map includes:

[0011] Based on the internal parameters, external parameters, and distortion parameters of the camera, construct the mapping relationship corresponding to the phase map; the mapping relationship is used to undistort and stereo-rectify the phase map.

[0012] In some of these embodiments, constructing the mapping relationship corresponding to the phase diagram based on the internal parameters, external parameters, and distortion parameters of the camera includes:

[0013] Determine the second target position after undistortion of each pixel point in the phase diagram according to the internal parameters and the distortion parameters of the camera;

[0014] Perform stereo rectification on each pixel point at the second target position according to the internal parameters and the external parameters of the camera to obtain the first target position after rectification of each pixel point;

[0015] Construct the mapping relationship corresponding to the phase diagram based on the first target position after rectification of each pixel point.

[0016] In some of these embodiments, preprocessing the phase diagram based on a preset mapping relationship includes:

[0017] Traverse each pixel point in the mapping relationship to determine the original image position corresponding to the first target position in the phase diagram;

[0018] Determine the phase value corresponding to the original image position, and fill the phase value into the corresponding first target position.

[0019] In some of these embodiments, determining the phase value corresponding to the original image position and filling the phase value into the corresponding first target position includes:

[0020] When the original image position is an integer pixel position, obtain the phase value at the original image position;

[0021] Fill the phase value into the corresponding first target position.

[0022] In some of these embodiments, determining the phase value corresponding to the original image position further includes:

[0023] When the original image position is not an integer pixel position, process multiple integer pixel positions adjacent to the original image position through an interpolation algorithm to obtain the phase value corresponding to the original image position;

[0024] Fill the phase value into the corresponding first target position.

[0025] Second aspect, in this embodiment, a three-dimensional reconstruction device is provided, and the device includes:

[0026] A preprocessing module, configured to obtain a phase map corresponding to a structured light image collected by a camera, and preprocess the phase map based on a pre-constructed mapping relationship; the mapping relationship indicates a first target position of each pixel in the phase map after preprocessing;

[0027] A reconstruction module, configured to perform three-dimensional reconstruction based on the preprocessed phase map.

[0028] In a third aspect, in this embodiment, a computer device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the three-dimensional reconstruction method described in the first aspect above is implemented.

[0029] In a fourth aspect, in this embodiment, a storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the three-dimensional reconstruction method described in the first aspect above is implemented.

[0030] Compared with the related art, in the three-dimensional reconstruction method, device, computer device, and storage medium provided in this embodiment, by obtaining a phase map corresponding to a structured light image collected by a camera, preprocessing the phase map based on a pre-constructed mapping relationship; the mapping relationship indicates a first target position of each pixel in the phase map after preprocessing; and performing three-dimensional reconstruction based on the preprocessed phase map, the problem that the existing reconstruction strategy requires a large amount of real-time calculations, resulting in low three-dimensional reconstruction efficiency is solved. While ensuring accurate preprocessing of the phase map, a large amount of real-time calculations for each pixel in the phase map are avoided, effectively improving the three-dimensional reconstruction efficiency.

[0031] Details of one or more embodiments of the present application are set forth in the following drawings and description, so that other features, objects, and advantages of the present application become more comprehensible. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The illustrative embodiments and descriptions thereof of the present application are used to explain the present application, and do not constitute an improper limitation of the present application. In the drawings:

[0033] Figure 1 is a hardware structure block diagram of a terminal device for the three-dimensional reconstruction method provided in an embodiment of the present application;

[0034] Figure 2 is a flowchart of the three-dimensional reconstruction method provided in an embodiment of the present application;

[0035] Figure 3 is a flowchart of a mapping relationship construction method provided in an embodiment of the present application;

[0036] Figure 4It is a flowchart of a phase diagram preprocessing method provided by an embodiment of the present application;

[0037] Figure 5 It is a flowchart of a three-dimensional reconstruction method provided by a preferred embodiment of the present application;

[0038] Figure 6 It is a structural block diagram of a three-dimensional reconstruction device provided by an embodiment of the present application.

[0039] In the figure: 102, a processor; 104, a memory; 106, a transmission device; 108, an input / output device; 10, a preprocessing module; 20, a reconstruction module. Detailed implementation manners

[0040] To understand the purpose, technical solution and advantages of the present application more clearly, the present application will be described and illustrated below with reference to the accompanying drawings and embodiments.

[0041] Unless otherwise defined, the technical terms or scientific terms involved in the present application shall have the general meaning understood by those with ordinary skills in the technical field to which the present application belongs. In the present application, words such as "a", "one", "a kind of", "the", "these" and the like do not represent a limitation in quantity, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in the present application are intended to cover non-exclusive inclusion; for example, a process, method, system, product or device including a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent in these processes, methods, products or devices. The terms "connected", "coupled" and the like involved in the present application do not limit to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The term "plurality" involved in the present application means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" may represent: A exists alone, A and B exist simultaneously, and B exists alone. Usually, the character " / " indicates that the objects associated before and after are in an "or" relationship. The terms "first", "second", "third" and the like involved in the present application are only used to distinguish similar objects and do not represent a specific sorting for the objects.

[0042] The method embodiments provided in this embodiment can be executed on a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 It is a hardware structural block diagram of the terminal of the three-dimensional reconstruction method in this embodiment. As Figure 1 shown, the terminal may include one or more ( Figure 1Only one processor 102 and a memory 104 for storing data are shown (only one is shown), where the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that Figure 1 The structure shown is only schematic and does not limit the structure of the above terminal. For example, the terminal may further include more or fewer components than those Figure 1 shown in, or have a different configuration from that Figure 1 shown.

[0043] The memory 104 can be used to store computer programs. For example, software programs and modules of application software, such as the computer program corresponding to the three-dimensional reconstruction method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implements the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely set relative to the processor 102, and these remote memories can be connected to the terminal through a network. Examples of the above network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0044] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one instance, the transmission device 106 includes a network adapter (Network Interface Controller, abbreviated as NIC), which can be connected to other network devices through a base station and thus can communicate with the Internet. In one instance, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0045] In this embodiment, a three-dimensional reconstruction method is provided. Figure 2 is a flowchart of the three-dimensional reconstruction method of this embodiment, as Figure 2 shown, and this process includes the following steps:

[0046] Step S210, obtain a phase diagram corresponding to the structured light image collected by the camera, and preprocess the phase diagram based on a pre-constructed mapping relationship; the mapping relationship indicates the first target position of each pixel point in the phase diagram after preprocessing;

[0047] Step S220, perform three-dimensional reconstruction based on the preprocessed phase diagram.

[0048] Specifically, project light with specific structures and features, such as structured light patterns like lines, grids, or point clouds, onto the target object, and collect the structured light image on the surface of the target object through a camera. Herein, the camera can be a monocular camera, a binocular camera, or a multi-camera. For example, when using a binocular camera, collect the structured light images on the surface of the target object through the left and right cameras to obtain the information of the target object from different angles.

[0049] Preprocess the collected structured light image. The preprocessing methods include grayscale conversion, filtering, etc. Adopt a phase-shift algorithm adapted to factors such as the currently projected pattern or measurement accuracy to calculate the phase of the preprocessed structured light image, and obtain a phase map corresponding to the structured light image. Taking the stripe image collected on the surface of the target object as an example, the three-step phase-shift method or the four-step phase-shift method can be used to achieve phase calculation.

[0050] Furthermore, obtain the pre-constructed mapping relationship, and preprocess the phase map based on the pre-constructed mapping relationship. It should be noted that in this embodiment, the mapping relationship is constructed based on the calibration parameters of the camera, and is used to indicate the first target position after preprocessing for each pixel point in the phase map. The mapping relationship can be in the form of a mapping table, a graphic structure, etc., which is not limited herein. When the camera is a monocular camera, construct a mapping relationship corresponding to the phase map based on the internal parameters and distortion parameters of the camera, and this mapping relationship is used to undistort the phase map; when the camera is a binocular camera or a multi-camera, construct a mapping relationship corresponding to the phase map based on the internal parameters, external parameters, and distortion parameters of the camera, and this mapping relationship is used to undistort and stereo correct the phase map.

[0051] After that, perform three-dimensional reconstruction based on the preprocessed phase map. Taking the structured light reconstruction using a binocular camera as an example, obtain the left and right phase maps after undistortion and stereo correction processing, match the preprocessed left and right phase maps to obtain the corresponding feature points in the left and right phase maps, calculate the disparity based on the extracted corresponding feature points, and calculate the depth information of the points on the object surface according to the disparity and phase information to reconstruct the three-dimensional model of the target object.

[0052] In the existing structured light reconstruction strategies, a large amount of real-time calculations usually need to be performed on the phase map corresponding to the image collected by the camera to ensure the accuracy of subsequent reconstruction, resulting in low three-dimensional reconstruction efficiency.

[0053] In contrast to the prior art, the present application obtains a phase map corresponding to the structured light image collected by the camera, and preprocesses the phase map based on a pre-constructed mapping relationship; the mapping relationship indicates the first target position after preprocessing for each pixel point in the phase map; and three-dimensional reconstruction is performed based on the preprocessed phase map. Based on this, by pre-constructing a mapping relationship for preprocessing the phase map, which records the pixel positions after preprocessing for pixel points at different positions, in the actual structured light reconstruction process, according to the mapping relationship, the pixel positions after preprocessing for each pixel in the phase map can be determined, without performing real-time calculations such as de-distortion and stereo correction for each pixel one by one, solving the problem that the existing reconstruction strategy requires a large amount of real-time calculations, resulting in low three-dimensional reconstruction efficiency, and achieving the goal of ensuring accurate preprocessing of the phase map while avoiding a large amount of real-time calculations for each pixel point in the phase map, effectively improving the three-dimensional reconstruction efficiency.

[0054] In some of these embodiments, the camera is a monocular camera; constructing a mapping relationship for preprocessing the phase map includes the following steps:

[0055] Based on the internal parameters and distortion parameters of the camera, construct a mapping relationship corresponding to the phase map; the mapping relationship is used to de-distort the phase map.

[0056] Specifically, in the case of using a monocular camera for structured light reconstruction, calibrate the internal parameters and distortion parameters of the camera. Among them, the internal parameters of the camera include but are not limited to the focal length of the camera, the coordinates of the principal point, and the distortion parameters include but are not limited to the radial distortion coefficient and the tangential distortion coefficient.

[0057] Furthermore, based on the internal parameters and distortion parameters of the camera, calculate the first target position after de-distortion for pixel points at different positions, that is, for a pixel point with an original position of (x0, y0), perform de-distortion calculation on this pixel point according to the internal parameters and distortion parameters of the camera to obtain the first target position (x1, y1) after de-distortion of this pixel point. The pixel positions (x0, y0) and (x1, y1) are in one-to-one correspondence, and each pixel point (x0, y0) is associated and recorded with its corresponding first target position (x1, y1) to obtain a mapping relationship corresponding to the phase map. The mapping relationship can be in the form of a mapping table, a graphical structure, etc. For example, the mapping relationship records that after de-distortion processing of the pixel point (100, 100), the first target position of the pixel point is (10, 10).

[0058] It should be noted that during the de-distortion process, when the internal parameters and distortion parameters of the camera are determined, the corresponding pixel positions of each pixel point in the image after de-distortion transformation are deterministic and are not affected by other factors.

[0059] In this embodiment, when the camera is a monocular camera, based on the internal parameters and distortion parameters of the camera, a mapping relationship corresponding to the phase map is constructed. The mapping relationship is used to undistort the phase map, thereby establishing a mapping relationship for undistortion processing. When undistorting the phase map subsequently, a large amount of real-time calculations for each pixel point in the phase map can be avoided, which helps improve the 3D reconstruction efficiency.

[0060] In some of these embodiments, the camera is a binocular camera or a multi-camera; constructing a mapping relationship for preprocessing the phase map includes the following steps:

[0061] Based on the internal parameters, external parameters and distortion parameters of the camera, a mapping relationship corresponding to the phase map is constructed; the mapping relationship is used to undistort and stereo rectify the phase map.

[0062] Specifically, in the case of structured light reconstruction using a binocular camera or a multi-camera, the internal parameters, external parameters and distortion parameters of the camera are calibrated. Among them, the internal parameters of the camera include but are not limited to the focal length of the camera and the coordinates of the principal point. The external parameters of the camera include but are not limited to the rotation matrix and the translation vector. The rotation matrix is used to describe the rotation relationship of the camera coordinate system relative to the world coordinate system, and the translation vector is used to describe the position offset of the origin of the camera coordinate system in the world coordinate system. The distortion parameters include but are not limited to the radial distortion coefficient and the tangential distortion coefficient.

[0063] Further, undistortion calculations are performed on pixel points at different positions according to the internal parameters and distortion parameters of the camera to obtain the positions of the pixel points after undistortion, and stereo rectification is performed on the undistorted pixel points according to the internal parameters and external parameters of the camera to obtain the corrected pixel positions of each pixel, that is, the first target positions. Among them, for a pixel point with the original position (x0, y0), after undistortion and stereo rectification are respectively performed based on the internal parameters, external parameters and distortion parameters of the camera, the first target position (x1, y1) of the pixel point is obtained, and the pixel positions (x0, y0) and (x1, y1) correspond one by one. Then, each pixel point (x0, y0) is associated and recorded with its corresponding first target position (x1, y1) to obtain a mapping relationship corresponding to the phase map. The mapping relationship can be in the form of a mapping table, a graphical structure, etc.

[0064] It should be noted that the mapping relationship in this embodiment is used to indicate the first target position after undistortion and stereo rectification of each pixel point in the phase map, so as to combine the undistortion and stereo rectification processes into the same step to simplify the preprocessing process of the phase map and effectively improve the reconstruction efficiency. In addition, during the stereo rectification process, when the internal and external parameters of the camera are determined, the corresponding pixel positions of each pixel point in the image after stereo rectification transformation are deterministic and are not affected by other factors.

[0065] Through this embodiment, when the camera is a binocular camera or a multi-camera, based on the internal parameters, external parameters, and distortion parameters of the camera, a mapping relationship corresponding to the phase map is constructed. The mapping relationship is used to perform distortion removal and stereo correction on the phase map, thereby combining multiple preprocessing steps of the phase map. At the same time, the distortion removal and stereo correction of the phase map can be realized by using the mapping relationship, avoiding a large number of real-time calculations for each pixel point in the phase map during the actual reconstruction process, which helps to improve the efficiency of 3D reconstruction.

[0066] In some of these embodiments, as Figure 3 shown, based on the internal parameters, external parameters, and distortion parameters of the camera, constructing a mapping relationship corresponding to the phase map includes the following steps:

[0067] Step S201, according to the internal parameters and distortion parameters of the camera, determine the second target position of each pixel point in the phase map after distortion removal;

[0068] Step S202, according to the internal parameters and external parameters of the camera, perform stereo correction on each pixel point at the second target position to obtain the first target position of each pixel point after correction;

[0069] Step S203, based on the first target position of each pixel point after correction, construct a mapping relationship corresponding to the phase map.

[0070] Specifically, before performing structured light reconstruction, the internal parameters, external parameters, and distortion parameters of the camera are calibrated in advance. According to the internal parameters and distortion parameters of the camera, distortion removal calculations are performed on pixel points at different positions to obtain the second target positions of each pixel point after distortion removal, and stereo correction is performed on the distorted pixel points according to the internal parameters and external parameters of the camera to obtain the first target position of each pixel after correction.

[0071] Among them, for a pixel point with an original position of (x0, y0), distortion removal is performed based on the internal parameters and distortion parameters of the camera to obtain the second target position (x2, y2) of the pixel point, and stereo correction is performed on the pixel point at the second target position based on the internal parameters and external parameters of the camera to obtain the first target position (x1, y1) of the pixel point. The pixel positions (x0, y0) and (x1, y1) are in one-to-one correspondence. Then, each pixel point (x0, y0) is associated and recorded with its corresponding first target position (x1, y1) to obtain a mapping relationship corresponding to the phase map. The mapping relationship can be in the form of a mapping table, a graphical structure, etc.

[0072] Through this embodiment, according to the internal parameters and distortion parameters of the camera, the second target position after undistortion of each pixel point in the phase map is determined. According to the internal parameters and external parameters of the camera, stereo rectification is performed on each pixel point at the second target position to obtain the first target position after rectification of each pixel point. Furthermore, based on the first target position after rectification of each pixel point, a mapping relationship corresponding to the phase map is constructed, so that the undistortion and stereo rectification of the phase map can be realized by using the mapping relationship, avoiding a large amount of real-time calculations for each pixel point in the phase map during the actual reconstruction process, which helps to improve the efficiency of 3D reconstruction.

[0073] In some of these embodiments, as Figure 4 shown, the preprocessing of the phase map based on the preset mapping relationship in step S210 includes the following steps:

[0074] Step S211, traverse each pixel point in the mapping relationship to determine the original image position corresponding to the first target position in the phase map;

[0075] Step S212, determine the phase value corresponding to the original image position and fill the phase value into the corresponding first target position.

[0076] Specifically, in the actual structured light reconstruction process, each pixel point in the phase map can be represented as (x, y, φ), where (x, y) represents the pixel point position and φ represents the phase value of the pixel point, and this phase value is used as the stored value. After the phase map is input, traverse each pixel point in the pre-constructed mapping relationship. For the pixel point at the desired first target position in the mapping relationship, determine the original image position corresponding to the first target position in the currently obtained phase map according to the mapping relationship. That is, for the pixel point (x3, y3) (both x3 and y3 are integers) at the first target position, calculate its position (x4, y4) in the original phase map.

[0077] It should be noted that the pixel points at each desired first target position refer to the pixel points included in the phase map after expected preprocessing, and the position coordinates of each pixel point are taken as integers. Correspondingly, when saving the mapping relationship, only the mapping relationship of the pixel points expected to be output can be retained to save the storage space of the mapping table.

[0078] Furthermore, determine whether the original image position is an integer pixel position, and obtain the phase value corresponding to the original image position according to the judgment result, and fill the phase value into the corresponding first target position. If the original image position is an integer pixel position, obtain the phase value at the original image position; if the original image position is not an integer pixel position, process multiple adjacent integer pixel positions corresponding to the original image position through an interpolation algorithm to obtain the phase value corresponding to the original image position.

[0079] Through this embodiment, each pixel point in the mapping relationship is traversed to determine the position of the original image corresponding to the first target position in the phase diagram. In this way, the corresponding position of the original image is determined by using the inverse mapping method, and then the phase value corresponding to the position of the original image can be determined. The phase value is filled into the corresponding first target position, realizing fast mapping and avoiding the situation where the pixel position after mapping is non-integer in the forward mapping method.

[0080] In some embodiments, determining the phase value corresponding to the position of the original image in step S212 and filling the phase value into the corresponding first target position includes the following steps:

[0081] When the position of the original image is an integer pixel position, obtain the phase value at the position of the original image;

[0082] Fill the phase value into the corresponding first target position.

[0083] Specifically, after obtaining the position of the original image corresponding to the first target position, determine whether the position of the original image is an integer pixel position. If the position of the original image is an integer pixel position, obtain the phase value at the position of the original image and fill the phase value into the corresponding first target position.

[0084] Exemplarily, according to the mapping relationship, it is determined that the position of the original image corresponding to the first target position (10, 10) is (100, 100). At this time, the position of the original image is an integer pixel position, and the phase value at the position of the original image (100, 100) is filled into the first target position (10, 10).

[0085] Through this embodiment, when the position of the original image is an integer pixel position, the phase value at the position of the original image is obtained and filled into the corresponding first target position, thereby quickly realizing the preprocessing of the phase diagram.

[0086] In some embodiments, determining the phase value corresponding to the position of the original image in step S212 further includes the following steps:

[0087] When the position of the original image is not an integer pixel position, process multiple integer pixel positions adjacent to the position of the original image through an interpolation algorithm to obtain the phase value corresponding to the position of the original image;

[0088] Fill the phase value into the corresponding first target position.

[0089] Specifically, after obtaining the original image position corresponding to the first target position, it is determined whether the original image position is an integer pixel position. If the original image position is not an integer pixel position, that is, at least one of the x - coordinate and y - coordinate of the original image position is non - integer, the interpolation algorithm is used to process multiple integer pixel positions adjacent to the original image position to obtain the phase value corresponding to the original image position. The interpolation algorithm includes but is not limited to bilinear interpolation, nearest - neighbor interpolation, and bicubic interpolation.

[0090] Exemplarily, according to the mapping relationship, it is determined that the original image position corresponding to the first target position (10, 10) is (8.18, 6.67). Bilinear interpolation is used to perform interpolation operations on the phase values of the four integer pixel points (8, 6), (9, 6), (8, 7), and (9, 7) around the original image position to obtain the corresponding phase value.

[0091] Through this embodiment, in the case where the original image position is not an integer pixel position, the interpolation algorithm is used to process multiple integer pixel positions adjacent to the original image position to obtain the phase value corresponding to the original image position, and the phase value is filled into the corresponding first target position, thereby quickly realizing the pre - processing of the phase map.

[0092] The following describes and illustrates this embodiment through preferred embodiments.

[0093] Figure 5 is the flowchart of the three - dimensional reconstruction method of this preferred embodiment, as Figure 5 shown, the three - dimensional reconstruction method includes the following steps:

[0094] Step S510, perform structured light reconstruction using a binocular camera, and pre - construct a mapping relationship corresponding to the phase map based on the internal parameters, external parameters, and distortion parameters of the camera; the mapping relationship indicates the first target position after undistortion and stereo rectification of each pixel point in the phase map;

[0095] Step S520, obtain the phase map corresponding to the structured light images collected by the left and right cameras, traverse each pixel point in the mapping relationship, and determine the original image position corresponding to the first target position in the phase map;

[0096] Step S530, determine the phase value corresponding to the original image position, and judge whether the original image position is an integer pixel position;

[0097] Step S540, in the case where the original image position is an integer pixel position, obtain the phase value at the original image position, and fill the phase value into the corresponding first target position;

[0098] Step S550: When the original image position is not an integer pixel position, use an interpolation algorithm to process multiple integer pixel positions adjacent to the original image position to obtain a phase value corresponding to the original image position, and fill the phase value into the corresponding first target position.

[0099] In this embodiment, binocular cameras are used for structured light reconstruction. Based on the internal parameters, external parameters, and distortion parameters of the cameras, a mapping relationship corresponding to the phase map is constructed in advance. This mapping relationship indicates the first target position after undistortion and stereo rectification for each pixel point in the phase map.

[0100] Furthermore, obtain the phase map corresponding to the structured light images collected by the left and right cameras, traverse each pixel point in the mapping relationship, determine the original image position corresponding to the first target position in the phase map, and determine the phase value corresponding to the original image position. Then, judge whether the original image position is an integer pixel position. When the original image position is an integer pixel position, obtain the phase value at the original image position; fill the phase value into the corresponding first target position. When the original image position is not an integer pixel position, use an interpolation algorithm to process multiple integer pixel positions adjacent to the original image position to obtain a phase value corresponding to the original image position; fill the phase value into the corresponding first target position. This solves the problem that the existing reconstruction strategy requires a large amount of real-time calculation, resulting in low 3D reconstruction efficiency. It realizes ensuring accurate preprocessing of the phase map while avoiding a large amount of real-time calculation for each pixel point in the phase map, effectively improving the 3D reconstruction efficiency.

[0101] It should be noted that the steps shown in the above process or the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0102] In this embodiment, a 3D reconstruction device is also provided. This device is used to implement the above embodiment and the preferred implementation manners, and those that have been described will not be repeated. The following terms such as "module", "unit", "sub-unit", etc. can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0103] Figure 6 is the structural block diagram of the 3D reconstruction device in this embodiment. As Figure 6 shown, the device includes:

[0104] A preprocessing module 10 is configured to obtain a phase map corresponding to a structured light image captured by a camera and preprocess the phase map based on a pre-constructed mapping relationship; the mapping relationship indicates a first target position of each pixel in the phase map after preprocessing.

[0105] A reconstruction module 20 is configured to perform three-dimensional reconstruction based on the preprocessed phase map.

[0106] With the device provided in this embodiment, a phase map corresponding to a structured light image captured by a camera is obtained, and the phase map is preprocessed based on a pre-constructed mapping relationship; the mapping relationship indicates a first target position of each pixel in the phase map after preprocessing; three-dimensional reconstruction is performed based on the preprocessed phase map, solving the problem that existing reconstruction strategies require a large amount of real-time calculations, resulting in low three-dimensional reconstruction efficiency, and achieving avoiding a large amount of real-time calculations for each pixel in the phase map while ensuring accurate preprocessing of the phase map, effectively improving the three-dimensional reconstruction efficiency.

[0107] In some embodiments, based on Figure 6 the device further includes a construction module configured to construct a mapping relationship corresponding to the phase map based on the internal parameters and distortion parameters of the camera; the mapping relationship is used to undistort the phase map.

[0108] In some embodiments, based on Figure 6 the device further includes a construction module configured to construct a mapping relationship corresponding to the phase map based on the internal parameters, external parameters and distortion parameters of the camera; the mapping relationship is used to undistort and stereo rectify the phase map.

[0109] In some embodiments, based on Figure 6 the device further includes a construction module configured to determine a second target position of each pixel in the phase map after undistortion according to the internal parameters and distortion parameters of the camera; perform stereo rectification on each pixel at the second target position according to the internal parameters and external parameters of the camera to obtain a first target position of each pixel after rectification; construct a mapping relationship corresponding to the phase map based on the first target position of each pixel after rectification.

[0110] In some embodiments, the preprocessing module 10 is further configured to traverse each pixel in the mapping relationship to determine an original image position in the phase map corresponding to the first target position; determine a phase value corresponding to the original image position and fill the phase value into the corresponding first target position.

[0111] In some embodiments, the preprocessing module 10 is further configured to, when the original image position is an integer pixel position, obtain the phase value at the original image position; fill the phase value into the corresponding first target position.

[0112] In some of these embodiments, the preprocessing module 10 is further configured to, when the original image position is not an integer pixel position, process multiple integer pixel positions adjacent to the original image position through an interpolation algorithm to obtain a phase value corresponding to the original image position; and fill the phase value into the corresponding first target position.

[0113] It should be noted that the above-mentioned respective modules can be functional modules or program modules, and can be implemented either by software or by hardware. For the modules implemented by hardware, the above-mentioned respective modules can be located in the same processor; or the above-mentioned respective modules can also be located in different processors in any combined form.

[0114] In this embodiment, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0115] Optionally, the above computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the above processor, and the input / output device is connected to the above processor.

[0116] Optionally, in this embodiment, the above processor may be configured to execute the following steps through a computer program:

[0117] S1, obtain a phase map corresponding to the structured light image collected by the camera, and preprocess the phase map based on a pre-constructed mapping relationship; the mapping relationship indicates the first target position of each pixel point in the phase map after preprocessing;

[0118] S2, perform three-dimensional reconstruction based on the preprocessed phase map.

[0119] It should be noted that specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation manners, and will not be elaborated in this embodiment.

[0120] In addition, in combination with the three-dimensional reconstruction method provided in the above embodiments, a storage medium can also be provided to implement it in this embodiment. A computer program is stored on the storage medium; when the computer program is executed by the processor, it implements any one of the three-dimensional reconstruction methods in the above embodiments.

[0121] It should be understood that the specific embodiments described here are only used to explain this application, rather than to limit it. According to the embodiments provided in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present application.

[0122] Obviously, the accompanying drawings are only some examples or embodiments of the present application. For those of ordinary skill in the art, the present application can also be applied to other similar situations based on these drawings without creative efforts. Additionally, it can be understood that although the work done during the development here may be complex and time-consuming, for those of ordinary skill in the art, certain design, manufacturing, or production changes based on the technical content disclosed in the present application are only routine technical means and should not be regarded as insufficient disclosure of the present application.

[0123] The term "embodiment" in the present application means that the specific features, structures, or characteristics described in connection with the embodiments can be included in at least one embodiment of the present application. The phrase appears in various positions in the specification and does not necessarily mean the same embodiment, nor does it mean being independent or alternative to other embodiments and mutually exclusive. Those of ordinary skill in the art can clearly or implicitly understand that the embodiments described in the present application can be combined with other embodiments without conflict.

[0124] The above-described embodiments only represent several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be construed as a limitation on the scope of patent protection. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A three-dimensional reconstruction method, characterized in that: The method comprises: Acquire a phase map corresponding to a structured light image acquired by a camera, and preprocess the phase map based on a pre-constructed mapping relationship; the mapping relationship indicates a first target position of each pixel in the phase map after preprocessing; Three-dimensional reconstruction is performed based on the preprocessed phase image.

2. The three-dimensional reconstruction method according to claim 1, characterized in that: The camera is a monocular camera; Constructing the mapping relationship for preprocessing the phase map, comprising: Based on the intrinsic parameters and distortion parameters of the camera, construct the mapping relationship corresponding to the phase map; The mapping relationship is used to dedistort the phase image.

3. The three-dimensional reconstruction method according to claim 1, characterized in that: The camera is a binocular camera or a multi-eye camera; Constructing the mapping relationship for preprocessing the phase map, comprising: Based on the intrinsic parameters, extrinsic parameters and distortion parameters of the camera, construct the mapping relationship corresponding to the phase map; The mapping relationship is used to perform dedistortion and stereo correction on the phase image.

4. The three-dimensional reconstruction method according to claim 3, characterized in that: The step of constructing the mapping relationship corresponding to the phase map based on the intrinsic parameters, extrinsic parameters and distortion parameters of the camera includes: Determine a second target position after dedistortion of each pixel point in the phase image according to the intrinsic parameter of the camera and the distortion parameter; Performing stereo correction on each of the pixels at the second target position according to the intrinsic parameters and the extrinsic parameters of the camera to obtain the first target position after correction of each of the pixels; Based on the first target position after correction of each pixel point, the mapping relationship corresponding to the phase map is constructed.

5. The three-dimensional reconstruction method according to claim 1, characterized in that: The preprocessing of the phase map based on a preset mapping relationship includes: Traversing each pixel point in the mapping relationship, and determining an original image position corresponding to the first target position in the phase map; A phase value corresponding to the original image position is determined, and the phase value is filled into the corresponding first target position.

6. The three-dimensional reconstruction method according to claim 5, characterized in that: The determining of the phase value corresponding to the original image position and filling the phase value into the corresponding first target position includes: In the case where the original image position is an integer pixel position, obtaining the phase value at the original image position; Fill the phase value into the corresponding first target position.

7. The three-dimensional reconstruction method according to claim 5, characterized in that: The determining of the phase value corresponding to the original image position further includes: In the case where the original image position is not an integer pixel position, a plurality of integer pixel positions adjacent to the original image position are processed by an interpolation algorithm to obtain the phase value corresponding to the original image position; Fill the phase value into the corresponding first target position.

8. A three-dimensional reconstruction device, characterized in that: The device comprises: A preprocessing module, used to obtain a phase map corresponding to a structured light image acquired by a camera, and preprocess the phase map based on a pre-constructed mapping relationship; the mapping relationship indicates a first target position of each pixel in the phase map after preprocessing; A reconstruction module is used to perform three-dimensional reconstruction based on the preprocessed phase image.

9. A computer device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the steps of the three-dimensional reconstruction method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the three-dimensional reconstruction method according to any one of claims 1 to 7 are implemented.

Images