Three-dimensional reconstruction method and device based on fusion phase unwrapping, equipment and medium

Abstract

The invention relates to the field of three-dimensional reconstruction, and discloses a three-dimensional reconstruction method, device and equipment based on fusion phase unwrapping and a medium, and the method comprises the steps: collecting deformed stripe images of a target object under different phases; processing the deformed fringe images under different phases to obtain wrapped phase information, modulation intensity information and background intensity information of a single channel; fusing and constructing a three-channel composite image according to the wrapped phase information, the modulation intensity information and the background intensity information of the single channel; inputting the three-channel composite image into a pre-trained convolutional neural network for reliability identification, and outputting a point-by-point classification graph according to an identification result; and performing phase unwrapping according to the point-by-point classification graph and the wrapped phase information, and constructing three-dimensional contour information of the target object in combination with preset calibration parameters. Through information fusion of multiple channels, unreliable points are automatically and accurately identified before phase unwrapping, the phase unwrapping precision in a complex scene is improved, and the quality and effect of three-dimensional reconstruction are further improved.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional reconstruction technology and financial technology, and in particular to a three-dimensional reconstruction method, device, equipment and medium based on fusion phase unfolding. Background Art

[0002] 3D reconstruction refers to the process of reconstructing 3D information based on single-view or multi-view images. 3D reconstruction is of great significance for scenarios such as virtual reality, AR / VR, and metaverse construction. In the financial sector, with the advancement of digital transformation, 3D reconstruction technology can improve people's participation and interaction in virtual scenes, such as virtual customer service, marketing video virtual scene construction, etc., helping people get interactive services, understand the layout of offline business halls, and participate in various activities without leaving home.

[0003] Fringe projection profilometry (FPP) based on phase measurement has high measurement accuracy and is therefore widely used in many fields. In terms of 3D imaging speed and system cost, a single-camera system that projects a single-frequency pattern is the ideal choice among all current fringe projection profilometry systems, which requires a robust spatial phase unwrapping (SPU) algorithm. However, in complex scenes, for spatial phase unwrapping in FPP, there are many unreliable points before unwrapping, making it difficult to perform accurate phase unwrapping, reducing the phase unwrapping accuracy, and thus affecting the accuracy and reliability of 3D reconstruction. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the object of the present invention is to provide a three-dimensional reconstruction method, device, equipment and medium based on fusion phase unfolding that can be applied to financial technology or other related fields. Its main purpose is to improve the phase unfolding accuracy and thereby improve the accuracy of three-dimensional reconstruction.

[0005] The technical solution of the present invention is as follows:

[0006] A first aspect of the present invention provides a three-dimensional reconstruction method based on fusion phase unwrapping, comprising:

[0007] Collect deformed fringe images of the target object at different phases;

[0008] Processing the deformed fringe images at different phases to obtain single-channel wrapping phase information, modulation intensity information, and background intensity information;

[0009] According to the wrapped phase information, modulation intensity information and background intensity information of the single channel, a corresponding three-channel composite image is constructed by fusing;

[0010] Inputting the three-channel composite image into a pre-trained convolutional neural network for reliability recognition, and outputting a corresponding point-by-point classification map according to the recognition result;

[0011] Phase unwrapping is performed according to the point-by-point classification map and the wrapped phase information, and the three-dimensional contour information of the target object is constructed in combination with preset calibration parameters.

[0012] A second aspect of the present invention provides a three-dimensional reconstruction device based on fusion phase unwrapping, comprising:

[0013] An image acquisition module, used for acquiring deformed fringe images of a target object at different phases;

[0014] An image processing module is used to process the deformed fringe images at different phases to obtain single-channel wrapping phase information, modulation intensity information and background intensity information;

[0015] A multi-channel fusion module, used for fusing and constructing a corresponding three-channel composite image according to the wrapped phase information, modulation intensity information and background intensity information of the single channel;

[0016] A point-by-point classification module, used for inputting the three-channel composite image into a pre-trained convolutional neural network for reliability recognition, and outputting a corresponding point-by-point classification map according to the recognition result;

[0017] The phase unwrapping and reconstruction module is used to perform phase unwrapping according to the point-by-point classification map and the wrapped phase information, and to construct the three-dimensional contour information of the target object in combination with preset calibration parameters.

[0018] A third aspect of the present invention provides a computer device, comprising at least one processor; and

[0019] a memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the above-mentioned three-dimensional reconstruction method based on fusion phase unwrapping.

[0021] A fourth aspect of the present invention provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the one or more processors can execute the above-mentioned three-dimensional reconstruction method based on fusion phase unfolding.

[0022] Beneficial effects: The present invention discloses a three-dimensional reconstruction method, device, equipment and medium based on fusion phase unwrapping. Compared with the prior art, the embodiment of the present invention collects deformed fringe images of the target object at different phases; processes the deformed fringe images at different phases to obtain single-channel wrapped phase information, modulation intensity information and background intensity information; fuses and constructs a corresponding three-channel composite image based on the single-channel wrapped phase information, modulation intensity information and background intensity information; inputs the three-channel composite image into a pre-trained convolutional neural network for reliability identification, and outputs a corresponding point-by-point classification map based on the identification result; performs phase unwrapping based on the point-by-point classification map and the wrapped phase information, and constructs the three-dimensional contour information of the target object in combination with preset calibration parameters. By extracting the wrapped phase, modulation intensity and background intensity information from the deformed fringe images at different phases and fusing them to construct a composite image, the multi-channel information fusion enables the automatic and accurate identification of unreliable points before phase unwrapping, thereby improving the phase unwrapping accuracy, better adapting to the phase unwrapping requirements in complex scenes, and improving the quality and effect of three-dimensional reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the scheme in the present invention, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1 A schematic diagram of an application environment of a three-dimensional reconstruction method based on fusion phase unwrapping provided in an embodiment of the present invention;

[0025] Figure 2 A flow chart of a three-dimensional reconstruction method based on fusion phase unwrapping provided in an embodiment of the present invention;

[0026] Figure 3 A flow chart of step S201 in the three-dimensional reconstruction method based on fusion phase unwrapping provided in an embodiment of the present invention;

[0027] Figure 4 A flow chart of step S202 in the three-dimensional reconstruction method based on fusion phase unwrapping provided in an embodiment of the present invention;

[0028] Figure 5 A flow chart of step S203 in the three-dimensional reconstruction method based on fusion phase unwrapping provided in an embodiment of the present invention;

[0029] Figure 6 A flow chart of step S204 in the three-dimensional reconstruction method based on fusion phase unwrapping provided in an embodiment of the present invention;

[0030] Figure 7 A flow chart of step S205 in the three-dimensional reconstruction method based on fusion phase unwrapping provided in an embodiment of the present invention;

[0031] Figure 8 A schematic diagram of functional modules of a three-dimensional reconstruction device based on fusion phase unwrapping provided in an embodiment of the present invention;

[0032] Fig. 9 A schematic diagram of the hardware structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail below. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. The embodiments of the present invention are described below in conjunction with the accompanying drawings.

[0034] The three-dimensional reconstruction method based on fusion phase unwrapping provided by the embodiment of the present invention can be applied in the following aspects: Figure 1 In the application environment of FIG. 1 , it includes a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104 and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0035] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (only as examples).

[0036] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0037] The server 105 may be a server that provides various services, such as a background server that provides support for the content browsed by the user using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (for example only). The background server may analyze and process the received data such as the user request, and feed back the processing results (such as web pages, information, or data obtained or generated according to the user request) to the terminal device. The server 105 may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server 105 may also be a server for a distributed system, or a server combined with a blockchain.

[0038] It should be noted that the three-dimensional reconstruction method based on fusion phase unwrapping provided in the embodiment of the present application can generally be executed by the first terminal device 101, the second terminal device 102 or the third terminal device 103. Accordingly, the three-dimensional reconstruction device based on fusion phase unwrapping provided in the embodiment of the present invention can also be set in the first terminal device 101, the second terminal device 102 or the third terminal device 103. Alternatively, the three-dimensional reconstruction method based on fusion phase unwrapping provided in the embodiment of the present invention can also generally be executed by the server 105. Accordingly, the three-dimensional reconstruction device based on fusion phase unwrapping provided in the embodiment of the present invention can generally be set in the server 105.

[0039] It should be understood that the numbers of the above terminal devices, networks and servers are only illustrative. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0040] like Figure 2 As shown, the 3D reconstruction method based on fusion phase unwrapping provided in the embodiment of the present invention specifically includes the following steps:

[0041] S201, collecting deformed fringe images of a target object at different phases.

[0042] In this embodiment, when it is necessary to perform three-dimensional reconstruction on the target object, the target object is imaged by a single camera system that projects a single frequency pattern, and deformed fringe images on the target object at different phases are obtained. Specifically, a series of fringe patterns at different phases are projected onto the target object by a projector, and the images of these fringe patterns on the surface of the target object are captured by a camera. At this time, these images will be deformed due to the ups and downs of the surface of the object, thereby obtaining multiple deformed fringe patterns at different phases. For example, four fringe images at different phases can be collected, and the fringe pattern of each image has a fixed phase offset relative to the previous image, that is, each phase shift is π / 2. By collecting fringe images at different phases, accurate input data is provided for subsequent phase analysis and three-dimensional reconstruction.

[0043] For example, in the construction scenario of a virtual business hall, the ATM machines or other service equipment in the business hall can be scanned by a single-camera system that projects a single-frequency pattern. A projector is used to project the stripe pattern onto the device at different phases (such as 0°, 45°, 90°, and 135°), and a camera is used to capture these deformed stripe images from a fixed position. These images will contain phase information caused by the height change of the device surface, which can be used for high-precision three-dimensional measurement of the device, helping to improve the authenticity and interactivity of the equipment display in the virtual business hall.

[0044] S202, processing the deformed fringe images at different phases to obtain single-channel wrapping phase information, modulation intensity information, and background intensity information.

[0045] In this embodiment, data processing is performed on the collected deformed stripe images at multiple different phases, and phase calculation, modulation calculation and background intensity calculation are performed respectively, and corresponding wrapped phase information, modulation intensity information and background intensity information are obtained in three independent channels, thereby forming three single-channel data. The wrapped phase information refers to the fact that in actual measurement, since the phase value is periodic (usually a 2π period), when the phase change exceeds π, when the actual phase change exceeds π, a phase jump will occur, causing the phase value to "wrap" back to the range of -π,π. This phase value is called the wrapped phase. The modulation intensity information is an indicator to measure the contrast of the stripe pattern, which reflects the visibility and quality of the stripe pattern. The background intensity information refers to the light intensity reflected from the surface of the target or object in the absence of a stripe pattern. By further processing the modulation intensity information and background intensity information on the basis of obtaining the wrapped phase information, more comprehensive auxiliary information is provided for the subsequent identification of unreliable points in the image to improve the phase unwrapping accuracy.

[0046] S203 , constructing a corresponding three-channel composite image by fusing the wrapped phase information, the modulation intensity information and the background intensity information of the single channel.

[0047] In this embodiment, the extracted wrapped phase information, modulation intensity information and background intensity information are respectively taken as three channels for multi-channel data fusion, which can be achieved, for example, by simple stacking, weighted fusion, feature-level fusion, etc., so that the three channel data are combined into a multi-channel image, wherein each channel represents a kind of information, and the information amount of the image is enhanced through multi-channel fusion, so that in the subsequent phase unfolding, various information such as phase, modulation and background can be fused for unfolding, and the information from multiple aspects is integrated to help the correct phase unfolding and improve the accuracy of phase unfolding.

[0048] Preferably, in order to ensure that no errors are introduced during fusion and the reliability of the fusion process, the fused three-channel composite image can be further verified by preset indicators, such as by visual inspection or calculation of indicators such as signal-to-noise ratio and contrast, to ensure the quality of the three-channel composite image and avoid affecting the accuracy of three-dimensional reconstruction.

[0049] S204, inputting the three-channel composite image into a pre-trained convolutional neural network for reliability recognition, and outputting a corresponding point-by-point classification map according to the recognition result.

[0050] In this embodiment, a convolutional neural network is pre-trained to identify the reliability of pixels. Specifically, a lightweight convolutional neural network is first determined, such as U-Net. U-Net is a popular network for image segmentation. It has a symmetrical encoder-decoder structure and can learn rich feature representations from images. The convolutional neural network is trained by labeled training data. The specific training data may include a three-channel composite image and its corresponding point-by-point classification map (i.e., a mask map). The point-by-point classification map is an image of the same size as the three-channel composite image, wherein the value of each pixel indicates whether the point is reliable (for example, 1 indicates reliable, and 0 indicates unreliable). During the training process, the convolutional neural network learns how to map the three-channel composite image to the point-by-point classification map. When the convolutional neural network is trained, the three-channel composite image of the target object is used as input, and the convolutional neural network is used to identify whether each pixel in the image is reliable, and outputs the corresponding point-by-point classification map. Reliable pixels and unreliable pixels can be distinguished by the point-by-point classification map.

[0051] Since there are many unreliable points in the wrapped phase before phase unwrapping, if the wrapped phase image is directly unwrapped, the unwrapping ability will be weakened and the accuracy will be affected. The unreliable point wrapping stage mainly includes background points and unreliable target points. Different scenes require different thresholds to correctly remove unreliable points. By training and deploying convolutional neural networks, the three-channel composite image combining phase, modulation and background is reliability identified, so that the unreliable points in the image can be automatically and accurately identified before phase unwrapping, so as to extract reliable points for subsequent phase unwrapping, thereby improving the accuracy of phase unwrapping.

[0052] S205 , performing phase unwrapping according to the point-by-point classification map and the wrapped phase information, and constructing three-dimensional contour information of the target object in combination with preset calibration parameters.

[0053] In this embodiment, a point-by-point classification map output by convolutional neural network recognition is obtained, and the point-by-point classification map distinguishes reliable points from unreliable points in the image. Phase unfolding is performed based on the point-by-point classification map and the wrapped phase information, that is, the point-by-point classification map is used to guide the phase unfolding process, such as only performing phase unfolding on pixels marked as reliable in the classification map, deleting or repairing the phase of unreliable points, etc., so as to unfold the jump in the wrapped phase (i.e., the "wrapping" effect) and restore it to its true, continuous phase value, so that the unfolded phase value is no longer subject to periodic restrictions and can directly reflect the height change of the object surface. Combined with preset calibration parameters (such as calibration parameters of cameras and projectors), the unfolded phase information is converted into the height information of the object, thereby constructing the three-dimensional contour of the target object and achieving efficient and accurate three-dimensional measurement.

[0054] In the above embodiment, the present invention discloses a three-dimensional reconstruction method based on fusion phase unwrapping, by collecting deformed fringe images of the target object at different phases; processing the deformed fringe images at different phases to obtain single-channel wrapped phase information, modulation intensity information and background intensity information; fusing and constructing a corresponding three-channel composite image according to the single-channel wrapped phase information, modulation intensity information and background intensity information; inputting the three-channel composite image into a pre-trained convolutional neural network for reliability identification, and outputting a corresponding point-by-point classification map according to the identification result; performing phase unwrapping according to the point-by-point classification map and the wrapped phase information, and constructing the three-dimensional contour information of the target object in combination with preset calibration parameters. By extracting the wrapped phase, modulation intensity and background intensity information from the deformed fringe images at different phases and fusing them to construct a composite image, the multi-channel information fusion makes it possible to automatically and accurately identify unreliable points before phase unwrapping, thereby improving the phase unwrapping accuracy, better adapting to the phase unwrapping requirements in complex scenes, and improving the quality and effect of three-dimensional reconstruction.

[0055] In one embodiment, Figure 3 As shown, step S201 includes:

[0056] S301, generating an initial fringe pattern, and performing phase modulation on the initial fringe pattern to obtain a plurality of target fringe patterns with different phases;

[0057] S302, sending a first trigger signal and a second trigger signal to the projector and the monocular camera respectively according to a preset acquisition frequency;

[0058] S303, using the projector to project a target fringe pattern with different phases onto the target object each time the first trigger signal is received, and using the monocular camera to capture a deformed fringe image on the current target object each time the second trigger signal is received, until the deformed fringe images of the target fringe patterns of all phases projected onto the target object are captured.

[0059] In this embodiment, an initial black and white stripe pattern is first designed and generated, and then the initial pattern is modulated by phase modulation technology (such as phase shift method) to generate several target stripe patterns with different phases, for example, four stripe patterns with different phases (0°, 45°, 90°, 135°) are modulated. These patterns will be used to project onto the target object in order to capture the three-dimensional information of its surface. When collecting stripe images, a collection frequency is pre-set, for example, an image is collected every 100 milliseconds. When the projector is ready to project the stripe pattern of the next phase, the system automatically sends a first trigger signal to the projector and sends a second trigger signal to the monocular camera. The first trigger signal and the second trigger signal can be triggered synchronously or asynchronously to ensure that the camera captures the image at the same time or immediately after the projector projects the new phase pattern, and ensure that the projector and the camera can coordinate and work according to the preset collection frequency. The time interval between the two trigger signals can be flexibly set according to the needs, and this embodiment does not limit this.

[0060] During the acquisition of the deformed fringe image, the projector will sequentially project four fringe patterns of different phases onto the target object upon receiving the first trigger signal, and after each projection, the monocular camera will immediately capture the deformed fringe image on the target object at the current phase upon receiving the second trigger signal. This process will be repeated until all fringe patterns of four phases are projected and captured. For example, first project a fringe pattern of 0° phase, and the camera captures the image; then project a fringe pattern of 45° phase, and the camera captures the image again; then project a fringe pattern of 90° and 135° phases, and repeat the above steps until the deformed fringe images of the target fringe pattern projected on the target object at all phases are acquired. By continuously acquiring deformed fringe images at different phases, the deformation information of the target object surface under fringe patterns of different phases is included, providing a complete data set for subsequent phase unwrapping and three-dimensional reconstruction, ensuring the accuracy and reliability of three-dimensional reconstruction.

[0061] In one embodiment, Figure 4 As shown, step S202 includes:

[0062] S401, obtaining the intensity value of each pixel point in the deformed fringe image at different phases, and determining the maximum intensity value and the minimum intensity value of each pixel point among multiple intensity values ​​at different phases;

[0063] S402, calculating the phase value of each pixel point according to the multiple intensity values ​​at different phases to obtain the wrapped phase information of the first channel;

[0064] S403, calculating the modulation intensity of each pixel point according to the maximum intensity value and the minimum intensity value of each pixel point, and obtaining the modulation intensity information of the second channel;

[0065] S404: Estimating the background intensity according to the multiple intensity values ​​at different phases, calculating the background intensity of each pixel, and obtaining background intensity information of a third channel.

[0066] In this embodiment, after the deformed fringe images at different phases are acquired, the grayscale value (intensity value) of each pixel in the image in the different phase images is acquired. For example, if there are deformed fringe images with four different phases (0°, 45°, 90°, 135°), the intensity values ​​I1(x,y), I2(x,y), I3(x,y), and I4(x,y) of each pixel (x,y) in the fringe images with four different phases are acquired, and the maximum intensity value Imax(x,y) and the minimum intensity value Imin(x,y) of the pixel (x,y) are also found from these four values, which provide basic data for subsequent phase calculation and modulation intensity calculation.

[0067] Specifically, the phase value of each pixel is calculated based on multiple intensity values ​​at different phases through the phase shift algorithm. For example, for a pixel (x, y), its phase value φ(x, y) can be calculated by the following formula:

[0068]

[0069] By calculating the wrapped phase value, the wrapped phase information of the first channel is obtained, which provides key phase information for three-dimensional reconstruction. And on the basis of the wrapped phase information, the modulation intensity of each pixel is further calculated according to the maximum intensity value and minimum intensity value of each pixel in multiple fringe images to obtain the modulation intensity information of the second channel. Specifically, the modulation intensity M(x,y) = [Imax(x,y)-Imin(x,y)] / [Imax(x,y)+Imin(x,y)]. The modulation intensity information reflects the modulation effect of the fringe pattern on the surface of the object, reflects the visibility and quality of the fringe pattern, and helps to identify the surface features of the object and improve the robustness of the phase unwrapping.

[0070] Furthermore, background intensity estimation is performed based on multiple intensity values ​​at different phases, and the background intensity of each pixel is calculated. For example, the background intensity estimation is performed by the average intensity method, that is, the average intensity value of each pixel at different phases is calculated as the background intensity. Since the background light (such as ambient light or the part of the projector that is not reflected by the surface of the object) is relatively stable in the stripe images of different phases and does not change with the phase change, the intensity of the two stripe patterns will change periodically with the phase change. In the stripe images using four different phases, the intensity value of each pixel will be different due to the brightness change of the stripes. When the stripe images of four different phases are averaged, due to the periodic change of the stripe pattern, its intensity values ​​will cancel each other out during the averaging process, while the background light intensity will not cancel out during the averaging process due to its stability, and thus will be retained in the final average image.

[0071] Background intensity information helps to distinguish the surface of an object from the background and reduce the impact of background noise on phase unwrapping. In complex scenes, background intensity information can be used to effectively identify and exclude background noise, which can improve the accuracy of phase unwrapping and thus improve the quality of the entire 3D reconstruction.

[0072] In one embodiment, Figure 5 As shown, step S203 includes:

[0073] S501, performing data alignment and normalization processing on the wrapped phase information, modulation intensity information and background intensity information of each single channel to obtain three sets of spatially aligned and normalized single channel data;

[0074] S502 , assigning a corresponding weight value to each channel according to a preset weight strategy, performing weighted fusion processing on the three groups of single-channel data according to the weight value of each channel, and constructing the three-channel composite image.

[0075] In this embodiment, when fusing and constructing a three-channel composite image, the wrapped phase information, modulation intensity information, and background intensity information of each single channel are first aligned and normalized. For example, three single-channel data sets include wrapped phase information (P), modulation intensity information (M), and background intensity information (B). Each data set contains the same number of pixels, but their numerical ranges may be different. Therefore, these data are first spatially aligned to have the same spatial resolution and alignment to ensure that the position of each pixel in the three data sets is consistent. Then, each data set is normalized to unify its numerical range. For example, all data are scaled to the range of [0,1] by minimum-maximum normalization, Z-score normalization, etc. The consistency between data of different channels is ensured by data alignment, and the data of different channels are made comparable by normalization, eliminating the influence caused by differences in numerical ranges, and providing a unified standard for subsequent weighted fusion.

[0076] According to the preset weight strategy, a corresponding weight value is assigned to each channel. The specific weight value can be used to optimize the modulation according to different application scenarios. For example, the weights assigned to the wrapped phase information (P), modulation intensity information (M) and background intensity information (B) are 0.5, 0.3 and 0.2 respectively. The normalized single-channel data is weighted fused by the configured weights to construct a three-channel composite image (C). The value of each pixel in the composite image can be obtained by the following formula C=P norm ×0.5+M norm ×0.3+B norm ×0.2, where P norm 、M norm and B norm They are the normalized wrapped phase, modulation intensity and background intensity values ​​respectively. Through weighted fusion processing, the information of different channels can be comprehensively considered to construct a three-channel composite image containing more information, which allows the phase information, modulation intensity and background intensity to be considered in subsequent expansion, better handles discontinuity and noise problems in complex scenes, and improves the accuracy and robustness of phase expansion.

[0077] In one embodiment, Figure 6 As shown, step S204 includes:

[0078] S601, inputting the three-channel composite image into a pre-trained convolutional neural network, performing multi-scale feature extraction on the three-channel composite image, and obtaining a plurality of feature maps of different sizes;

[0079] S602, upsampling the feature maps of different sizes to obtain a plurality of upsampled feature maps;

[0080] S603, performing feature fusion on the plurality of up-sampled feature maps and the plurality of feature maps of different sizes to generate a fused feature map;

[0081] S604, performing reliability identification on the fused feature map, and outputting the reliability category of each pixel in the fused feature map;

[0082] S605 , assigning corresponding pixel values ​​according to the reliability category of each pixel point to generate the point-by-point classification map.

[0083] In this embodiment, the three-channel composite image is sent to the convolutional neural network as input data, and the image is processed through multiple convolutional layers and pooling layers, and features of different levels are gradually extracted to generate feature maps of multiple sizes. For example, a feature map of 1 / 2 size may be obtained after the first layer, and a feature map of 1 / 4 size may be obtained after the second layer, and so on. After obtaining feature maps of different sizes, the sizes of these feature maps are gradually enlarged by upsampling to restore them to the size of the original image. For example, a feature map of 1 / 4 size is upsampled back to 1 / 2 size, and then upsampled back to the original size. The upsampled feature map is fused with the original feature map of the corresponding size. For example, the upsampled feature map of 1 / 2 size is fused with the original feature map of 1 / 2 size, and the upsampled feature map of 1 / 4 size is fused with the original feature map of 1 / 4 size, etc. Feature fusion integrates feature information of different levels, which helps to retain the detail information in the image. The fused feature map is sent to the output layer of the network, and an activation function (such as the sigmoid function) is applied to map the feature value of each pixel to between 0 and 1 to represent the probability that each pixel belongs to each category, and then the reliability category of each pixel is output, for example, the pixel is marked as "reliable" or "unreliable", and then the corresponding value is assigned to each pixel according to the reliability category of each pixel. For example, a reliable pixel is assigned a value of 1, and an unreliable pixel is assigned a value of 0, thereby generating a point-by-point classification map. Each pixel value in the map represents the reliability of the corresponding pixel, which provides an important mask for phase unwrapping and can be used to guide the phase unwrapping process and improve the accuracy of the unwrapping.

[0084] In one embodiment, Figure 7 As shown, step S205 includes:

[0085] S701, retaining the phase values ​​of reliable pixels in the parcel phase information according to the pixel values ​​of each pixel in the point-by-point classification map, and correcting the phase values ​​of unreliable pixels in the parcel phase information;

[0086] S702, integrating the phase value of the reliable pixel point with the corrected phase value of the unreliable pixel point to generate a complete absolute phase map;

[0087] S703, calculating the height value of each pixel point on the surface of the target object relative to the reference plane according to the absolute phase map and preset calibration parameters;

[0088] S704: Convert the height value into three-dimensional coordinates to generate three-dimensional contour information of the target object.

[0089] In this embodiment, based on the pixel values ​​of each pixel in the point-by-point classification map, which pixels are reliable pixels and which pixels are unreliable pixels are identified, and the phase values ​​of all pixels are included in the wrapped phase information. Under the guidance of the point-by-point classification map, for pixels marked as reliable, the corresponding phase values ​​are directly retained, and for unreliable pixels, their phase values ​​are corrected by interpolating neighboring reliable pixels or using other algorithms (such as least squares method, etc.), ensuring that only reliable phase information is used in the phase unwrapping process, while correcting unreliable phase values, thereby improving the accuracy and robustness of phase unwrapping.

[0090] After correcting the phase values ​​of unreliable pixels, these corrected values ​​are combined with the phase values ​​of the originally retained reliable pixels to generate a complete absolute phase map, so that each pixel in the map has a certain phase value, indicating its position relative to the reference phase. Based on the absolute phase map and preset calibration parameters, the height value of each pixel on the surface of the target object relative to the reference plane is calculated. For example, the relationship between the absolute phase and the object height is:

[0091]

[0092] Among them, φ is the absolute phase, λ is the wavelength of the projection light source, h is the height of the target object surface point relative to the reference plane, and d is the distance between the monocular and the projector. According to the above relationship, after obtaining the absolute phase map, the height value can be calculated in combination with various preset parameters. Then the height value is further converted into a three-dimensional coordinate, that is, for each pixel in the image, its position (x, y) in the image and the corresponding height value h, as well as the internal and external parameters of the camera, are used to calculate its three-dimensional coordinates (X, Y, Z) in the world coordinate system to generate the three-dimensional contour information of the target object. The three-dimensional contour information can be further applied to three-dimensional printing, virtual reality, augmented reality, etc., to achieve accurate and reliable three-dimensional reconstruction.

[0093] In one embodiment, the pixel value of the reliable pixel in the point-by-point classification map is 1, and the pixel value of the unreliable pixel is 0, and the step S701 includes:

[0094] Performing a point-by-point multiplication operation on the point-by-point classification map and the wrapped phase information to obtain a multiplication result, wherein the phase value of the reliable pixel point is retained in the multiplication result, and the phase value of the unreliable pixel point is set to 0;

[0095] Phase correction is performed on the unreliable pixel points that are set to 0 in the multiplication result to obtain a corrected phase value of the unreliable pixel point.

[0096] In this embodiment, the point-by-point classification map is used to identify whether each pixel is reliable by a pixel value of 1 or 0. When phase unfolding is performed under the guidance of the point-by-point classification map, the point-by-point classification map and the wrapped phase information are multiplied point by point to obtain a multiplication result. At this time, the phase value of the reliable pixel is retained in the multiplication result, and the phase value of the unreliable pixel is set to 0, so as to quickly and accurately screen out the unreliable pixel points that need to be corrected. For the unreliable pixel points marked as 0 in the point-by-point classification map, the difference, neighborhood information or other algorithms (such as least squares method, iterative algorithm, etc.) are further used to estimate the correct phase values ​​of these points, so as to eliminate the influence of unreliable points as much as possible and achieve accurate phase unfolding.

[0097] It should be noted that there is not necessarily a certain order between the above-mentioned steps. A person skilled in the art can understand, based on the description of the embodiments of the present invention, that in different embodiments, the above-mentioned steps may have different execution orders, that is, they may be executed in parallel, may be executed interchangeably, and so on.

[0098] Further references Figure 8 , as a response to the above Figure 2 The present invention provides an embodiment of a three-dimensional reconstruction device based on fusion phase unwrapping, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0099] like Figure 8 As shown, the three-dimensional reconstruction device 80 based on fusion phase unwrapping described in this embodiment includes:

[0100] An image acquisition module 801 is used to acquire deformed fringe images of a target object at different phases;

[0101] An image processing module 802 is used to process the deformed fringe images at different phases to obtain single-channel wrapping phase information, modulation intensity information, and background intensity information;

[0102] A multi-channel fusion module 803 is used to fuse and construct a corresponding three-channel composite image according to the wrapped phase information, modulation intensity information and background intensity information of the single channel;

[0103] The point-by-point classification module 804 is used to input the three-channel composite image into a pre-trained convolutional neural network for reliability recognition, and output a corresponding point-by-point classification map according to the recognition result;

[0104] The phase unwrapping and reconstruction module 805 is used to perform phase unwrapping according to the point-by-point classification map and the wrapped phase information, and construct the three-dimensional contour information of the target object in combination with preset calibration parameters.

[0105] The module referred to in the present invention refers to a series of computer program instruction segments that can perform specific functions, which is more suitable for describing the execution process of three-dimensional reconstruction based on fusion phase unfolding than a program. For the specific implementation of each module, please refer to the corresponding method embodiment above, which will not be repeated here.

[0106] In one embodiment, the image acquisition module 801 includes:

[0107] A pattern generating unit, used to generate an initial fringe pattern, and perform phase modulation on the initial fringe pattern to obtain a plurality of target fringe patterns with different phases;

[0108] A signal sending unit, used to send a first trigger signal and a second trigger signal to the projector and the monocular camera respectively according to a preset acquisition frequency;

[0109] The projection and acquisition unit is used to use the projector to project a target fringe pattern with different phases onto the target object each time the first trigger signal is received, and to use the monocular camera to acquire a deformed fringe image on the current target object each time the second trigger signal is received, until the deformed fringe images of the target fringe patterns of all phases projected onto the target object are acquired.

[0110] In one embodiment, the image processing module 802 includes:

[0111] A data acquisition unit, used to acquire the intensity value of each pixel point in the deformed fringe image at different phases, and determine the maximum intensity value and the minimum intensity value of each pixel point among the multiple intensity values ​​at different phases;

[0112] A phase calculation unit, used to calculate the phase value of each pixel point according to the multiple intensity values ​​at different phases to obtain the wrapped phase information of the first channel;

[0113] A modulation intensity calculation unit, used to calculate the modulation intensity of each pixel point according to the maximum intensity value and the minimum intensity value of each pixel point, so as to obtain the modulation intensity information of the second channel;

[0114] The background intensity calculation unit is used to estimate the background intensity according to the multiple intensity values ​​under different phases, calculate the background intensity of each pixel point, and obtain the background intensity information of the third channel.

[0115] In one embodiment, the multi-channel fusion module 803 includes:

[0116] A data processing unit, used for performing data alignment and normalization processing on the wrapped phase information, modulation intensity information and background intensity information of each single channel, to obtain three sets of spatially aligned and normalized single channel data;

[0117] The weighted fusion unit is used to assign a corresponding weight value to each channel according to a preset weight strategy, and perform weighted fusion processing on the three groups of single-channel data according to the weight value of each channel to construct the three-channel composite image.

[0118] In one embodiment, the point-by-point classification module 804 includes:

[0119] A feature extraction unit, used for inputting the three-channel composite image into a pre-trained convolutional neural network, performing multi-scale feature extraction on the three-channel composite image, and obtaining a plurality of feature maps of different sizes;

[0120] An upsampling unit, used for upsampling the feature maps of different sizes to obtain a plurality of upsampled feature maps;

[0121] A feature fusion unit, used to perform feature fusion on the plurality of upsampled feature maps and the plurality of feature maps of different sizes to generate a fused feature map;

[0122] A reliability identification unit, used to perform reliability identification on the fused feature map and output a reliability category of each pixel in the fused feature map;

[0123] The classification output unit is used to assign corresponding pixel values ​​according to the reliability category of each pixel point to generate the point-by-point classification map.

[0124] In one embodiment, the phase unwrapping and reconstruction module 805 includes:

[0125] A phase extraction and correction unit, used to retain the phase values ​​of reliable pixels in the wrapped phase information and correct the phase values ​​of unreliable pixels in the wrapped phase information according to the pixel values ​​of each pixel in the point-by-point classification map;

[0126] A phase integration unit, used to integrate the phase value of the reliable pixel point with the corrected phase value of the unreliable pixel point to generate a complete absolute phase map;

[0127] A height calculation unit, which calculates the height value of each pixel point on the surface of the target object relative to a reference plane according to the absolute phase map and preset calibration parameters;

[0128] The three-dimensional reconstruction unit is used to convert the height value into three-dimensional coordinates to generate three-dimensional contour information of the target object.

[0129] In one embodiment, the pixel value of the reliable pixel point in the point-by-point classification map is 1, and the pixel value of the unreliable pixel point is 0, and the phase extraction correction unit includes:

[0130] A multiplication unit, configured to perform a point-by-point multiplication operation on the point-by-point classification map and the wrapped phase information to obtain a multiplication result, wherein the phase value of the reliable pixel point is retained in the multiplication result, and the phase value of the unreliable pixel point is set to 0;

[0131] The correction unit is used to perform phase correction on the unreliable pixel points that are set to 0 in the multiplication result to obtain the corrected phase value of the unreliable pixel points.

[0132] In the above embodiment, the present invention discloses a three-dimensional reconstruction device based on fusion phase unwrapping, which collects deformed fringe images of the target object at different phases; processes the deformed fringe images at different phases to obtain single-channel wrapped phase information, modulation intensity information and background intensity information; fuses and constructs a corresponding three-channel composite image based on the single-channel wrapped phase information, modulation intensity information and background intensity information; inputs the three-channel composite image into a pre-trained convolutional neural network for reliability identification, and outputs a corresponding point-by-point classification map based on the identification result; performs phase unwrapping based on the point-by-point classification map and the wrapped phase information, and constructs the three-dimensional contour information of the target object in combination with preset calibration parameters. By extracting the wrapped phase, modulation intensity and background intensity information from the deformed fringe images at different phases and fusing them to construct a composite image, the multi-channel information fusion makes it possible to automatically and accurately identify unreliable points before phase unwrapping, thereby improving the phase unwrapping accuracy, better adapting to the phase unwrapping requirements in complex scenes, and improving the quality and effect of three-dimensional reconstruction.

[0133] Another embodiment of the present invention provides a computer device, such as Fig. 9 As shown, the computer device 90 includes:

[0134] One or more processors 901 and memory 902, Fig. 9A processor 901 is used as an example for description. The processor 901 and the memory 902 may be connected via a bus or other means. Fig. 9 The example of connecting through bus is taken in the following.

[0135] Processor 901 is used to complete various control logics of computer device 90, which can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISC Machine) or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component or any combination of these components. In addition, processor 901 can also be any traditional processor, microprocessor or state machine. Processor 901 can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP and / or any other such configuration.

[0136] The memory 902 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions corresponding to the three-dimensional reconstruction method based on fusion phase unwrapping in the embodiment of the present invention. The processor 901 executes various functional applications and data processing of the computer device 90 by running the non-volatile software programs, instructions and units stored in the memory 902, that is, realizing the three-dimensional reconstruction method based on fusion phase unwrapping in the above method embodiment.

[0137] The memory 902 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created according to the use of the computer device 90, etc. In addition, the memory 902 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 902 may optionally include a memory remotely arranged relative to the processor 901, and these remote memories may be connected to the computer device 90 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. One or more units are stored in the memory 902, and when executed by one or more processors 901, the steps of the three-dimensional reconstruction method based on fusion phase unwrapping in any of the above-mentioned method embodiments are executed.

[0138] In the above embodiment, the present invention discloses a computer device, which collects deformed fringe images of a target object at different phases; processes the deformed fringe images at different phases to obtain single-channel wrapped phase information, modulation intensity information and background intensity information; fuses and constructs a corresponding three-channel composite image based on the single-channel wrapped phase information, modulation intensity information and background intensity information; inputs the three-channel composite image into a pre-trained convolutional neural network for reliability identification, and outputs a corresponding point-by-point classification map based on the identification result; performs phase unfolding based on the point-by-point classification map and the wrapped phase information, and constructs the three-dimensional contour information of the target object in combination with preset calibration parameters. By extracting the wrapped phase, modulation intensity and background intensity information from the deformed fringe images at different phases and fusing them to construct a composite image, the multi-channel information fusion makes it possible to automatically and accurately identify unreliable points before phase unfolding, thereby improving the phase unfolding accuracy, better adapting to the phase unfolding requirements in complex scenes, and improving the quality and effect of three-dimensional reconstruction.

[0139] An embodiment of the present invention provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the steps of the three-dimensional reconstruction method based on fusion phase unfolding in any of the above method embodiments are executed.

[0140] In the above embodiment, the present invention discloses a non-volatile computer-readable storage medium, which collects deformed fringe images of a target object at different phases; processes the deformed fringe images at different phases to obtain single-channel wrapped phase information, modulation intensity information, and background intensity information; fuses and constructs a corresponding three-channel composite image based on the single-channel wrapped phase information, modulation intensity information, and background intensity information; inputs the three-channel composite image into a pre-trained convolutional neural network for reliability identification, and outputs a corresponding point-by-point classification map based on the identification result; performs phase unfolding based on the point-by-point classification map and the wrapped phase information, and constructs the three-dimensional contour information of the target object in combination with preset calibration parameters. By extracting the wrapped phase, modulation intensity, and background intensity information from the deformed fringe images at different phases and fusing them to construct a composite image, the multi-channel information fusion makes it possible to automatically and accurately identify unreliable points before phase unfolding, thereby improving the phase unfolding accuracy, better adapting to the phase unfolding requirements in complex scenes, and improving the quality and effect of three-dimensional reconstruction.

[0141] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0142] The present invention can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0143] In summary, the three-dimensional reconstruction method, device, equipment and medium based on fusion phase unwrapping disclosed in the present invention include: collecting deformed fringe images of the target object at different phases; processing the deformed fringe images at different phases to obtain single-channel wrapped phase information, modulation intensity information and background intensity information; fusing and constructing a corresponding three-channel composite image based on the single-channel wrapped phase information, modulation intensity information and background intensity information; inputting the three-channel composite image into a pre-trained convolutional neural network for reliability identification, and outputting a corresponding point-by-point classification map based on the identification result; performing phase unwrapping based on the point-by-point classification map and the wrapped phase information, and constructing the three-dimensional contour information of the target object in combination with preset calibration parameters. By extracting the wrapped phase, modulation intensity and background intensity information from the deformed fringe images at different phases and fusing them to construct a composite image, the multi-channel information fusion makes it possible to automatically and accurately identify unreliable points before phase unwrapping, thereby improving the phase unwrapping accuracy, better adapting to the phase unwrapping requirements in complex scenes, and improving the quality and effect of three-dimensional reconstruction.

[0144] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium, and the computer program can include the processes of the above-mentioned method embodiments when executed. The storage medium can be a memory, a disk, a floppy disk, a flash memory, an optical storage device, etc.

[0145] It should be noted that if any software tools or components other than those of the Company appear in the embodiments of the present application, they are merely used for illustration and do not represent actual use. It should be understood that the application of the present invention is not limited to the above examples, and that those skilled in the art can make improvements or changes based on the above description, and all such improvements and changes shall fall within the scope of protection of the claims attached to the present invention.

Claims

1. A three-dimensional reconstruction method based on fusion phase unwrapping, characterized in that: include: Collect deformed fringe images of the target object at different phases; Processing the deformed fringe images at different phases to obtain single-channel wrapping phase information, modulation intensity information, and background intensity information; According to the wrapped phase information, modulation intensity information and background intensity information of the single channel, a corresponding three-channel composite image is constructed by fusing; Inputting the three-channel composite image into a pre-trained convolutional neural network for reliability recognition, and outputting a corresponding point-by-point classification map according to the recognition result; Phase unwrapping is performed according to the point-by-point classification map and the wrapped phase information, and the three-dimensional contour information of the target object is constructed in combination with preset calibration parameters.

2. The three-dimensional reconstruction method based on fusion phase unwrapping according to claim 1, characterized in that: The collecting of deformed fringe images of the target object at different phases includes: generating an initial fringe pattern, performing phase modulation on the initial fringe pattern, and obtaining a plurality of target fringe patterns with different phases; Sending a first trigger signal and a second trigger signal to the projector and the monocular camera respectively according to a preset acquisition frequency; The projector is used to project a target fringe pattern with different phases onto the target object each time the first trigger signal is received, and the monocular camera is used to capture a deformed fringe image on the current target object each time the second trigger signal is received, until the deformed fringe images of the target fringe patterns of all phases projected onto the target object are captured.

3. The three-dimensional reconstruction method based on fusion phase unwrapping according to claim 1, characterized in that: The processing of the deformed fringe images at different phases to obtain single-channel wrapping phase information, modulation intensity information, and background intensity information includes: Acquire the intensity value of each pixel point in the deformed fringe image at the different phases, and determine the maximum intensity value and the minimum intensity value of each pixel point among the multiple intensity values ​​at the different phases; Calculate the phase value of each pixel point according to the multiple intensity values ​​at different phases to obtain the wrapped phase information of the first channel; Calculating the modulation intensity of each pixel point according to the maximum intensity value and the minimum intensity value of each pixel point to obtain the modulation intensity information of the second channel; The background intensity is estimated according to the multiple intensity values ​​at different phases, the background intensity of each pixel is calculated, and the background intensity information of the third channel is obtained.

4. The three-dimensional reconstruction method based on fusion phase unwrapping according to claim 1, characterized in that: The step of fusing and constructing a corresponding three-channel composite image according to the wrapped phase information, the modulation intensity information and the background intensity information of the single channel includes: Perform data alignment and normalization on the wrapped phase information, modulation intensity information, and background intensity information of each single channel to obtain three sets of spatially aligned and normalized single channel data; A corresponding weight value is assigned to each channel according to a preset weight strategy, and weighted fusion processing is performed on the three groups of single-channel data according to the weight value of each channel to construct the three-channel composite image.

5. The three-dimensional reconstruction method based on fusion phase unwrapping according to claim 1, characterized in that: The three-channel composite image is input into a pre-trained convolutional neural network for reliability recognition, and a corresponding point-by-point classification map is output according to the recognition result, including: Inputting the three-channel composite image into a pre-trained convolutional neural network, performing multi-scale feature extraction on the three-channel composite image, and obtaining a plurality of feature maps of different sizes; Upsampling the feature maps of different sizes to obtain a plurality of upsampled feature maps; Performing feature fusion on the plurality of upsampled feature maps and the plurality of feature maps of different sizes to generate a fused feature map; Performing reliability identification on the fused feature map, and outputting the reliability category of each pixel in the fused feature map; A corresponding pixel value is assigned according to the reliability category of each pixel point to generate the point-by-point classification map.

6. The three-dimensional reconstruction method based on fusion phase unwrapping according to claim 1, characterized in that: The performing phase unwrapping according to the point-by-point classification map and the wrapped phase information, and constructing the three-dimensional contour information of the target object in combination with preset calibration parameters, includes: According to the pixel values ​​of each pixel in the point-by-point classification map, the phase values ​​of reliable pixels in the parcel phase information are retained, and the phase values ​​of unreliable pixels in the parcel phase information are corrected; Integrating the phase value of the reliable pixel point with the corrected phase value of the unreliable pixel point to generate a complete absolute phase map; Calculating the height value of each pixel point on the surface of the target object relative to the reference plane according to the absolute phase map and preset calibration parameters; The height value is converted into a three-dimensional coordinate to generate three-dimensional contour information of the target object.

7. The three-dimensional reconstruction method based on fusion phase unwrapping according to claim 6, characterized in that: The pixel value of the reliable pixel in the point-by-point classification map is 1, the pixel value of the unreliable pixel is 0, and the phase value of the reliable pixel in the parcel phase information is retained according to the pixel value of each pixel in the point-by-point classification map, and the phase value of the unreliable pixel in the parcel phase information is corrected, including: Performing a point-by-point multiplication operation on the point-by-point classification map and the wrapped phase information to obtain a multiplication result, wherein the phase value of the reliable pixel point is retained in the multiplication result, and the phase value of the unreliable pixel point is set to 0; Phase correction is performed on the unreliable pixel points that are set to 0 in the multiplication result to obtain a corrected phase value of the unreliable pixel point.

8. A three-dimensional reconstruction device based on fusion phase unwrapping, characterized in that: include: An image acquisition module, used for acquiring deformed fringe images of a target object at different phases; An image processing module is used to process the deformed fringe images at different phases to obtain single-channel wrapping phase information, modulation intensity information and background intensity information; A multi-channel fusion module, used for fusing and constructing a corresponding three-channel composite image according to the wrapped phase information, modulation intensity information and background intensity information of the single channel; A point-by-point classification module, used for inputting the three-channel composite image into a pre-trained convolutional neural network for reliability recognition, and outputting a corresponding point-by-point classification map according to the recognition result; The phase unwrapping and reconstruction module is used to perform phase unwrapping according to the point-by-point classification map and the wrapped phase information, and to construct the three-dimensional contour information of the target object in combination with preset calibration parameters.

9. A computer device, characterized in that: comprising at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the three-dimensional reconstruction method based on fusion phase unwrapping according to any one of claims 1 to 7.

10. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores computer-executable instructions, which, when executed by one or more processors, enable the one or more processors to execute the three-dimensional reconstruction method based on fusion phase unwrapping as described in any one of claims 1-7.

Images