Three-dimensional reconstruction method, device and system and storage medium

By combining Gray code and high-frequency inverse projection technology, the Gray code projection image, Gray code complement projection image and high-frequency inverse projection image are used to match left and right code jumps to generate a three-dimensional point cloud, which solves the existing technology It is susceptible to ambient light interference and insufficient accuracy, and achieves high-precision and high-density three-dimensional reconstruction.

CN118982622APending Publication Date: 2024-11-19HANGZHOU YUNJIA DIGITAL TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411124576.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-16
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

The existing three-dimensional structured light reconstruction technology is susceptible to ambient light interference, and the accuracy and point cloud density are insufficient, making it difficult to achieve high-precision three-dimensional reconstruction in complex scenarios.

Method used

The combination of Gray code projection images, Gray code complement projection images and high-frequency inverse projection images is used to determine the three-dimensional point cloud through the jump matching of left and right codes, and the defects of simple Gray code and multi-frequency phase shift encoding are overcome.

Benefits of technology

The accuracy and point cloud density of three-dimensional reconstruction are improved, the resistance to ambient light interference is enhanced, and high-precision three-dimensional reconstruction in complex scenarios is achieved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118982622A_ABST
    Figure CN118982622A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of three-dimensional measurement, in particular to a three-dimensional reconstruction method, device and system and a storage medium. The method comprises the steps that projection images generated by projection of a projection device to a target scene are acquired, the projection images comprise a Gray code projection image, a Gray code complement projection image and a high-frequency anti-phase projection image, and the high-frequency anti-phase projection image comprises multiple groups of black and white stripes; the Gray code image and the Gray code complement image are used for determining the hopping period of the multiple groups of black and white stripe edges; according to the Gray code projection image, the Gray code complement projection image and the high-frequency anti-phase projection image, left and right same-code hopping matching is carried out to determine left and right view hopping matching points; and performing triangulation based on the left and right view jump matching points to generate a three-dimensional point cloud corresponding to the target scene. According to the method, the Gray code and the high-frequency anti-phase projection image are combined, high-precision structured light reconstruction is realized, and the technical problems that the prior art is easily interfered by the environment and the precision and the point cloud density are not high enough are solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of three-dimensional measurement technology, and in particular to a three-dimensional reconstruction method, device, system and storage medium. Background Art

[0002] Binocular 3D reconstruction is the process of restoring the geometric structure and depth information of a 3D scene using image information obtained simultaneously by two cameras. Binocular systems can generally be divided into passive and active types. Passive binocular reconstruction does not require special light sources or projection equipment and can be performed using ordinary cameras. However, it has low accuracy in complex scenes or areas with less texture, is easily disturbed by ambient light, and has high computational complexity. Active binocular reconstruction projects or illuminates the scene with the help of special light sources or projection equipment (such as structured light, laser projection, etc.), and reconstructs after optically decoding the scene image. It has high accuracy and strong resistance to environmental interference.

[0003] Common structured light encoding methods include Gray code and multi-frequency phase shift. The accuracy of Gray code encoding depends on the width of the thinnest stripe. In practical applications, too thin stripes are easily interfered with, which eventually leads to decoding failure. Therefore, the point cloud density and accuracy of the pure Gray code method are limited. Although the multi-frequency phase shift encoding method obtains a higher point cloud density with thicker stripes, the sinusoidal stripes are still easily interfered by ambient light, resulting in phase decoding failure.

[0004] Therefore, it is necessary to propose a 3D reconstruction method with high accuracy, high point cloud density and not easily disturbed by ambient light. Summary of the invention

[0005] One purpose of an embodiment of the present application is to provide a three-dimensional reconstruction method to solve the technical problems in the current structured light reconstruction technology that is easily affected by environmental interference and has insufficient accuracy and point cloud density.

[0006] In a first aspect, an embodiment of the present application provides a three-dimensional reconstruction method, comprising:

[0007] Acquire a projection image generated by projecting the projection device onto the target scene, the projection image comprising a Gray code projection image, a Gray code complement projection image and a high-frequency inverted projection image, the high-frequency inverted projection image comprising a plurality of groups of black and white stripes, the Gray code image and the Gray code complement image being used to determine the period of edge jumps of the plurality of groups of black and white stripes;

[0008] According to the Gray code projection image, the Gray code complement projection image and the high-frequency inverted projection image, performing left and right same-code jump matching to determine left and right view jump matching points;

[0009] Based on the left and right view jump matching points, triangulation is performed to generate a three-dimensional point cloud corresponding to the target scene.

[0010] In combination with the first aspect, as a feasible implementation manner, before acquiring the projection image generated by the projection device projecting onto the target scene, the method further includes:

[0011] Determine a pattern sequence, the pattern sequence comprising a Gray code pattern sequence, a Gray code complement pattern sequence and a high frequency inversion pattern sequence, the Gray code pattern sequence and the;

[0012] According to the pattern sequence, the projection device is controlled to project toward the target scene to collect the Gray code projection image, the Gray code complement projection image and the high-frequency inverted projection image.

[0013] In combination with the first aspect, as a feasible implementation manner, determining the pattern sequence includes:

[0014] determining a resolution of the projection device;

[0015] Determining the Gray code pattern sequence according to the resolution of the projection device and a preset number of Gray code bits, wherein the number of Gray code bits corresponds to a minimum stripe width in the Gray code projection image;

[0016] The Gray code complement pattern sequence is determined according to the Gray code pattern sequence.

[0017] In combination with the first aspect, as a feasible implementation manner, the high-frequency inverted pattern sequence includes a plurality of groups of black and white stripes, and the method further includes:

[0018] Determining the total number of cycles corresponding to the Gray code complement and the stripe width of the Gray code complement pattern sequence;

[0019] Determine the number of groups of the black and white stripes according to the total number of preset black and white stripe edge transitions and the total number of cycles corresponding to the Gray code complement;

[0020] Determining the stripe width of the high-frequency inverted pattern sequence according to the stripe width of the Gray code complement pattern sequence;

[0021] The high-frequency inversion pattern sequence is determined according to the number of groups of the black and white stripes and the stripe width.

[0022] In combination with the first aspect, as a feasible implementation manner, the 3D reconstruction system further includes a left camera and a right camera, the left camera and the right camera are used to collect a left view and a right view respectively, and the left and right same-code jump matching is performed according to the Gray code projection image, the Gray code complement projection image and the high-frequency inverted projection image to determine the left and right view jump matching points, including:

[0023] Determining the sub-pixel position of the black-white edge transition in the high-frequency inverted projection image;

[0024] Determining the code value corresponding to the sub-pixel position in the Gray code projection image, the Gray code complement projection image and the high-frequency inverse projection image respectively;

[0025] According to the code value, determining the sub-pixel position of each black-white edge transition in the left view;

[0026] By using epipolar constraints, a straight line is determined in the right view, which corresponds to the sub-pixel position of each black-white voltage transition in the left view;

[0027] Based on the straight line where the corresponding points are located, the black and white edge transition sub-pixel positions with consistent black and white edge transition code values ​​are determined, and matching is performed to obtain the left and right view transition matching points.

[0028] In combination with the first aspect, as a feasible implementation manner, the high-frequency inverted projection image includes a plurality of black and white stripe groups, each of the black and white stripe groups includes a target image and an inverted image, the target image is a black and white pattern of equal width, the inverted image is an inverted black and white pattern of equal width corresponding to the target image, and determining the sub-pixel position of the black and white edge jump in the high-frequency inverted projection image includes:

[0029] determining the inverted image in the high-frequency inverted pattern projection image;

[0030] Performing difference processing on the inverted image to obtain a difference image;

[0031] Traversing each pixel of the difference image, determining the pixel point with opposite left and right differences as the position of black-white edge transition;

[0032] The sub-pixel position is determined by linear fitting, and the difference of the sub-pixel position is 0.

[0033] In combination with the first aspect, as a feasible implementation manner, the determining the code value corresponding to the sub-pixel position in the Gray code projection image, the Gray code complement projection image, and the high-frequency inverted projection image respectively includes:

[0034] Binarizing the Gray code projection image and the Gray code complement projection image;

[0035] According to the sub-pixel position, sequentially extracting the binary Gray code projection image and the Gray code complement projection image respectively to determine the Gray code and the Gray code complement;

[0036] Determine the black and white edge transition position of the high-frequency inverted projection image, and the black and white edge transition of the minimum width stripe in the Gray code;

[0037] Determine whether the black and white edge transition position of the high-frequency inverse projection image corresponds to the black and white edge transition of the minimum width stripe in the Gray code;

[0038] If yes, determining that the Gray code complement is the code value of the high-frequency inverse projection image;

[0039] If not, it is determined that the Gray code is the code value of the high-frequency inverse projection image.

[0040] In a second aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory is connected to the processor, and the processor is used to execute one or more computer programs stored in the memory. When the processor executes the one or more computer programs, the electronic device implements a method as described in any one of the first aspects above.

[0041] In a third aspect, an embodiment of the present application provides a three-dimensional reconstruction system, including:

[0042] A projection device, used for projecting onto a target scene;

[0043] The acquisition device includes a left camera and a right camera, which are used to acquire the projection image in the target scene;

[0044] The electronic device as described above is connected to the projection device and the acquisition device respectively.

[0045] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes any of the methods described above.

[0046] The embodiments of the present application can achieve the following technical effects:

[0047] Based on the method proposed in the embodiment of the present application, in view of the limited point cloud density and accuracy of the pure Gray code method, and although the multi-frequency phase shift encoding method obtains a higher point cloud density with thicker stripes, the sinusoidal stripes are still susceptible to interference from ambient light. Therefore, when realizing three-dimensional reconstruction, the embodiment of the present application combines Gray code with high-frequency inverted projection technology to project Gray code projection images, Gray code complement projection images and high-frequency inverted projection images into the scene respectively, so as to perform jump matching of the same code on the left and right views according to the three projection images, and jointly determine the jump matching points of the left and right views, and finally perform triangulation based on the jump matching points of the left and right views to generate a three-dimensional point cloud, thereby overcoming the defects of pure Gray code and pure multi-frequency phase shift encoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0049] Figure 1 A schematic diagram of an application scenario of a three-dimensional reconstruction system provided in an embodiment of the present application;

[0050] Figure 2 A schematic diagram of a flow chart of a three-dimensional reconstruction method provided in an embodiment of the present application;

[0051] Figure 3 is a schematic structural diagram of a three-dimensional reconstruction device provided in an embodiment of the present application;

[0052] Figure 4 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0054] It should be noted that, if there is no conflict, the various features in the embodiments of the present application can be combined with each other, all within the scope of protection of the present application. In addition, although the functional module division is performed in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a sequence different from the module division in the device or the flow chart. Furthermore, the words "first", "second", "third", etc. used in this application do not limit the data and execution order, but only distinguish the same items or similar items with basically the same functions and effects.

[0055] An embodiment of the present application provides a three-dimensional reconstruction method, which is applied to a three-dimensional reconstruction system.

[0056] In the first place, see Figure 1 In one application scenario, the three-dimensional reconstruction system 100 includes a projection device 11, a collection device 12, and a processing device 13. The projection device and any one of the projection devices are connected to the processing device 13 respectively.

[0057] The projection device 11 at least includes a structured light projector, and the projection device 11 is used to project onto a target scene to form a projection image.

[0058] The acquisition device 12 at least includes a left camera and a right camera, and the left camera and the right camera are used to acquire a left view and a right view respectively.

[0059] The processing device 13 may be a computer, including at least a processor and a memory. The processing device 13 processes the captured projection image through the processor, performs image decoding, matching and three-dimensional reconstruction, and stores the captured image data and the reconstructed three-dimensional point cloud model through the memory.

[0060] In addition, the 3D reconstruction system 100 may also include a display device, which is connected to the processing device 13 and is used to display the reconstructed 3D point cloud model and related image data, so that the user can intuitively view and analyze the reconstruction results. The display device may be a liquid crystal display, a 3D display, a VR (Virtual Reality) head mounted display, or the like.

[0061] In the second aspect, see Figure 2 The embodiment of the present application provides a three-dimensional reconstruction method, using the above-mentioned processing device as the execution subject, and the method includes:

[0062] Step S21, obtaining a projection image generated by projecting the target scene by the projection device, wherein the projection image includes a Gray code projection image, a Gray code complement projection image, and a high-frequency inverted projection image, wherein the high-frequency inverted projection image includes a plurality of groups of black and white stripes, and the Gray code image and the Gray code complement image are used to determine the period of edge jumps of the plurality of groups of black and white stripes;

[0063] It should be noted that before obtaining the projection image generated by the projection device projecting onto the target scene, the projection device should first be controlled to project onto the target scene, specifically including: determining a pattern sequence, the pattern sequence including a Gray code pattern sequence, a Gray code complement pattern sequence and a high-frequency inverted pattern sequence, the Gray code pattern sequence being consistent with the above; according to the pattern sequence, controlling the projection device to project onto the target scene to capture the Gray code projection image, the Gray code complement projection image and the high-frequency inverted projection image.

[0064] Specifically, the process of determining the pattern sequence includes: determining the resolution of the projection device; determining the Gray code pattern sequence according to the resolution of the projection device and a preset number of Gray code bits, wherein the number of Gray code bits corresponds to the minimum stripe width in the Gray code projection image; and determining the Gray code complement pattern sequence according to the Gray code pattern sequence.

[0065] Furthermore, the high-frequency inverted pattern sequence includes multiple groups of black and white stripes, and the method also includes: determining the total number of cycles corresponding to the Gray code complement and the stripe width of the Gray code complement pattern sequence; determining the number of groups of black and white stripes based on a preset total number of black and white stripe edge jumps and the total number of cycles corresponding to the Gray code complement; determining the stripe width of the high-frequency inverted pattern sequence based on the stripe width of the Gray code complement pattern sequence; determining the high-frequency inverted pattern sequence based on the number of groups of black and white stripes and the stripe width.

[0066] Specifically, the required pattern sequence can be generated according to the encoding method of the Gray code based on the resolution of the projector and the required number of Gray code bits. The required number of Gray code bits is determined by the minimum stripe width in the actually collected Gray code projection image sequence, which is based on the minimum stripe width that can be correctly decoded and the sub-pixel position where the black and white edge transition can be extracted in S3.

[0067] After that, a Gray code complement projection pattern sequence is designed. Since it is easy to make decoding errors at the black-white transition of the Gray code projection pattern, a Gray code complement projection pattern is added after the above Gray code projection pattern to improve the decoding stability. The complement pattern is generated according to the Gray code encoding method. The Gray code projection pattern sequence and the complement sequence determine the period where the black-white stripe edge transitions are located.

[0068] Finally, a high-frequency inverted projection pattern sequence is designed. The sequence consists of multiple groups of black and white stripes, where one group of black and white stripes includes a black and white pattern of equal width and a corresponding inverted black and white pattern of equal width. Subtracting the inverted images to obtain a difference image, and extracting the black and white jump edges on the difference image is beneficial to increase the extraction stability. The high-frequency inverted projection pattern is used to further determine the code value where the black and white stripe edge jumps. The required number of black and white stripe groups G is calculated based on the total number of black and white stripe edge jumps N and the total number of cycles M determined by the Gray code complement, and the stripe width of the high-frequency inverted projection pattern is set to 1 / 2 of the stripe width of the complement projection pattern. Therefore The starting positions of each group of black and white stripes differ A complement cycle, that is, 1 pixel.

[0069] Step S22, performing left and right same-code jump matching according to the Gray code projection image, the Gray code complement projection image and the high-frequency inverse projection image to determine left and right view jump matching points;

[0070] As described above, the three-dimensional reconstruction system also includes a left camera and a right camera, and the left camera and the right camera are used to collect the left view and the right view respectively. Then, according to the Gray code projection image, the Gray code complement projection image and the high-frequency inverted projection image, left and right same-code jump matching is performed to determine the left and right view jump matching points, including: determining the sub-pixel position of the black and white edge jump in the high-frequency inverted projection image; determining the code value corresponding to the sub-pixel position in the Gray code projection image, the Gray code complement projection image and the high-frequency inverted projection image respectively; according to the code value, determining the sub-pixel position of each black and white edge jump in the left view; through the polar line constraint, determining the straight line where the corresponding points corresponding to the sub-pixel position of each black and white voltage change jump in the left view are located in the right view; based on the straight line where the corresponding points are located, determine the black and white edge jump sub-pixel position with consistent black and white edge jump code value, match, and obtain the left and right view jump matching points.

[0071] Further, the high-frequency inverted projection image includes a plurality of black and white stripe groups, each of the black and white stripe groups includes a target image and an inverted image, the target image is a black and white pattern of equal width, the inverted image is an inverted black and white pattern of equal width corresponding to the target image, and determining the sub-pixel position of the black and white edge jump in the high-frequency inverted projection image includes: determining the inverted image in the high-frequency inverted pattern projection image; performing difference processing on the inverted image to obtain a difference image; traversing each pixel of the difference image to determine the pixel points with opposite left and right differences as the positions of the black and white edge jump; determining the sub-pixel position by linear fitting, and the difference of the sub-pixel position is 0.

[0072] Furthermore, the code values ​​corresponding to the sub-pixel positions are determined in the Gray code projection image, the Gray code complement projection image and the high-frequency inverted projection image, respectively, including: binarizing the Gray code projection image and the Gray code complement projection image; sequentially extracting the Gray code projection image and the Gray code complement projection image after the binarization according to the sub-pixel position to determine the Gray code and the Gray code complement; determining the black and white edge transition position of the high-frequency inverted projection image and the black and white edge transition of the minimum width stripe in the Gray code; determining whether the black and white edge transition position of the high-frequency inverted projection image corresponds to the black and white edge transition of the minimum width stripe in the Gray code; if so, determining that the Gray code complement is the code value of the high-frequency inverted projection image; if not, determining that the Gray code is the code value of the high-frequency inverted projection image.

[0073] It is easy to understand that the Gray code projection image sequence, the Gray code complement projection image sequence and the high frequency inverse projection image sequence are collected by the left and right cameras.

[0074] First, extract the sub-pixel position of the black-white edge jump in the high-frequency inverted projection image sequence, and subtract the inverted image to obtain the difference image. Search for pixel points with opposite left and right differences pixel by pixel on the difference image. For example, you can set the three consecutive differences on the left (right) side to be negative and the three consecutive differences on the right (left) side to be positive, then the pixel is the position of the black-white edge jump. Sub-pixelization finds the image position corresponding to the difference of 0 by linear fitting. Extract the sub-pixel position of the black-white edge jump in each group of black-white stripes.

[0075] Afterwards, the code values ​​corresponding to the black and white edge transition positions are calculated on the Gray code projection image sequence, the Gray code complement projection image sequence and the high frequency inverse projection image sequence, including:

[0076] On the Gray code projection image sequence, binarization is performed, and at the black and white edge jump position, the Gray code projection image sequence after binarization is extracted to obtain the Gray code, and the Gray code is converted into a decimal number K1, which is the period where the black and white edge jump position is located. The specific conversion method is: retain the highest bit of the Gray code as the highest bit of the binary code, and the second highest binary code is obtained by the difference between the high binary code and the second highest Gray code, and the remaining bits of the binary code are similar to the method of obtaining the second highest binary code, and the obtained binary code is converted into a decimal number.

[0077] The same operation as S4.1 is performed on the Gray code projection image sequence and the Gray code complement projection image sequence to obtain the decimal number V. According to the complement formula, K2=INT[(V+1) / 2] is obtained, where INT is rounded down. K2 is also the period where the black and white edge jump position is located.

[0078] The code value of the black-white edge transition is calculated on the high-frequency inverted projection image sequence. In the high-frequency inverted projection pattern designed in S1.3, if the black-white edge transition position corresponds to the black-white edge transition of the Gray code projection pattern with the minimum stripe width, then the cycle K selects K2, otherwise selects K1. The code value of the black-white edge transition C = 8K + g, where g is the black-white transition sub-sequence number from left to right in a single cycle in the high-frequency inverted projection image sequence, and the range is [0,8).

[0079] Finally, the left and right same-code transitions are matched under the epipolar constraint. For each black-white edge transition sub-pixel position in the left view, the epipolar constraint is used to obtain the straight line where the possible corresponding point in the right view is located; on this straight line, the black-white edge transition sub-pixel position with the same black-white edge transition code value is searched for matching.

[0080] Step S23: performing triangulation based on the left and right view jump matching points to generate a three-dimensional point cloud corresponding to the target scene.

[0081] It is easy to understand that by triangulating all valid left and right view jump matching point pairs, a series of three-dimensional points can be obtained, and these points constitute the three-dimensional point cloud of the target scene. The specific process of triangulation and filtering, smoothing or other post-processing operations on the three-dimensional point cloud are all contents that those skilled in the art should understand, and the implementation methods are relatively diverse. Therefore, this embodiment does not make too many restrictions on this, so that it can achieve the basic effect.

[0082] It should be noted that, in each of the above-mentioned embodiments, there is not necessarily a certain order between the above-mentioned steps. A person skilled in the art can understand, based on the description of the embodiments of the present application, that in different embodiments, the above-mentioned steps may have different execution orders, that is, they may be executed in parallel, may be executed interchangeably, and so on.

[0083] As another aspect of the embodiment of the present application, the embodiment of the present application provides a 3D reconstruction device, wherein the 3D reconstruction device can be a software module, the software module includes a number of instructions, which are stored in a memory, and the processor can access the memory and call the instructions for execution to complete the 3D reconstruction method described in each of the above embodiments.

[0084] In some embodiments, the 3D reconstruction device can also be constructed by hardware devices, for example, the 3D reconstruction device can be constructed by one or more chips, and the chips can work in coordination with each other to complete the 3D reconstruction methods described in the above embodiments. For another example, the 3D reconstruction device can also be constructed by various logic devices, such as a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISC Machine) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of these components.

[0085] See also Figure 3 The three-dimensional reconstruction device 300 of the embodiment of the present application includes a data acquisition module 31, a projection matching module 32 and a point cloud output module 33.

[0086] The data acquisition module 31 is used to acquire a projection image generated by the projection device projecting onto the target scene, wherein the projection image includes a Gray code projection image, a Gray code complement projection image and a high-frequency inverted projection image, wherein the high-frequency inverted projection image includes a plurality of groups of black and white stripes, and the Gray code image and the Gray code complement image are used to determine the period of edge jumps of the plurality of groups of black and white stripes.

[0087] The projection matching module 32 is used to perform left and right same-code jump matching according to the Gray code projection image, the Gray code complement projection image and the high-frequency inverted projection image to determine the left and right view jump matching points;

[0088] The point cloud output module 33 is used to perform triangulation based on the left and right view jump matching points to generate a three-dimensional point cloud corresponding to the target scene.

[0089] It should be noted that the above-mentioned 3D reconstruction device can execute the 3D reconstruction method provided in the embodiment of the present application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in the embodiment of the 3D reconstruction device, please refer to the 3D reconstruction method provided in the embodiment of the present application.

[0090] See also Figure 4 , Figure 4 1 is a schematic diagram of the structure of an electronic device 400 provided in an embodiment of the present application, wherein the electronic device 400 is a processing device in the above-mentioned three-dimensional reconstruction system. The electronic device 400 includes one or more processors 41 and a memory 42. The memory 42 is connected to the one or more processors 41, for example, connected to the processor 41 via a bus.

[0091] The processor 41 is configured to support the electronic device 400 to perform the corresponding functions in the method in the above method embodiment. The processor 41 can be a central processing unit (CPU), a network processor (NP), a hardware chip or any combination thereof. The above hardware chip can be an application specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0092] The memory 42 is used to store program codes, etc. The memory may include a volatile memory (VM), such as a random access memory (RAM); the memory may also include a non-volatile memory (NVM), such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); the memory 42 may also include a combination of the above-mentioned types of memory.

[0093] The memory 42 can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the three-dimensional reconstruction method in the embodiment of the present application. The processor 41 executes various functional applications and data processing of the three-dimensional reconstruction method and the three-dimensional reconstruction device by running the non-volatile software programs, instructions and modules stored in the memory 42, that is, realizes the functions of the three-dimensional reconstruction method and each module or unit of the three-dimensional reconstruction device provided in the above method embodiment.

[0094] The memory 42 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required by at least one function. The data storage area may store data created according to the use of the three-dimensional reconstruction device, etc. In some embodiments, the memory 42 may optionally include a memory remotely arranged relative to the processor 41, and these remote memories may be connected to the three-dimensional reconstruction device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0095] One or more modules are stored in the memory 42, and when executed by one or more processors 41, the three-dimensional reconstruction method in any of the above method embodiments is executed, for example, the method steps described in the above method embodiments are executed to realize the functions of the modules described in the above device embodiments.

[0096] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the method of the aforementioned embodiment.

[0097] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0098] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A three-dimensional reconstruction method, characterized in that: Applied to a three-dimensional reconstruction system, the three-dimensional reconstruction system includes a projection device, the projection device is used to project onto a target scene, the method includes: Acquire a projection image generated by projecting the projection device onto the target scene, the projection image comprising a Gray code projection image, a Gray code complement projection image and a high-frequency inverted projection image, the high-frequency inverted projection image comprising a plurality of groups of black and white stripes, the Gray code image and the Gray code complement image being used to determine the period of edge jumps of the plurality of groups of black and white stripes; According to the Gray code projection image, the Gray code complement projection image and the high-frequency inverted projection image, performing left and right same-code jump matching to determine left and right view jump matching points; Based on the left and right view jump matching points, triangulation is performed to generate a three-dimensional point cloud corresponding to the target scene.

2. The method according to claim 1, characterized in that Before acquiring the projection image generated by the projection device projecting onto the target scene, the method further includes: Determine a pattern sequence, the pattern sequence comprising a Gray code pattern sequence, a Gray code complement pattern sequence and a high frequency inversion pattern sequence, the Gray code pattern sequence and the; According to the pattern sequence, the projection device is controlled to project toward the target scene to collect the Gray code projection image, the Gray code complement projection image and the high-frequency inverted projection image.

3. The method according to claim 2, characterized in that The determining pattern sequence comprises: determining a resolution of the projection device; Determining the Gray code pattern sequence according to the resolution of the projection device and a preset number of Gray code bits, wherein the number of Gray code bits corresponds to a minimum stripe width in the Gray code projection image; The Gray code complement pattern sequence is determined according to the Gray code pattern sequence.

4. The method according to claim 2, characterized in that: The high-frequency inverted pattern sequence includes a plurality of groups of black and white stripes, and the method further includes: Determining the total number of cycles corresponding to the Gray code complement and the stripe width of the Gray code complement pattern sequence; Determine the number of groups of the black and white stripes according to the total number of preset black and white stripe edge transitions and the total number of cycles corresponding to the Gray code complement; Determining the stripe width of the high-frequency inverted pattern sequence according to the stripe width of the Gray code complement pattern sequence; The high-frequency inversion pattern sequence is determined according to the number of groups of the black and white stripes and the stripe width.

5. The method according to claim 1, characterized in that The three-dimensional reconstruction system further includes a left camera and a right camera, wherein the left camera and the right camera are used to collect a left view and a right view respectively, and the left and right same-code jump matching is performed according to the Gray code projection image, the Gray code complement projection image and the high-frequency inverted projection image to determine the left and right view jump matching points, including: Determining the sub-pixel position of the black-white edge transition in the high-frequency inverted projection image; Determining the code value corresponding to the sub-pixel position in the Gray code projection image, the Gray code complement projection image and the high-frequency inverse projection image respectively; According to the code value, determining the sub-pixel position of each black-white edge transition in the left view; By using epipolar constraints, a straight line is determined in the right view, which corresponds to the sub-pixel position of each black-white voltage transition in the left view; Based on the straight line where the corresponding points are located, the black and white edge transition sub-pixel positions with consistent black and white edge transition code values ​​are determined, and matching is performed to obtain the left and right view transition matching points.

6. The method according to claim 5, characterized in that The high-frequency inverted projection image includes a plurality of black and white stripe groups, each of the black and white stripe groups includes a target image and an inverted image, the target image is a black and white pattern of equal width, the inverted image is an inverted black and white pattern of equal width corresponding to the target image, and determining the sub-pixel position of the black and white edge jump in the high-frequency inverted projection image includes: determining the inverted image in the high-frequency inverted pattern projection image; Performing difference processing on the inverted image to obtain a difference image; Traversing each pixel of the difference image, determining the pixel point with opposite left and right differences as the position of black-white edge transition; The sub-pixel position is determined by linear fitting, and the difference of the sub-pixel position is 0.

7. The method according to claim 5, characterized in that The step of respectively determining the code value corresponding to the sub-pixel position in the Gray code projection image, the Gray code complement projection image, and the high-frequency inverse projection image includes: Binarizing the Gray code projection image and the Gray code complement projection image; According to the sub-pixel position, sequentially extracting the binary Gray code projection image and the Gray code complement projection image respectively to determine the Gray code and the Gray code complement; Determine the black and white edge transition position of the high-frequency inverted projection image, and the black and white edge transition of the minimum width stripe in the Gray code; Determine whether the black and white edge transition position of the high-frequency inverse projection image corresponds to the black and white edge transition of the minimum width stripe in the Gray code; If yes, determining that the Gray code complement is the code value of the high-frequency inverse projection image; If not, it is determined that the Gray code is the code value of the high-frequency inverse projection image.

8. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory is connected to the processor, and the processor is used to execute one or more computer programs stored in the memory, and when the processor executes the one or more computer programs, the electronic device implements the method according to any one of claims 1 to 7.

9. A three-dimensional reconstruction system, characterized in that: Set in a target scene, the system includes: A projection device, used for projecting onto a target scene; The acquisition device includes a left camera and a right camera, which are used to acquire the projection image in the target scene; The electronic device as claimed in claim 8, wherein the electronic device is connected to the projection device and the acquisition device respectively.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 7.