High dynamic range structured light three-dimensional measurement method, system and medium

Abstract

The present invention discloses a high dynamic range structured light three-dimensional measurement method, system and medium, wherein the method comprises: projecting a white picture onto a surface to be measured, collecting a first image of the surface to be measured; after histogramming the first image, calculating and obtaining an optimal exposure time; projecting a stripe structured light picture onto the surface to be measured, and obtaining an exposure image pair of the surface to be measured according to the optimal exposure time; inputting the exposure image pair into a trained HDR structured light image reconstruction model for reading, and obtaining a high dynamic range structured light image; performing phase decoding and three-dimensional reconstruction on the high dynamic range structured light image, and completing the three-dimensional measurement of the surface to be measured. The present invention inputs the exposure image pair into the HDR structured light image reconstruction model to obtain a high dynamic range structured light image, and uses a small number of LDR structured light images to reconstruct a high-quality HDR structured light image, which greatly improves the efficiency of three-dimensional measurement and can be widely used in the field of optical measurement technology.

Description

Technical Field

[0001] The present invention relates to the field of optical measurement technology, and in particular to a high dynamic range structured light three-dimensional measurement method, system and medium. Background Art

[0002] Structured light 3D measurement technology has a wide range of applications in robotics, reverse engineering, and human-computer interaction due to its ultra-high measurement accuracy. Although structured light 3D measurement technology has achieved great success in key technology research and industrialization, there are still some unresolved problems. Efficient and accurate measurement of high-contrast surfaces is one of the problems in this field, and it is also a hot research direction in this field. The reflectivity between different areas of high-contrast surfaces varies greatly. Limited by the limited dynamic range of ordinary industrial cameras, it is often difficult to obtain high-quality high-contrast surface images, which leads to low accuracy of 3D measurement or even measurement failure. Efficient and accurate measurement of high-contrast surfaces has a large demand in actual production, such as defect detection of various metal workpieces in industrial manufacturing and industrial inspection. The dynamic measurement range of structured light 3D measurement technology, as well as the measurement efficiency and measurement accuracy under high dynamic measurement range, are the main bottlenecks limiting the application of this technology in high-contrast scenes.

[0003] In order to solve the above problems, domestic and foreign technicians have proposed many solutions, such as polarizer filtering and multi-view imaging. These solutions improve the three-dimensional measurement accuracy of high-contrast surfaces, but they also bring some problems. Either the measured light intensity needs to be sacrificed, resulting in a decrease in the signal-to-noise ratio, or hardware equipment needs to be added, and the reconstruction algorithm is complex, resulting in an increase in measurement costs. In addition to the above methods, the most effective method that does not increase the cost of hardware equipment is high dynamic range (HDR) technology, which is divided into two types: 1) adjusting the intensity of the projected stripes, and 2) adjusting the exposure intensity of the camera. Although the HDR technology that adjusts the intensity of the projected stripes can obtain a better projected stripe pattern, the complex implementation algorithm leads to low measurement efficiency of this method; the HDR technology that adjusts the exposure intensity of the camera can obtain high-quality HDR images, but to reconstruct high-quality HDR images, a large number of exposures are often required, and the measurement efficiency is also low.

[0004] Terminology explanation:

[0005] HDR: High dynamic range.

[0006] LDR: Lower dynamic range.

[0007] DCNN: Deep Convolution Neural Network. Summary of the invention

[0008] In order to solve at least one of the technical problems existing in the prior art to a certain extent, an object of the present invention is to provide a high dynamic range structured light three-dimensional measurement method, system and medium.

[0009] The technical solution adopted by the present invention is:

[0010] A high dynamic range structured light three-dimensional measurement method, comprising the following steps:

[0011] Projecting a white picture onto the surface to be measured, and collecting a first image of the surface to be measured;

[0012] After performing histogram processing on the first image, an optimal exposure time is calculated;

[0013] Projecting a stripe structured light image onto the surface to be measured, and acquiring an exposure image pair of the surface to be measured according to the optimal exposure time;

[0014] The exposure image is input into a trained HDR structured light image reconstruction model to read and obtain a high dynamic range structured light image;

[0015] Performing phase decoding and three-dimensional reconstruction on the high dynamic range structured light image to complete three-dimensional measurement of the surface to be measured;

[0016] The exposure image pair includes an under-exposed image and an over-exposed image.

[0017] Furthermore, the HDR structured light image reconstruction model is a module based on a deep convolutional neural network, including a feature extraction layer, a feature fusion layer and an HDR image reconstruction layer;

[0018] The feature extraction layer is used to extract the underlying features of the exposure image pair, and obtain two groups of feature maps with the same feature type;

[0019] The feature fusion layer is used to perform feature fusion on the two groups of feature maps by adding them;

[0020] The HDR image reconstruction layer is used to use the fused feature map to complete the reconstruction of the HDR image through multiple layers of convolution and upsampling layers.

[0021] Furthermore, when training the HDR structured light image reconstruction model, a loss function based on image structure similarity is used for model training, and a back propagation algorithm is used to train the model.

[0022] Furthermore, the loss function is expressed as:

[0023]

[0024] Among them, SSIN(p) represents the similarity between the image block of the input image and the image block processed by the model; N is the total number of pixels of a single input image, and P represents the set of all pixels of the input image.

[0025] Furthermore, before training the HDR structured light image reconstruction model, a training set establishment step is also included:

[0026] Use DLP high-speed projection equipment to project white images onto the preset surface;

[0027] An image acquisition device is used to acquire images at an initial exposure time, and brightness histogram processing is performed on the acquired images;

[0028] Calculate the optimal exposure time of low-reflection area and the optimal exposure time of high-reflection area based on the brightness histogram of the image;

[0029] Using a DLP high-speed projection device to project a stripe structured light image onto the preset surface, and acquiring an exposure image pair according to the optimal exposure time;

[0030] The above steps are repeated multiple times to obtain multiple exposure image pairs as training sets.

[0031] Furthermore, the step of establishing a training set also includes the following steps:

[0032] Using data enhancement technology to expand the number of samples in the training set;

[0033] The data enhancement technology includes at least one of a rotation transformation, a reflection transformation, a flip transformation, a scaling transformation, a translation transformation or a scale transformation.

[0034] Further, the phase decoding and three-dimensional reconstruction of the high dynamic range structured light image includes:

[0035] The relative phase of the image is solved by using a three-step phase shift method, and the absolute phase is obtained by combining the relative phase and the low-frequency decoded image;

[0036] The projection coordinates are acquired according to the absolute phase, and the surface to be measured is three-dimensionally reconstructed by combining the projection coordinates and the high dynamic range structured light image.

[0037] Another technical solution adopted by the present invention is:

[0038] A high dynamic range structured light three-dimensional measurement system, comprising:

[0039] A first acquisition module, used for projecting a white picture onto a surface to be measured, and acquiring a first image of the surface to be measured;

[0040] An exposure time module, used to calculate and obtain an optimal exposure time after performing a histogram processing on the first image;

[0041] A second acquisition module is used to project a stripe structured light image onto the surface to be measured, and acquire an exposure image pair of the surface to be measured according to the optimal exposure time;

[0042] An image reconstruction module is used to read the exposure image into a trained HDR structured light image reconstruction model to obtain a high dynamic range structured light image;

[0043] A three-dimensional measurement module, used for performing phase decoding and three-dimensional reconstruction on the high dynamic range structured light image to complete three-dimensional measurement of the surface to be measured;

[0044] The exposure image pair includes an under-exposed image and an over-exposed image.

[0045] Another technical solution adopted by the present invention is:

[0046] A high dynamic range structured light three-dimensional measurement system, comprising:

[0047] at least one processor;

[0048] at least one memory for storing at least one program;

[0049] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0050] Another technical solution adopted by the present invention is:

[0051] A storage medium stores processor-executable instructions, which are used to execute the above method when executed by the processor.

[0052] The beneficial effects of the present invention are as follows: the present invention inputs an exposure image pair (an underexposed image and an overexposed image) into an HDR structured light image reconstruction model to obtain a high dynamic range structured light image, and uses a smaller number of LDR structured light images to reconstruct a high-quality HDR structured light image, thereby greatly improving the efficiency of three-dimensional measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the embodiments of the present invention or the drawings of related technical solutions in the prior art are introduced below. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0054] Figure 1 is a flowchart of the steps of a high dynamic range structured light three-dimensional measurement method in an embodiment of the present invention;

[0055] Figure 2 Schematic diagram of the architecture of the HDR structured light image reconstruction model in an embodiment of the present invention;

[0056] Figure 3 is a schematic diagram of the steps of establishing a training set in an embodiment of the present invention;

[0057] Figure 4 is a low-frequency stripe structured light decoded image in an embodiment of the present invention;

[0058] Figure 5 is a high-frequency stripe structured light decoded image in an embodiment of the present invention;

[0059] Figure 6 1 is a schematic diagram of the steps of the high dynamic range structured light three-dimensional measurement method in an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limitations of the present invention. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0061] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0062] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0063] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0064] like Figure 1 As shown, this embodiment provides a high dynamic range structured light three-dimensional measurement method, comprising the following steps:

[0065] S1, projecting a white picture onto the surface to be measured, and collecting a first image of the surface to be measured;

[0066] S2, after performing histogram processing on the first image, calculate and obtain the optimal exposure time;

[0067] S3. Projecting a stripe structured light image onto the surface to be measured, and acquiring an exposure image pair of the surface to be measured according to an optimal exposure time; wherein the exposure image pair includes an under-exposed image and an over-exposed image.

[0068] S4, reading the exposure image into the trained HDR structured light image reconstruction model to obtain a high dynamic range structured light image;

[0069] S5. Perform phase decoding and three-dimensional reconstruction on the high dynamic range structured light image to complete the three-dimensional measurement of the surface to be measured.

[0070] Among them, the HDR structured light image reconstruction model is a module based on a deep convolutional neural network, which can be trained using existing training methods, including a feature extraction layer, a feature fusion layer, and an HDR image reconstruction layer:

[0071] The feature extraction layer is used to extract the underlying features of the exposure image pair, and obtains two sets of feature maps with the same feature type;

[0072] The feature fusion layer is used to fuse the two sets of feature maps by adding them together;

[0073] The HDR image reconstruction layer is used to use the fused feature map to complete the reconstruction of the HDR image through multiple layers of convolution and upsampling layers.

[0074] This embodiment introduces deep learning into the structured light three-dimensional measurement of complex high-contrast surfaces from the perspective of digital image photography. This embodiment focuses on the cross-integration research of multiple disciplines, applies and develops new HDR technology based on deep learning, which can use a small number of LDR structured light images to reconstruct high-quality HDR structured light images, which will greatly improve the efficiency of three-dimensional measurement. Then, by solving the phase and three-dimensional reconstruction, the final three-dimensional measurement is completed, and then a new high-efficiency and high-dynamic range structured light three-dimensional measurement system is developed, which provides a new idea and method for realizing efficient and accurate measurement of complex high-contrast surfaces, and is expected to break through the technical bottleneck of the application of structured light three-dimensional measurement technology in complex high-contrast scenes.

[0075] The above technical solution is explained in detail below in conjunction with specific embodiments.

[0076] This embodiment uses the deep learning method as a research tool, and realizes the dynamic range expansion of structured light images through HDR technology based on deep learning. A pair of LDR structured light images are used as input to establish an end-to-end mapping between LDR and HDR structured light images. The mapping is represented by a deep convolutional neural network (DCNN), and the three-dimensional measurement is further completed through the principle of phase solution and three-dimensional reconstruction. Through the integrated research of the system, a research plan for a high-speed high dynamic range structured light three-dimensional adaptive measurement system is finally realized. Therefore, the technical solution includes: 1. Construction of HDR structured light image fusion model based on deep learning; 2. Establishment of deep learning training data set for high contrast surfaces; 3. Solving phase and three-dimensional reconstruction; 4. System integration of high-speed high dynamic range structured light three-dimensional adaptive measurement.

[0077] (1) Construct an HDR structured light image reconstruction model based on deep learning.

[0078] See also Figure 2 The HDR structured light image fusion network based on deep learning (i.e., the HDR structured light image reconstruction model) proposed in this embodiment generally includes three parts: feature extraction layer, feature fusion layer, and HDR image reconstruction layer. The network architecture of deep learning is as follows: Figure 2As shown in the figure, the underexposed image obtained by short exposure time and the overexposed image obtained by long exposure time (a pair of LDR structured light images) are input into the upper and lower channels of the network respectively. In the feature extraction layer, k m*m filters are first used to extract the underlying features of the image, and the two channels use the same filter for feature extraction. This design can force the network to learn the same features in the underexposed and overexposed images. The two sets of feature maps finally obtained have the same feature type, which is conducive to the subsequent fusion of the two sets of feature images. The downsampling pooling layer in the network is used for feature dimensionality reduction, compressing the number of data and parameters, reducing overfitting, and improving the fault tolerance and operation rate of the model. In the feature fusion layer, feature fusion is performed by adding two sets of feature maps. In the HDR image reconstruction layer, the fused feature map is used to complete the reconstruction of the HDR image through multiple layers of convolution and upsampling layers.

[0079] This embodiment uses a loss function based on image structure similarity for model training. This loss function does not require a true value, so it can greatly reduce the difficulty of constructing a training data set. k represents the image patch at the same pixel position p in the input image pair of the deep learning model, k = 1 corresponds to the overexposed image, k = 2 corresponds to the underexposed image, y f represents the image patch at the same pixel position p in the HDR image output by the deep learning model, and the input image y k The corresponding expected image block is expressed as follows:

[0080]

[0081] and They are the desired brightness, contrast and structure respectively. This project will conduct a detailed study on the construction methods of the three. With y f Calculate the similarity of image patches:

[0082]

[0083] and They are With y f The average brightness of the corresponding image block, and They are With y f The variance and covariance between them, C1 and C2 are constants to maintain stability. The specific values ​​will be determined by subsequent research. The final loss function based on image structure similarity can be expressed as follows:

[0084]

[0085] N is the total number of pixels of a single input image, P represents the set of all pixels of the input image, and the model is trained using the back propagation algorithm.

[0086] (2) Establish a deep learning training dataset for complex high-contrast surfaces

[0087] The technical solution for establishing a training data set is as follows: Figure 3 As shown, first, a DLP high-speed projector is used to project a white image onto the surface to be measured. The camera collects images at the initial exposure time and performs brightness histogram processing on the collected images. Secondly, a camera lighting model of the structured light three-dimensional measurement system is constructed, and the optimal exposure time of the low-reflection area and the high-reflection area is calculated in combination with the brightness histogram of the image. Again, the DLP projects stripe structured light onto the surface to be measured, and over-exposed and under-exposed LDR structured light image pairs are collected at the obtained optimal exposure time. Finally, the above technical route is repeated in a loop to capture and collect a large number of LDR image pairs of complex high-contrast surfaces. In addition, this technology will also use data enhancement technology, including rotation / reflection / flip transformation, scaling transformation, translation transformation, scale transformation, etc., to expand the number of samples in the training set and quickly create a large-scale training data set.

[0088] (3) Solve the phase and three-dimensional reconstruction.

[0089] After obtaining high-quality HDR structured light stripe images through the deep learning model, in order to ensure that the measurement system has high measurement accuracy, this technology uses stripe structured light and uses the three-step phase shift method to solve the relative phase, and then combines the low-frequency decoded image to finally obtain the absolute phase. In the three-step phase shift method, the light intensities I1, I2, and I3 captured at three different times for the same pixel point (x, y) can be expressed as:

[0090] I1(x, y) = I b (x,y)+I m (x, y)cos(φ-2π / 3),

[0091] I2(x,y)=I b (x,y)+I m (x, y)cos(φ),

[0092] I3(x,y)=I b (x,y)+I m (x, y)cos(φ+2π / 3),

[0093] I b and I m are the background light intensity and the modulated light intensity respectively, and the phase obtained by decoding is:

[0094]

[0095] In actual measurement, in order to improve the measurement accuracy, a fringe period of T is often used. h The high-frequency decoded image of Figure 5 As shown, the projector coordinates are related to T h The relationship is as follows:

[0096]

[0097] In the above formula, N is the number of high-frequency cycles before the measurement point, φ h is the relative phase. To obtain the absolute phase, a low-frequency decoded image needs to be projected, such as Figure 4 As shown. For the low-frequency decoded image, the phase φ obtained by phase shifting method is l Between 0 and 2π, the fringe period T of the projector coordinates and the low-frequency decoded image l Relationship:

[0098]

[0099] Combining the above two equations, we can get:

[0100]

[0101] Round(·) is a rounding function. Through the above calculations, the absolute phase can be obtained, thus completing the phase solution step. After obtaining the phase map, the projection coordinates can be obtained, and then the parameters of the camera and projector obtained in the calibration process and the translation and rotation matrices between the projector and the camera can be used to complete the three-dimensional reconstruction of the surface to be measured.

[0102] (4) System integration of high-efficiency and high-dynamic range structured light 3D measurement

[0103] Figure 6 A system integration solution for efficient and high dynamic range three-dimensional measurement is proposed: a white screen is projected onto the surface to be measured by a DLP high-speed projector, and the image of the surface to be measured is collected by a camera. The collected image is histogrammed and then the optimal exposure time is calculated. The optimal exposure time is fed back to the camera. The DLP high-speed projector projects a stripe structured light image onto the surface to be measured. The camera takes pictures at the optimal exposure time to obtain over-exposed and under-exposed LDR image pairs. The deep learning network model reads the LDR image pairs and infers the HDR structured light image. The entire measurement process is completed through phase decoding and three-dimensional reconstruction.

[0104] The efficiency of the integrated measurement solution of this embodiment is mainly achieved from three aspects: First, reduce the number of LDR images. This measurement system only needs a pair of LDR images to complete the reconstruction of the corresponding HDR image; second, optimize the deep learning model. This project will study methods such as downsampling and GPU parallel computing to achieve feature dimension reduction and compress the number of data and parameters to improve the speed of model operation; third, in the three-dimensional reconstruction stage, GPU parallel computing will be used to complete the calculation of a large number of three-dimensional point clouds at the same time. This embodiment will use the Jetson TX1 embedded platform developer kit provided by NVIDIA for HDR deep learning model training and three-dimensional point cloud generation. The development kit has a built-in NVIDIA Maxwell GPU with 256 CUDA cores. Its powerful computing power can further improve the measurement efficiency of the measurement system.

[0105] In summary, this embodiment introduces deep learning into the structured light three-dimensional measurement of complex high-contrast surfaces from the perspective of digital image photography, and innovatively provides a new structured light three-dimensional measurement system with high efficiency and high dynamic range. At present, the research on deep learning in structured light three-dimensional measurement has just started, mainly focusing on phase solution, noise elimination and sub-surface scattering. This embodiment focuses on the cross-disciplinary fusion research, applies and develops new HDR technology based on deep learning, uses a small number of LDR structured light images, and reconstructs high-quality HDR structured light images, which is expected to greatly improve the efficiency of three-dimensional measurement, and then completes the final three-dimensional measurement by solving the phase and three-dimensional reconstruction, and then develops a new structured light three-dimensional measurement system with high efficiency and high dynamic range, which provides a new idea and method for realizing efficient and accurate measurement of complex high-contrast surfaces, and is expected to break through the technical bottleneck of the application of structured light three-dimensional measurement technology in complex high-contrast scenes.

[0106] This embodiment also provides a high dynamic range structured light three-dimensional measurement system, including:

[0107] A first acquisition module, used for projecting a white picture onto the surface to be measured, and acquiring a first image of the surface to be measured;

[0108] An exposure time module, used to calculate and obtain an optimal exposure time after performing histogram processing on the first image;

[0109] The second acquisition module is used to project the stripe structured light image onto the surface to be measured, and acquire the exposure image pair of the surface to be measured according to the optimal exposure time;

[0110] An image reconstruction module is used to read the exposure image into a trained HDR structured light image reconstruction model to obtain a high dynamic range structured light image;

[0111] A 3D measurement module, which is used to perform phase decoding and 3D reconstruction on high dynamic range structured light images to complete 3D measurement of the surface to be measured;

[0112] The exposure image pair includes an under-exposed image and an over-exposed image.

[0113] A high dynamic range structured light 3D measurement system of this embodiment can execute a high dynamic range structured light 3D measurement method provided by the method embodiment of the present invention, can execute any combination of implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.

[0114] This embodiment also provides a high dynamic range structured light three-dimensional measurement system, including:

[0115] at least one processor;

[0116] at least one memory for storing at least one program;

[0117] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0118] A high dynamic range structured light 3D measurement system of this embodiment can execute a high dynamic range structured light 3D measurement method provided by the method embodiment of the present invention, can execute any combination of implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.

[0119] The present application also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes Figure 1 The method shown.

[0120] This embodiment also provides a storage medium storing instructions or programs that can execute a high dynamic range structured light three-dimensional measurement method provided by an embodiment of the method of the present invention. When the instructions or programs are run, any combination of implementation steps of the method embodiment can be executed, and the corresponding functions and beneficial effects of the method can be obtained.

[0121] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0122] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions, and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0123] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0124] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0125] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0126] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0127] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0128] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0129] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A high dynamic range structured light three-dimensional measurement method, characterized in that: The following steps are involved: Projecting a white picture onto the surface to be measured, and collecting a first image of the surface to be measured; After performing histogram processing on the first image, an optimal exposure time is calculated; Projecting a stripe structured light image onto the surface to be measured, and acquiring an exposure image pair of the surface to be measured according to the optimal exposure time; The exposure image is input into a trained HDR structured light image reconstruction model to read and obtain a high dynamic range structured light image; Performing phase decoding and three-dimensional reconstruction on the high dynamic range structured light image to complete three-dimensional measurement of the surface to be measured; Wherein, the exposure image pair includes an under-exposed image and an over-exposed image; The HDR structured light image reconstruction model is a module based on a deep convolutional neural network, including a feature extraction layer, a feature fusion layer and an HDR image reconstruction layer; The feature extraction layer is used to extract the underlying features of the exposure image pair, and obtain two groups of feature maps with the same feature type; The feature fusion layer is used to perform feature fusion on the two groups of feature maps by adding them; The HDR image reconstruction layer is used to use the fused feature map to complete the reconstruction of the HDR image through multiple layers of convolution and upsampling layers; The performing phase decoding and three-dimensional reconstruction on the high dynamic range structured light image comprises: The relative phase of the image is solved by a three-step phase shift method, and the absolute phase is obtained by combining the relative phase and the low-frequency decoded image; the projection coordinates are obtained according to the absolute phase, and the three-dimensional reconstruction of the surface to be measured is performed by combining the projection coordinates and the high dynamic range structured light image.

2. The high dynamic range structured light three-dimensional measurement method according to claim 1, characterized in that: When training the HDR structured light image reconstruction model, a loss function based on image structure similarity is used for model training, and a back propagation algorithm is used to train the model.

3. The high dynamic range structured light three-dimensional measurement method according to claim 2, characterized in that: The expression of the loss function is: Among them, SSIM(p) represents the similarity between the image block of the input image and the image block processed by the model; N is the total number of pixels of a single input image, and P represents the set of all pixels of the input image.

4. The high dynamic range structured light three-dimensional measurement method according to claim 1, characterized in that: Before training the HDR structured light image reconstruction model, the following steps are also included: Use DLP high-speed projection equipment to project white images onto the preset surface; An image acquisition device is used to acquire images at an initial exposure time, and brightness histogram processing is performed on the acquired images; Calculate the optimal exposure time of low-reflection area and the optimal exposure time of high-reflection area based on the brightness histogram of the image; Using a DLP high-speed projection device to project a stripe structured light image onto the preset surface, and acquiring an exposure image pair according to the optimal exposure time; The above steps are repeated multiple times to obtain multiple exposure image pairs as training sets.

5. The high dynamic range structured light three-dimensional measurement method according to claim 4, characterized in that: The step of establishing a training set also includes the following steps: Using data enhancement technology to expand the number of samples in the training set; The data enhancement technology includes at least one of a rotation transformation, a reflection transformation, a flip transformation, a scaling transformation, a translation transformation or a scale transformation.

6. A high dynamic range structured light 3D measurement system, used to implement a high dynamic range structured light 3D measurement method according to any one of claims 1 to 5, characterized in that: include: A first acquisition module, used for projecting a white picture onto a surface to be measured, and acquiring a first image of the surface to be measured; An exposure time module, used to calculate and obtain an optimal exposure time after performing a histogram processing on the first image; A second acquisition module is used to project a stripe structured light image onto the surface to be measured, and acquire an exposure image pair of the surface to be measured according to the optimal exposure time; An image reconstruction module is used to read the exposure image into a trained HDR structured light image reconstruction model, Obtain high dynamic range structured light images; A three-dimensional measurement module, used for performing phase decoding and three-dimensional reconstruction on the high dynamic range structured light image to complete three-dimensional measurement of the surface to be measured; The exposure image pair includes an under-exposed image and an over-exposed image.

7. A high dynamic range structured light three-dimensional measurement system, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the high dynamic range structured light three-dimensional measurement method according to any one of claims 1 to 5.

8. A storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to execute the high dynamic range structured light three-dimensional measurement method as claimed in any one of claims 1 to 5 when executed by the processor.

Images