A multi-frame structured light reconstruction module

Abstract

A multi-frame structured light reconstruction module comprises the following modules: a first laser projector for projecting laser light to a target; an infrared camera for collecting a depth image and a background image of the target; a processor module for obtaining a preset frame frequency value, continuously and sequentially collecting a first depth image, a background image and a second depth image of the same target according to the frame frequency value, generating a first effective image according to the difference between the gray values of corresponding pixels of the first depth image and the background image; generating a second effective image according to the difference between the gray values of corresponding pixels of the second depth image and the background image; and generating a depth image according to the depth reconstruction or three-dimensional reconstruction of the first depth image and the second depth image. The application can improve the multi-frame depth reconstruction result of the measured target under the condition that the background light is strong, shorten the time interval between the collection of adjacent two frames of images, and increase the detection distance.

Description

Technical Field

[0001] This invention relates to structured light 3D reconstruction, and more specifically, to a multi-frame structured light remodeling group. Background Technology

[0002] Mobile payment has become the mainstream payment method in China and is playing an increasingly important role in more and more fields. As the core component of facial recognition payment terminals, the facial recognition camera module plays a crucial role. Currently, the more mature facial recognition camera modules adopt a structured light solution.

[0003] As a core component of facial recognition payment terminals, the facial recognition camera module plays a crucial role. Currently, the more mature facial recognition camera modules employ a structured light solution.

[0004] The structured light three-mode method is based on the principle of optical triangulation. An optical projector projects structured light of a specific pattern onto the surface of an object, forming a three-dimensional image of light stripes modulated by the shape of the object's surface. This three-dimensional image is detected by a camera at another location, thus obtaining a two-dimensional distorted image of the light stripes. The degree of distortion of the light stripes depends on the relative position between the optical projector and the camera, and the shape (height) of the object's surface. Intuitively, the displacement (or offset) along the light stripes is proportional to the height of the object's surface; twisting indicates changes in the plane, and discontinuity shows physical gaps on the surface. When the relative position between the optical projector and the camera is constant, the three-dimensional shape of the object's surface can be reconstructed from the coordinates of the distorted two-dimensional light stripe image.

[0005] Depth camera modules broaden the dimensions of front-end perception, effectively addressing the challenges of resisting spoofing attacks and reducing accuracy in extreme situations encountered in 2D face recognition. Their effectiveness has been recognized by the market, with strong demand, and they can be applied to scenarios such as door locks, access control, and payment systems based on 3D face recognition. For face recognition and similar applications, not only depth images of the target are needed, but also grayscale images. Generally, grayscale images are of two types: those captured by the same camera, those captured by different cameras, or a combination of both. In depth reconstruction applications, the acquired texture image includes the texture pattern projected by the projector and the background light. The texture pattern projected by the projector represents the effective signal, while the background light represents noise interference. In some situations, such as when the lighting intensity is strong, the background light is also strong, resulting in significant interference and a low signal-to-noise ratio. Summary of the Invention

[0006] To this end, the present invention controls the order and frequency of image acquisition and processes the images accordingly, which can improve the multi-frame depth reconstruction results of the target under strong background light, shorten the acquisition time interval between two adjacent frames, increase the detection distance, increase the difficulty of attacking the depth camera, and make the depth camera more secure and the data clearer.

[0007] This invention provides a multi-frame structured light remodeling group, characterized by comprising the following modules:

[0008] The first laser projector is used to project laser light onto the target;

[0009] An infrared camera is used to acquire depth and background images of the target;

[0010] The processor module is configured to acquire a preset frame rate value, sequentially acquire a first depth image, a background image, and a second depth image of the same target according to the frame rate value, generate a first valid image by subtracting the gray values ​​of corresponding pixels in the first depth image and the background image, generate a second valid image by subtracting the gray values ​​of corresponding pixels in the second depth image and the background image, and generate a depth image by performing depth reconstruction or three-dimensional reconstruction based on the first depth image and the second depth image.

[0011] Optionally, the multi-frame structured light remodeling group is characterized by further comprising:

[0012] A second laser projector is used to project a laser onto a target; wherein the type of laser projected by the second laser projector is different from the type of laser projected by the first laser projector.

[0013] Optionally, the multi-frame structured light remodeling group is characterized in that the first laser projector and the second laser projector have the same emission frequency.

[0014] Optionally, the multi-frame structured light remodeling group is characterized in that, when acquiring images based on the frame rate value, it includes the following steps:

[0015] Step S1: Obtain the preset frame rate threshold and the original frame rate value, wherein the original frame rate value is the frame rate value of the depth camera used to acquire the first depth image and the second depth image.

[0016] Step S2: Determine the multiple between the original frame rate value and the frame rate threshold. When the multiple is less than or equal to a preset multiple threshold, the preset frame rate value is determined to be the product of the multiple threshold and the frame rate threshold. When the multiple is greater than or equal to the preset multiple threshold, the preset frame rate value is determined to be the original frame rate value.

[0017] Step S3: Based on the frame rate value, continuously and sequentially acquire the first depth image, background image, and second depth image of the same target within a frame rate threshold sampling period.

[0018] Optionally, the multi-frame structured light remodeling group is characterized in that step S3 includes the following steps:

[0019] Step S1031: Project a laser beam onto the target through the light projector of the first laser projector;

[0020] Step S1032: When the multiplier value is less than or equal to the preset multiplier threshold, the first depth image, background image, and second depth image of the same target are continuously and sequentially acquired within the frame frequency threshold sampling period according to the frame frequency value;

[0021] Step S1033: When the multiplier value is greater than the preset multiplier threshold, the first depth image, background image and second depth image of the same target are sequentially acquired in any three consecutive frames in each of the multiple sampling periods determined according to the frame frequency threshold.

[0022] Optionally, the multi-frame structured light remodeling group is characterized in that the generation of the first valid image and the generation of the second valid image include the following steps:

[0023] Step M1: Determine the pixel value of each pixel in the background image, the first depth image, and the second depth image;

[0024] Step M2: Align the background image with the first depth image and the second depth image at the pixel level;

[0025] Step M3: Subtract the grayscale value of the corresponding pixel in the background image from each pixel in the first depth image to generate a first valid image; subtract the grayscale value of the corresponding pixel in the background image from each pixel in the second depth image to generate a second valid image.

[0026] Optionally, the multi-frame structured light remodeling group is characterized in that the generation of the depth image includes the following steps:

[0027] Step N1: Calculate the disparity images of the first depth image and the second depth image with known calibration information to obtain the disparity images of the first depth image and the second depth image;

[0028] Step N2: Determine the distance between the optical center of the infrared camera and each disparity value in the disparity map based on the principle of triangulation, and generate depth information for each pixel;

[0029] Step N3: Perform depth reconstruction or 3D reconstruction based on the depth information of each pixel to generate a depth image.

[0030] Optionally, the multi-frame structured light remodeling group is characterized in that the preset frame rate threshold is 15 FPS and the preset multiplier threshold is 3.

[0031] Optionally, the multi-frame structured light remodeling group is characterized in that the first laser projector is a structured light projector, including a light source, a light source driver, and a light modulator.

[0032] The light source driver is connected to the light source and is used to drive the light source to emit light;

[0033] The light modulator is used to modulate the light projected by the light source into discrete dot matrix light and then project it onto the target.

[0034] Optionally, the multi-frame structured light remodeling group is characterized in that the infrared camera includes an optical imaging lens and a photodetector array; the photodetector array includes multiple photodetectors arranged in an array.

[0035] The optical imaging lens is used to ensure that the direction vector of the dot matrix light entering the photodetector array through the optical imaging lens corresponds one-to-one with the photodetector.

[0036] The photodetector is used to receive dot matrix light reflected by the target object.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] In this invention, the first depth image, background image, and second depth image of the same target are continuously and sequentially acquired according to a preset frame rate value, thereby achieving continuous acquisition of three frames of images. This shortens the acquisition time interval between two adjacent frames, increases the difficulty of attacking the depth camera, and makes the depth camera more secure.

[0039] In this invention, a first effective image and a second effective image are generated by subtracting the gray values ​​of corresponding pixels in the first depth image and the background image, and the second depth image and the background image. Then, a depth image is generated by multi-frame depth reconstruction or three-dimensional reconstruction based on the first effective image and the second effective image, which reduces the interference of background light and makes the depth camera suitable for environments with strong light intensity. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort. Other features, objects, and advantages of the present invention will become more apparent by reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0041] Figure 1 This is a schematic diagram illustrating the working principle of a multi-frame structured light remodeling group in an embodiment of the present invention;

[0042] Figure 2 This is a schematic diagram of a transmission and reception method according to an embodiment of the present invention;

[0043] Figure 3 This is a schematic diagram illustrating the working principle of a multi-frame structured light remodeling group in an embodiment of the present invention;

[0044] Figure 4 This is a flowchart illustrating the steps for determining the frame rate value in an embodiment of the present invention;

[0045] Figure 5 This is a flowchart illustrating the steps of acquiring images based on frame rate values ​​in an embodiment of the present invention.

[0046] Figure 6 This is a flowchart illustrating the steps for generating a target structured light image in an embodiment of the present invention;

[0047] Figure 7 This is a flowchart illustrating the steps of depth reconstruction to generate a depth image in an embodiment of the present invention.

[0048] Figure 8 This is a schematic diagram of a structured light projector module in an embodiment of the present invention; and

[0049] Figure 9 This is a schematic diagram of the infrared camera module in an embodiment of the present invention. Detailed Implementation

[0050] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0051] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0052] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0053] This invention provides a multi-frame structured light remodeling group, which aims to solve the problems existing in the prior art.

[0054] The technical solutions of the present invention and how they solve the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0055] Figure 1 This is a schematic diagram illustrating the working principle of a multi-frame structured light remodeling group in an embodiment of the present invention, as shown below. Figure 1 As shown, the present invention provides a multi-frame structured light remodeling group, which includes the following modules:

[0056] The first laser projector is used to project laser light onto the target.

[0057] An infrared camera is used to acquire depth and background images of the target.

[0058] The processor module is configured to acquire a preset frame rate value, sequentially acquire a first depth image, a background image, and a second depth image of the same target according to the frame rate value, generate a first valid image by subtracting the gray values ​​of corresponding pixels in the first depth image and the background image, generate a second valid image by subtracting the gray values ​​of corresponding pixels in the second depth image and the background image, and generate a depth image by performing depth reconstruction or three-dimensional reconstruction based on the first depth image and the second depth image.

[0059] Specifically, the first and second depth images are depth images obtained by illuminating the target object with an active laser. The background image is a depth image obtained without active laser illumination, commonly an infrared image obtained by an infrared sensor. Since there is only one type of laser projector in this embodiment, both the first and second effective images are the same type of depth image, namely a structured light image or an infrared image.

[0060] In some embodiments, the processor module synthesizes the first and second valid images to obtain a third valid image, and performs multi-frame depth reconstruction or 3D reconstruction based on the third valid image to generate a depth image. Synthesizing the first and second valid images ensures that only one valid image, the third valid image, exists for each signal set. Compared to the first and second valid images, the third valid image has a better signal-to-noise ratio and better recognition in environments with high ambient light intensity. The third valid image is crucial for the reconstructed depth image; therefore, its quality needs to be considered, and different module current values, exposure times, and gain values ​​should be selected to ensure that the pixel values ​​of the obtained third valid image are within an unexposed range.

[0061] Figure 2 This is a schematic diagram of a transmission and reception method according to an embodiment of the present invention. Figure 2 As can be seen, f1 and f4 are the first depth images, f2 is the background image, and f3 and f5 are the second depth images. Furthermore, f1, f2, and f3 form one set of signals, while f4, f5, and f6 form another set. This method allows for a higher signal acquisition frequency, increasing the frame rate by sharing the background signal. It should be noted that f1 and f3 are both transmissions of a set of signals, not just a single signal, but possibly a combination of two or more signals. That is, f1 and f3 are each composed of two or more signals, thus enabling this embodiment to have more application forms and function in a wider range of applications.

[0062] Figure 3 This is a schematic diagram illustrating the working principle of another multi-frame structured light remodeling group in an embodiment of the present invention, such as... Figure 3 As shown, the multi-frame structured light remodeling group provided by the present invention is characterized by further comprising:

[0063] A second laser projector is used to project a laser onto a target; wherein the type of laser projected by the second laser projector is different from the type of laser projected by the first laser projector.

[0064] Specifically, the first laser projector and the second laser projector have the same emission frequency. In this embodiment, the first depth image and the second depth image can be either the same image type or different image types. When the first depth image and the second depth image are the same image type, both the first depth image and the second depth image are composed of at least two depth images, including both structured light images and infrared images. Figure 2 To clarify, f1 and f3 both contain structured light images and infrared images, and share the background image f2. When the first depth image and the second depth image are different image types, the first depth image and the second depth image are respectively a structured light image and an infrared image, or an infrared image and a structured light image. Figure 2 For illustration, f1 and f3 are respectively the structured light image and the infrared image, or the infrared image and the structured light image, and they share the background image f2. In some embodiments, the first depth image and the second depth image are synthesized to obtain a third effective image, and multi-frame depth reconstruction or 3D reconstruction is performed based on the third effective image to generate a depth image. This allows the camera to have a wider range of applications, enabling full-range detection without switching projectors. At the same time, it also greatly enhances the camera's anti-interference performance, providing a longer detection distance and enabling effective reconstruction even in environments with high external noise.

[0065] Further explanation is provided using a structured light projector and a flood illuminator as examples. When using the depth camera provided in this embodiment, an infrared structured light image can be acquired first by projecting structured light onto the target through the structured light projector, then a background image can be acquired through an infrared camera, and finally an infrared image can be acquired by projecting flood light onto the target through the flood illuminator; alternatively, an infrared image can be acquired first by projecting flood light onto the target through the flood illuminator, then a background image can be acquired through an infrared camera, and finally an infrared structured light image can be acquired by projecting structured light onto the target through the structured light projector. The processor processes and reconstructs the obtained images. The infrared camera is a 940nm infrared camera. The flood illuminator uses an LED light source.

[0066] Figure 4 This is a flowchart illustrating the steps for determining the frame rate value in an embodiment of the present invention, as follows: Figure 4 As shown, the present invention provides a multi-frame structured light remodeling group, characterized in that, when acquiring images based on the frame rate value, the following steps are included:

[0067] Step S1: Obtain the preset frame rate threshold and the original frame rate value, wherein the original frame rate value is the frame rate value of the depth camera used to acquire the first depth image and the second depth image.

[0068] Step S2: Determine the multiple between the original frame rate value and the frame rate threshold. When the multiple is less than or equal to a preset multiple threshold, the preset frame rate value is determined to be the product of the multiple threshold and the frame rate threshold. When the multiple is greater than or equal to the preset multiple threshold, the preset frame rate value is determined to be the original frame rate value.

[0069] Step S3: Based on the frame rate value, continuously and sequentially acquire the first depth image, background image, and second depth image of the same target within a frame rate threshold sampling period.

[0070] Specifically, the preset frame rate threshold is 15 FPS, and the preset multiplier threshold is 3. When the original frame rate value is 30 FPS, the multiplier value is 2. Since the multiplier value 2 is less than the preset multiplier threshold 3, the preset frame rate value is determined to be the preset multiplier threshold 3 multiplied by the preset frame rate threshold 15, resulting in a frame rate value of 45. When the original frame rate value is 60 FPS, the multiplier value is 4. Since the multiplier value 4 is greater than the preset multiplier threshold 3, the preset frame rate value is determined to be the original frame rate value of 60 FPS.

[0071] Figure 5 This is a flowchart illustrating the steps of acquiring images based on frame rate values ​​in an embodiment of the present invention, as follows: Figure 5 As shown, the present invention provides a multi-frame structured light remodeling group, characterized in that step S3 includes the following steps:

[0072] Step S1031: Project a laser beam onto the target through the light projector of the first laser projector;

[0073] Step S1032: When the multiplier value is less than or equal to the preset multiplier threshold, the first depth image, background image, and second depth image of the same target are continuously and sequentially acquired within the frame frequency threshold sampling period according to the frame frequency value;

[0074] Step S1033: When the multiplier value is greater than the preset multiplier threshold, the first depth image, background image and second depth image of the same target are sequentially acquired in any three consecutive frames in each of the multiple sampling periods determined according to the frame frequency threshold.

[0075] Specifically, when the multiplier value is 2, and the multiplier value is less than or equal to the preset multiplier threshold, the background image, infrared structured light image, and infrared image of the same target are continuously and sequentially acquired within 15 sampling periods of the frame frequency threshold according to the frame frequency value 45, or the infrared image, infrared structured light image, and background image of the same target are continuously and sequentially acquired.

[0076] Similarly, for example, when the multiplier value is 4, and the multiplier value is greater than the preset multiplier threshold, then the background image, infrared structured light image, and infrared image of the same target are continuously and sequentially acquired every 4 frames within 15 sampling periods of the frame frequency threshold, or the infrared image, infrared structured light image, and background image of the same target are continuously and sequentially acquired.

[0077] Figure 6 This is a flowchart illustrating the steps for generating a target structured light image in an embodiment of the present invention, as follows: Figure 6 As shown, the present invention provides a multi-frame structured light remodeling group, characterized in that the generation of the first valid image and the generation of the second valid image include the following steps:

[0078] Step M1: Determine the pixel value of each pixel in the background image, the first depth image, and the second depth image;

[0079] Step M2: Align the background image with the first depth image and the second depth image at the pixel level;

[0080] Step M3: Subtract the grayscale value of the corresponding pixel in the background image from each pixel in the first depth image to generate a first valid image; subtract the grayscale value of the corresponding pixel in the background image from each pixel in the second depth image to generate a second valid image.

[0081] Figure 7 This is a flowchart illustrating the steps of depth reconstruction to generate a depth image in an embodiment of the present invention, as follows: Figure 7 As shown, the present invention provides a multi-frame structured light remodeling group, characterized in that the generation of depth images includes the following steps:

[0082] Step N1: Calculate the disparity images of the first depth image and the second depth image with known calibration information to obtain the disparity images of the first depth image and the second depth image;

[0083] Step N2: Determine the distance between the optical center of the infrared camera and each disparity value in the disparity map based on the principle of triangulation, and generate depth information for each pixel;

[0084] Step N3: Perform depth reconstruction or 3D reconstruction based on the depth information of each pixel to generate a depth image.

[0085] The preset frame rate threshold is 15 FPS, and the preset multiplier threshold is 3.

[0086] Figure 8 This is a schematic diagram of a structured light projector module in an embodiment of the present invention, such as... Figure 8As shown, the present invention provides a multi-frame structured light remodeling group, characterized in that the first laser projector is a structured light projector, including a light source, a light source driver, and a light modulator.

[0087] The light source driver is connected to the light source and is used to drive the light source to emit light;

[0088] The light modulator is used to modulate the light projected by the light source into discrete dot matrix light and then project it onto the target.

[0089] Specifically, the optical modulator employs a diffraction grating (DOE) or a spatial light modulator (SLM).

[0090] Figure 9 This is a schematic diagram of the infrared camera module in an embodiment of the present invention, as shown below. Figure 9 As shown, the present invention provides a multi-frame structured light remodeling group, characterized in that the infrared camera includes an optical imaging lens 1 and a photodetector array 3; the photodetector array includes multiple photodetectors arranged in an array.

[0091] The optical imaging lens is used to ensure that the direction vector of the dot matrix light entering the photodetector array through the optical imaging lens corresponds one-to-one with the photodetector.

[0092] The photodetector is used to receive dot matrix light reflected by the target object.

[0093] Specifically, to filter background noise, the optical imaging lens typically includes a narrowband filter 2, ensuring that the photodetector array can only pass through an incident collimated beam of a preset wavelength. This preset wavelength can be the wavelength of the incident collimated beam, or it can be between 50 nanometers less and 50 nanometers greater than the incident collimated beam. The photodetector array can be arranged periodically or non-periodically. Depending on the required number of discrete collimated beams, the photodetector array can be a combination of multiple single-point photodetectors or a sensor chip integrating multiple photodetectors. To further optimize the sensitivity of the photodetectors, the illumination spot of a discrete collimated beam on the target can correspond to one or more photodetectors. When multiple photodetectors correspond to the same illumination spot, the signals from each detector can be connected via circuitry, thereby merging into a single photodetector with a larger detection area.

[0094] Specifically, the photodetector can be a CMOS photodetector, a CCD photodetector, or a SPAD photodetector. The detector end is an infrared detector, which receives the dot matrix light reflected by the target.

[0095] In this embodiment of the invention, the background image, infrared structured light image, and infrared image of the same target are continuously and sequentially acquired according to a preset frame rate value, or the infrared image, infrared structured light image, and background image of the same target are continuously acquired, thereby achieving continuous acquisition of three frames of images. This shortens the acquisition time interval between two adjacent frames, increases the difficulty of attacking the depth camera, and makes the depth camera more secure. In this embodiment of the invention, the target structured light image is generated by subtracting the gray values ​​of corresponding pixels in the background image and the infrared structured light image. Then, depth reconstruction or 3D reconstruction is performed based on the target structured light image to generate a depth image, reducing the interference of background light and enabling the depth camera to be used in environments with strong light intensity.

[0096] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The above description of the disclosed embodiments enables those skilled in the art to implement or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein. Specific embodiments of the invention have been described above. It should be understood that the invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the invention.

Claims

1. A multi-frame structured light remodeling group, characterized in that, Includes the following modules: The first laser projector is used to project laser light onto the target; An infrared camera is used to acquire depth and background images of the target; The processor module is configured to acquire a preset frame rate value, sequentially acquire a first depth image, a background image, and a second depth image of the same target according to the preset frame rate value, generate a first valid image by subtracting the gray values ​​of corresponding pixels in the first depth image and the background image, generate a second valid image by subtracting the gray values ​​of corresponding pixels in the second depth image and the background image, and generate a depth image by performing depth reconstruction or three-dimensional reconstruction based on the first valid image and the second valid image. When acquiring images based on the preset frame rate value, the following steps are included: Step S1: Obtain the preset frame rate threshold and the original frame rate value, wherein the original frame rate value is the frame rate value of the depth camera used to acquire the first depth image and the second depth image; Step S2: Determine the multiple between the original frame rate value and the frame rate threshold. When the multiple is less than or equal to a preset multiple threshold, the preset frame rate value is determined to be the product of the multiple threshold and the frame rate threshold. When the multiple is greater than or equal to the preset multiple threshold, the preset frame rate value is determined to be the original frame rate value. Step S3: According to the preset frame rate value, continuously and sequentially acquire the first depth image, background image, and second depth image of the same target within a frame rate threshold sampling period; Step S3 includes the following steps: Step S1031: Project a laser beam onto the target through the light projector of the first laser projector; Step S1032: When the multiplier value is less than or equal to the preset multiplier threshold, the first depth image, background image, and second depth image of the same target are continuously and sequentially acquired within the frame frequency threshold sampling period according to the preset frame frequency value. Step S1033: When the multiplier value is greater than the preset multiplier threshold, the first depth image, background image and second depth image of the same target are sequentially acquired in any three consecutive frames in each of the multiple sampling periods determined according to the preset frame rate value.

2. The multi-frame structured light remodeling group according to claim 1, characterized in that, Also includes: A second laser projector is used to project a laser onto a target; wherein the type of laser projected by the second laser projector is different from the type of laser projected by the first laser projector.

3. The multi-frame structured light remodeling group according to claim 2, characterized in that, The first laser projector and the second laser projector have the same emission frequency.

4. The multi-frame structured light remodeling group according to claim 1, characterized in that, The steps of generating the first valid image and generating the second valid image include the following: Step M1: Determine the pixel value of each pixel in the background image, the first depth image, and the second depth image; Step M2: Align the background image with the first depth image and the second depth image at the pixel level; Step M3: Subtract the grayscale value of the corresponding pixel in the background image from each pixel in the first depth image to generate a first valid image; subtract the grayscale value of the corresponding pixel in the background image from each pixel in the second depth image to generate a second valid image.

5. A multi-frame structured light remodeling group according to claim 1, characterized in that, The process of generating a depth image includes the following steps: Step N1: Calculate the disparity image of the first depth image and the second depth image with the known calibration information to obtain the disparity image of the first depth image and the second depth image; Step N2: Determine the distance between the optical center of the infrared camera and each disparity value in the disparity map based on the principle of triangulation, and generate depth information for each pixel; Step N3: Perform depth reconstruction or 3D reconstruction based on the depth information of each pixel to generate a depth image.

6. The multi-frame structured light remodeling group according to claim 1, characterized in that, The preset frame rate threshold is 15 FPS, and the preset multiplier threshold is 3.

7. A multi-frame structured light remodeling group according to claim 1, characterized in that, The first laser projector is a structured light projector, including a light source, a light source driver, and a light modulator; The light source driver is connected to the light source and is used to drive the light source to emit light; The light modulator is used to modulate the light projected by the light source into discrete dot matrix light and then project it onto the target.

8. A multi-frame structured light remodeling group according to claim 7, characterized in that, The infrared camera includes an optical imaging lens and a photodetector array; the photodetector array includes multiple photodetectors arranged in an array. The optical imaging lens is used to ensure that the direction vector of the dot matrix light entering the photodetector array through the optical imaging lens corresponds one-to-one with the photodetector. The photodetector is used to receive dot matrix light reflected by the target object.

Images