A method and system for generating depth images

Abstract

The present invention discloses a method and system for generating a depth image. The method comprises: emitting speckle structured light at a target to obtain a target speckle pattern; performing denoising processing on the target speckle pattern based on a binarization algorithm to obtain a denoised image; performing a matching operation on the denoised image with a reference image to obtain a disparity image; and generating a depth image of the target based on the disparity image. The present invention utilizes a secondary binarization design approach during the denoising process of the target speckle pattern, which can filter out bad pixels and noise in the target speckle pattern and reduce noise generated during the disparity image generation process, thereby reducing noise generated during the depth image generation process and making the distance information about the target in the depth image more accurate.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a depth image generation method and system. Background Art

[0002] With the development of biometric technology, the demand for biometric security is growing stronger, and research on 3D biometrics is becoming increasingly in-depth. Currently, the core of 3D facial recognition technology lies in depth image generation, a three-dimensional image that contains distance and depth information. Mainstream depth image generation technologies include speckle structured light, binocular infrared, and Time of Flight (TOF). Compared with the other two technologies, speckle structured light, due to its technical characteristics, offers higher accuracy and stronger security. It is now widely used in scenarios such as smart door locks, smart access control, and facial payment in homes, hotels, and offices.

[0003] The depth image calculation algorithm based on speckle structured light works as follows: First, a projector emits speckle structured light toward the target, and an infrared camera captures the target speckle pattern. Then, using a pre-calibrated reference image, a window matching operation is used to calculate the pixel offset of each pixel in the target speckle pattern relative to the reference image, generating a disparity image. Finally, a depth image with distance information is generated based on the disparity image and the parameters of the camera and projector. Because the pixel offset is derived from a window matching operation, it is subject to the threshold conditions of the matching operation, which inevitably introduces computational errors. This error is reflected in the final depth calculation as inaccurate depth and distance information, or so-called depth image noise. Furthermore, the target speckle pattern is captured in real time, and inherent image noise, due to factors such as the diverse shooting environment, the target's infrared light reflection efficiency, and the infrared camera's exposure control, can affect the final depth calculation.

[0004] Currently, a common denoising method involves performing image processing (such as median filtering and Gaussian filtering) on ​​the acquired target speckle pattern to remove isolated pixels. These denoising methods are 2D noise reduction algorithms, and while effective, they are not particularly effective for 3D images such as depth calculations. Therefore, effectively removing noise during depth image generation has become a hot topic of research. Summary of the Invention

[0005] The objective of the present invention is to provide a method and system for generating a depth image. Based on a binarization algorithm, a denoising process is performed on a captured target speckle pattern to obtain a denoised image. A matching operation is performed between the denoised image and a reference image to obtain a disparity image, thereby generating a depth image of the target. At the same time, the noise generated during the depth image generation process is effectively reduced.

[0006] In order to achieve the above object, the present invention is implemented through the following technical solutions:

[0007] A depth image generation method, comprising:

[0008] Emitting speckle structured light toward the target to obtain a target speckle pattern;

[0009] Based on a binarization algorithm, denoising the target speckle pattern to obtain a denoised image; matching the denoised image with a reference image to obtain a disparity image; and

[0010] A depth image of the target is generated according to the disparity image.

[0011] Preferably, the step of performing denoising processing on the target speckle pattern based on a binarization algorithm includes:

[0012] performing binarization processing on the target speckle pattern to obtain a binarized image;

[0013] performing restoration processing on the binarized image to obtain a restored speckle pattern; and

[0014] The restored speckle image is binarized to obtain the denoised image.

[0015] Preferably, the step of performing binarization processing on the target speckle pattern includes:

[0016] According to a preset binary window, the mean grayscale value of all pixels in the corresponding preset binary window centered at each pixel in the target speckle pattern is calculated; and the pixel point as the center in the target speckle pattern is the target center pixel point; and

[0017] According to the grayscale value of the target center pixel, the average grayscale value of all pixels in the preset binarization window corresponding to the target center pixel and the first preset threshold, the target center pixel is assigned 0 and 1 to obtain the binarized image.

[0018] Preferably, the step of restoring the binary image includes:

[0019] Selecting pixels with a grayscale value of 1 in the binary image as pixels to be restored; and

[0020] The grayscale value of the pixel to be restored is restored from 1 to 255 to obtain the restored speckle image.

[0021] Preferably, the step of performing binarization processing on the restored speckle pattern comprises:

[0022] According to the preset binarization window, the mean grayscale value of all pixels in the corresponding preset binarization window centered at each pixel in the restored speckle image is calculated; and the pixel as the center in the restored speckle image is the restored center pixel; and

[0023] According to the grayscale value of the restored central pixel, the average grayscale value of all pixels in the preset binarization window corresponding to the restored central pixel, and a second preset threshold, the restored central pixel is assigned 0 and 1 to obtain the denoised image.

[0024] Preferably, the step of generating a depth image according to the disparity image comprises:

[0025] generating an initial depth image according to the disparity image and the distance between the emission point of the speckle structured light and the acquisition point of the target speckle pattern; and

[0026] A filtering algorithm is used to perform denoising on the initial depth image to obtain a final depth image.

[0027] Based on the same inventive concept, the present invention also provides a depth image generation system, comprising:

[0028] A transmitting module, used for transmitting speckle structured light toward a target;

[0029] An acquisition module, used to obtain a target speckle pattern;

[0030] a denoising module, connected to the acquisition module, for performing denoising processing on the target speckle pattern based on a binarization algorithm to obtain a denoised image;

[0031] a matching module, connected to the denoising module, for performing a matching operation on the denoised image and a reference image to obtain a disparity image; and

[0032] An image generation module is connected to the matching module and is used to generate a depth image of the target according to the disparity image.

[0033] Preferably, the denoising module includes:

[0034] a first binarization unit, connected to the acquisition module, and configured to perform binarization processing on the target speckle pattern to obtain a binarized image;

[0035] a restoration unit connected to the first binarization unit, configured to perform restoration processing on the binarized image to obtain a restored speckle pattern; and

[0036] The second binarization unit is connected to the restoration unit and the matching module respectively, and is used to perform binarization processing on the restored speckle image to obtain the denoised image.

[0037] Preferably, the first binarization unit is specifically configured to calculate, according to a preset binarization window, the mean grayscale value of all pixels in a corresponding preset binarization window centered at each pixel in the target speckle image; and the pixel point serving as the center in the target speckle image is the target center pixel point; and

[0038] According to the grayscale value of the target center pixel, the average grayscale value of all pixels in the preset binarization window corresponding to the target center pixel and the first preset threshold, the target center pixel is assigned 0 and 1 to obtain the binarized image.

[0039] Preferably, the restoration unit is specifically configured to select pixels with a grayscale value of 1 in the binary image as pixels to be restored; and restore the grayscale value of the pixels to be restored from 1 to 255 to obtain the restored speckle image.

[0040] Preferably, the second binarization unit is specifically configured to calculate, according to the preset binarization window, the mean grayscale value of all pixels in the corresponding preset binarization window centered at each pixel in the restored speckle image; and the pixel in the restored speckle image that is the center is the restored center pixel; and

[0041] According to the grayscale value of the restored central pixel, the average grayscale value of all pixels in the preset binarization window corresponding to the restored central pixel, and a second preset threshold, the restored central pixel is assigned 0 and 1 to obtain the denoised image.

[0042] Preferably, the image generation module is specifically configured to generate an initial depth image according to the disparity image and the distance between the transmitting module and the acquiring module; and

[0043] A filtering algorithm is used to perform denoising on the initial depth image to obtain a final depth image.

[0044] Compared with the prior art, the present invention has at least one of the following advantages:

[0045] The present invention provides a depth image generation method and system, which can perform denoising on the collected target speckle pattern based on a binarization algorithm and obtain a denoised image; matching the denoised image with a reference image can obtain a disparity image, thereby generating a depth image of the target, and can effectively reduce the noise generated during the depth image generation process.

[0046] In the present invention, a secondary binarization design idea is adopted in the process of denoising the target speckle pattern. The first binarization is to binarize the target speckle pattern and obtain a binarized image. At this time, bad points and noise points in the target speckle pattern can be filtered out, thereby achieving the first denoising of the target speckle pattern. Since there are still relatively many noise points in the binarized image obtained after the first binarization, the binarized image is restored to a restored speckle pattern through restoration processing, and the restored speckle pattern is subjected to a second binarization processing to obtain a denoised image. At this time, the noise points in the restored speckle pattern can be effectively removed, and as many effective points in the restored speckle pattern as possible can be retained, thereby reducing the noise points generated in the subsequent disparity image generation process.

[0047] In the present invention, the grayscale values ​​of pixels in the target speckle image have a total of 256 values ​​ranging from 0 to 255, while the grayscale values ​​of pixels in the restored speckle image have only two values, 0 and 255. Therefore, the data composition of the restored speckle image is relatively simple and evenly distributed, and the situation in which the brightness (i.e., grayscale value) distribution of pixels in a corresponding preset binarization window centered on a certain pixel in the target speckle image is not significantly different will not occur. This avoids the phenomenon of introducing new noise points while removing noise points in the restored speckle image, thereby reducing the noise points generated in the depth image generation process and achieving a better denoising effect.

[0048] The present invention performs secondary binarization processing on the target speckle pattern to obtain a denoised image, which can be matched with a reference image to obtain a disparity image, thereby combining the denoising process with the depth image generation process, achieving a balance between processing efficiency and denoising effect. At the same time, the matching operation based on the denoised image is simpler and can effectively reduce the noise generated in the disparity image generation process, thereby reducing the noise generated in the depth image generation process, making the distance information about the target in the depth image more accurate.

[0049] The present invention fully considers the resource consumption of the chip platform, and processes the acquisition of the target speckle pattern and the first binarization operation in parallel, making full use of the CPU and chip hardware resources, and achieving a balance between processing efficiency and denoising effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a flowchart of a depth image generation method provided by one embodiment of the present invention;

[0051] Figure 2 This is a flowchart of denoising a target speckle pattern in a depth image generation method provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0052] The following is a further detailed description of a depth image generation method and system proposed in the present invention in conjunction with the accompanying drawings and specific embodiments. According to the following description, the advantages and features of the present invention will be clearer. It should be noted that the drawings are in a very simplified form and are not in precise proportions, which are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention. In order to make the purposes, features and advantages of the present invention more obvious and easy to understand, please refer to the accompanying drawings. It should be noted that the structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification for people familiar with this technology to understand and read, and are not used to limit the implementation conditions of the present invention, so they have no technical significance. Any modification of the structure, change in the proportional relationship or adjustment of the size should still fall within the scope of the technical content disclosed in the present invention without affecting the efficacy and purpose that can be achieved by the present invention.

[0053] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0054] Combined with attachment Figures 1-2 As shown, this embodiment provides a depth image generation method, including: step S110, emitting speckle structured light toward a target to obtain a target speckle pattern; step S120, denoising the target speckle pattern based on a binarization algorithm to obtain a denoised image; step S130, matching the denoised image with a reference image to obtain a disparity image; and step S140, generating a depth image of the target based on the disparity image.

[0055] Specifically, in the present embodiment, in step S110, a speckle projector may be used to emit locally random speckle structured light toward the target, and an infrared camera may be used to capture the speckle structured light incident on the target, thereby obtaining the target speckle pattern. However, the present invention is not limited thereto.

[0056] Please also refer to Figure 1 and Figure 2The step S120 includes: S1201, performing binarization processing on the target speckle pattern to obtain a binarized image; S1202, performing restoration processing on the binarized image to obtain a restored speckle pattern; and S1203, performing binarization processing on the restored speckle pattern to obtain the denoised image.

[0057] It can be understood that, in some other embodiments, the step S1201 includes: calculating, according to a preset binarization window, the mean of the grayscale values ​​of all pixels in the corresponding preset binarization window centered on each pixel in the target speckle image; and the pixel as the center in the target speckle image is the target center pixel; and assigning 0 and 1 to the target center pixel according to the grayscale value of the target center pixel, the mean of the grayscale values ​​of all pixels in the preset binarization window corresponding to the target center pixel, and a first preset threshold, to obtain the binarized image.

[0058] Specifically, in this embodiment, the size of the preset binary window can be N*N (where N is an odd number, for example, N can take the value of 3, 5, 9, etc.); the first preset threshold can be determined by conventional methods, such as the bimodal method, the P parameter method, the maximum inter-class variance method (Otsu method), the maximum entropy threshold method and the iterative method (optimal threshold method), etc.; and the principle of assigning 0 and 1 to the target center pixel point can be: if the grayscale value of the target center pixel point and / or the mean of the grayscale values ​​of all pixels in the preset binary window corresponding to the target center pixel point is greater than the first preset threshold value, the grayscale value of the target center pixel point becomes 1; if the grayscale value of the target center pixel point and / or the mean of the grayscale values ​​of all pixels in the preset binary window corresponding to the target center pixel point is not greater than the first preset threshold value, the grayscale value of the target center pixel point becomes 0. Thus, based on the grayscale value of each pixel in the target speckle pattern, the mean grayscale value of all pixels within a preset binarization window centered on that pixel, and the corresponding assignment principle, all pixels in the target speckle pattern can be assigned values ​​of 0 and 1, thereby completing the binarization of the target speckle pattern and obtaining the binarized image. Preferably, the target speckle pattern can be binarized while being acquired; once the acquisition of the target speckle pattern is complete, the binarized image can be obtained, effectively improving image processing efficiency. Furthermore, in other embodiments, the target center pixel can be assigned values ​​of 0 and 1 based on other assignment principles, but the present invention is not limited thereto.

[0059] Specifically, in this embodiment, the value of the first preset threshold can be relatively high to filter out bad pixels and noise (i.e., error noise caused by the camera sensor's light perception) in the target speckle pattern, thereby achieving a first denoising process for the target speckle pattern. At the same time, the value of the first preset threshold should not be too high to prevent excessive removal of valid points in the target speckle pattern, thereby avoiding loss of valid information. Furthermore, the value of the first preset threshold is also related to the speckle density range of the speckle projector and the intensity of the emitted speckle structured light, but the present invention is not limited thereto.

[0060] Specifically, in this embodiment, each pixel in the binary image may occupy 1 bit of memory, and a pixel grayscale value of 0 represents a dark point, and a pixel grayscale value of 1 represents a bright point, but the present invention is not limited thereto.

[0061] Please also refer to Figure 1 and Figure 2 The step S1202 includes: selecting pixels with a grayscale value of 1 in the binary image as pixels to be restored; and restoring the grayscale value of the pixels to be restored from 1 to 255 to obtain the restored speckle image.

[0062] It can be understood that, in some other embodiments, the step S1203 includes: calculating, according to the preset binarization window, the mean grayscale value of all pixels in the corresponding preset binarization window centered on each pixel in the restored speckle image; the pixel as the center of the restored speckle image is the restored center pixel; and assigning 0 and 1 to the restored center pixel according to the grayscale value of the restored center pixel, the mean grayscale value of all pixels in the preset binarization window corresponding to the restored center pixel, and a second preset threshold, to obtain the denoised image.

[0063] Specifically, because the binarization processing of the target speckle image in step S1201 is performed based on the preset binarization window, when the brightness (i.e., grayscale value) distribution of all pixels within the preset binarization window centered on a certain pixel in the target speckle image is significantly different (commonly seen when photographing the edge of a target), the assignment of 0 and 1 based on the first preset threshold may be distorted, inevitably introducing new noise into the binarized image. This introduced new noise cannot be removed by conventional 2D noise reduction algorithms. In this embodiment, a denoising process, i.e., binarization, can be performed on the binarized image in steps S1202 and S1203 to remove the new noise introduced in step S1201, but the present invention is not limited thereto.

[0064] Specifically, in this embodiment, in step S1202, the binarized image, in which each pixel occupies 1 bit of memory, can be restored to the restored speckle pattern, in which each pixel occupies 8 bits (i.e., 1 byte) of memory. This ensures that the restored speckle pattern and the target speckle pattern have the same image bit number, thereby facilitating subsequent binarization processing. More specifically, the pixels in the restored speckle pattern have only two grayscale values, 0 and 255, and new noise points in the binarized image will be inherited into the restored speckle pattern, but the present invention is not limited to this.

[0065] Specifically, in this embodiment, in step S1203, the restored speckle pattern can be used as input for a binarization operation, and a further binarization operation can be performed to achieve denoising of the restored speckle pattern, thereby obtaining the denoised image. Since the target speckle pattern has a total of 256 values ​​ranging from 0 to 255, and the restored speckle pattern has only two values, 0 and 255, compared to the target speckle pattern used as input for the first binarization operation in step S1201, the restored speckle pattern used as input for the second binarization operation in step S1203 has a simpler data composition and a more uniform distribution. There will not be a significant difference in the brightness (i.e., grayscale value) distribution of pixels within a preset binarization window centered on a certain pixel in the target speckle pattern. This avoids the phenomenon of reintroducing new noise while removing noise from the restored speckle pattern, thereby reducing noise generated in the subsequent disparity image generation process. However, the present invention is not limited to this.

[0066] Specifically, in the present embodiment, in the step S1203, the second preset threshold value can also be determined by conventional bimodal method, P parameter method, maximum inter-class variance method (Otsu method), maximum entropy threshold method and iterative method (optimal threshold method) and the like; and the principle of assigning 0 and 1 to the restored central pixel point can be: if the grayscale value of the restored central pixel point and / or the mean of the grayscale values ​​of all pixels in the preset binary window corresponding to the restored central pixel point is greater than the second preset threshold value, the grayscale value of the restored central pixel point becomes 1; if the grayscale value of the restored central pixel point and / or the mean of the grayscale values ​​of all pixels in the preset binary window corresponding to the restored central pixel point is not greater than the second preset threshold value, the grayscale value of the restored central pixel point becomes 0. In addition, in other embodiments, the restored central pixel point can also be assigned 0 and 1 based on other assignment principles, but the present invention is not limited thereto.

[0067] Furthermore, in this embodiment, the second preset threshold value can be set relatively low to effectively remove noise from the restored speckle pattern while preserving as many valid information pixels as possible. More specifically, the value of the second preset threshold value is also related to the speckle density range of the speckle projector and the intensity of the emitted speckle structured light. Furthermore, different second preset threshold values ​​can be set to meet the denoising level and denoising effect requirements of different devices, but the present invention is not limited thereto.

[0068] Specifically, since the speckle pattern moves as a whole at different distances, and the speckle size and distribution pattern do not change with distance, in step S130, the denoised image obtained based on the target can be matched with the reference image obtained based on the reference plane to obtain the disparity image of the target. More specifically, the matching operation process is as follows: based on a preset disparity calculation matching window, an image corresponding to the preset disparity calculation matching window is selected with any pixel coordinate (X, Y) as the center in the denoised image and recorded as the first image; similarly, an image corresponding to the preset disparity calculation matching window is selected with the coordinate (X, Y) as the center in the reference image and recorded as the second image; the central pixel point of the second image is moved along the x and y axes within a certain range, and an image corresponding to the preset disparity calculation matching window is selected with the moved central pixel as the center and recorded as the updated second image, and a matching operation is performed with the first image to find an image within the movement range that meets the preset matching requirements with the first image (for example, the grayscale image in the first image). The statistical value of the number of pixels with a grayscale value of 1 is 1 / 4 of the statistical value of the number of pixels with a grayscale value of 1 in the second image), and the image with the highest matching degree is recorded as the third image; then, based on the center coordinates (X', Y') of the third image, the coordinate deviation (XX', YY') of the pixel in the denoised image and the reference image is obtained, that is, the disparity coordinate; if none of the updated second image and the first image obtained after the entire x and y axes move within a certain range meets the preset matching requirements, it means that the best matching position has not been found, and the disparity coordinate of the pixel coordinate (X, Y) is forcibly set to a fixed invalid value; finally, all pixels in the denoised image are traversed to obtain the disparity image. Preferably, the size of the preset disparity calculation matching window is different from the size of the preset binarization window, and needs to be determined according to actual conditions, generally 25*25, but the present invention is not limited to this.

[0069] In this embodiment, the denoised image is obtained by indirectly performing a secondary binarization process on the target speckle pattern. In step S130, the denoised image is matched with the reference image to obtain the disparity image. This combines the denoising process with the depth image generation process, thereby ensuring a balance between processing efficiency and denoising effect. Furthermore, the matching operation based on the denoised image is simpler and can effectively reduce noise generated during the disparity image generation process. Furthermore, the reference image acquisition step specifically comprises: first, emitting speckle structured light at a preset distance toward a preset reference plane to obtain a reference speckle pattern; then, binarizing the reference speckle pattern to obtain the reference image.

[0070] Please continue to refer to Figure 1 The step S140 includes: generating an initial depth image according to the disparity image and the distance between the emission point of the speckle structured light and the acquisition point of the target speckle pattern; and performing denoising on the initial depth image using a filtering algorithm to obtain a final depth image.

[0071] Specifically, in this embodiment, the initial depth image can be obtained based on the disparity image, the focal length of the infrared camera, and the distance between the speckle projector and the infrared camera, and the initial depth image can be denoised using a median filtering algorithm to make the distance information about the target in the final depth image more accurate, but the present invention is not limited to this.

[0072] Based on the same inventive concept, this embodiment further provides a depth image generation system, comprising: an emission module for emitting speckle structured light toward a target; an acquisition module for acquiring a target speckle pattern; a denoising module, connected to the acquisition module, for denoising the target speckle pattern based on a binarization algorithm to obtain a denoised image; a matching module, connected to the denoising module, for matching the denoised image with a reference image to obtain a disparity image; and an image generation module, connected to the matching module, for generating a depth image of the target based on the disparity image.

[0073] Specifically, in this embodiment, the transmitting module can be a speckle projector, and the collecting module can be an infrared camera. By capturing the speckle structured light incident on the target, the target speckle pattern can be obtained. More specifically, prior to use, the speckle projector and the infrared camera must be initialized, for example, by initializing the camera register configuration and memory, but the present invention is not limited to this.

[0074] It can be understood that, in some embodiments, the denoising module includes: a first binarization unit, connected to the acquisition module, and configured to perform binarization processing on the target speckle pattern to obtain a binarized image; a restoration unit, connected to the first binarization unit, and configured to perform restoration processing on the binarized image to obtain a restored speckle pattern; and a second binarization unit, connected to the restoration unit and the matching module, respectively, and configured to perform binarization processing on the restored speckle pattern to obtain the denoised image.

[0075] In some embodiments, the first binarization unit is specifically configured to calculate, according to a preset binarization window, the mean grayscale value of all pixels in a corresponding preset binarization window centered on each pixel in the target speckle image; and the pixel in the target speckle image as the center is the target center pixel; and according to the grayscale value of the target center pixel, the mean grayscale value of all pixels in the preset binarization window corresponding to the target center pixel, and a first preset threshold, assign 0 and 1 to the target center pixel to obtain the binarized image.

[0076] In some embodiments, the restoration unit is specifically configured to select pixels with a grayscale value of 1 in the binary image as pixels to be restored; and restore the grayscale value of the pixels to be restored from 1 to 255 to obtain the restored speckle image.

[0077] In some embodiments, the second binarization unit is specifically configured to calculate, according to the preset binarization window, the mean grayscale value of all pixels in the corresponding preset binarization window centered on each pixel in the restored speckle image; and the pixel serving as the center in the restored speckle image is the restored center pixel; and according to the grayscale value of the restored center pixel, the mean grayscale value of all pixels in the preset binarization window corresponding to the restored center pixel, and a second preset threshold, assign 0 and 1 to the restored center pixel to obtain the denoised image.

[0078] Specifically, in this embodiment, the restoration unit can restore the binarized image in which each pixel occupies 1 bit of memory into the restored speckle pattern in which each pixel occupies 8 bits (i.e., 1 byte) of memory, so that the number of image bits of the restored speckle pattern and the target speckle pattern is the same, thereby allowing the restored speckle pattern and the target speckle pattern to share the input and output structure of the same binarization unit, that is, the first binarization unit and the second binarization unit can be the same binarization unit, thereby effectively reducing equipment costs and saving equipment installation space, but the present invention is not limited to this.

[0079] It can be understood that in some other embodiments, the image generation module is specifically used to generate an initial depth image based on the disparity image and the distance between the transmitting module and the acquisition module; and use a filtering algorithm to denoise the initial depth image to obtain a final depth image.

[0080] Specifically, in this embodiment, the initial depth image can be obtained based on the disparity image, the focal length of the infrared camera, and the distance between the speckle projector and the infrared camera, and the initial depth image can be denoised using a median filtering algorithm to make the distance information about the target in the final depth image more accurate, but the present invention is not limited to this.

[0081] In summary, this embodiment provides a depth image generation method and system. By emitting speckle structured light at a target, a target speckle pattern can be acquired. Based on a binarization algorithm, the target speckle pattern is subjected to denoising processing to obtain a denoised image. The denoised image is matched with a reference image to obtain a disparity image, thereby generating a depth image of the target. In this embodiment, a secondary binarization design approach is employed to denoise the target speckle pattern. The first binarization process involves binarizing the target speckle pattern to obtain a binarized image. This process filters out bad pixels and noise in the target speckle pattern, achieving the first denoising of the target speckle pattern. Since the binarized image obtained after the first binarization still contains a significant amount of noise, the binarized image is restored to a restored speckle pattern through restoration processing. The restored speckle pattern is then subjected to a second binarization process to obtain a denoised image. This process effectively removes noise in the restored speckle pattern while retaining as many valid points as possible in the restored speckle pattern, thereby reducing noise generated in the subsequent disparity image generation process. Since the grayscale values ​​of pixels in the target speckle image have a total of 256 values ​​from 0 to 255, and the grayscale values ​​of pixels in the restored speckle image have only two values, 0 and 255, the data composition of the restored speckle image is relatively simple and evenly distributed. There will be no large difference in the distribution of pixel brightness (i.e., grayscale value) within the corresponding preset binary window centered on a certain pixel, thereby avoiding the phenomenon of introducing new noise while removing noise in the restored speckle image, thereby reducing the noise generated in the depth image generation process and achieving a better denoising effect.

[0082] Although the present invention has been described in detail through the above preferred embodiments, it should be understood that the above description is not intended to limit the present invention. After reading the above description, various modifications and substitutions of the present invention will become apparent to those skilled in the art. Therefore, the scope of protection of the present invention should be defined by the appended claims.

Claims

1. A depth image generation method, characterized in that: include: Emitting speckle structured light toward the target to obtain a target speckle pattern; Based on a binarization algorithm, denoising the target speckle pattern to obtain a denoised image; Performing a matching operation on the denoised image and a reference image to obtain a disparity image; as well as generating a depth image of the target according to the disparity image; The step of performing denoising on the target speckle pattern based on a binarization algorithm includes: performing binarization processing on the target speckle pattern to obtain a binarized image; performing restoration processing on the binarized image to obtain a restored speckle pattern; and performing binarization processing on the restored speckle image to obtain the denoised image; The step of performing binarization processing on the target speckle pattern includes: According to a preset binary window, the mean grayscale value of all pixels in the corresponding preset binary window centered at each pixel in the target speckle pattern is calculated; and the pixel point as the center in the target speckle pattern is the target center pixel point; and Assigning 0 and 1 to the target center pixel according to the grayscale value of the target center pixel, the average grayscale value of all pixels in a preset binarization window corresponding to the target center pixel, and a first preset threshold to obtain the binarized image; If the mean of the grayscale values ​​of all pixels in the preset binary window corresponding to the target center pixel is greater than the first preset threshold, the grayscale value of the target center pixel becomes 1; if the mean of the grayscale values ​​of all pixels in the preset binary window corresponding to the target center pixel is not greater than the first preset threshold, the grayscale value of the target center pixel becomes 0; The step of restoring the binary image includes: Selecting pixels with a grayscale value of 1 in the binary image as pixels to be restored; and Restoring the grayscale value of the pixel to be restored from 1 to 255 to obtain the restored speckle image; The step of performing binarization processing on the restored speckle image includes: According to the preset binarization window, the mean grayscale value of all pixels in the corresponding preset binarization window centered at each pixel in the restored speckle image is calculated; and the pixel as the center in the restored speckle image is the restored center pixel; and Assigning 0 and 1 to the restored central pixel according to the grayscale value of the restored central pixel, the average grayscale value of all pixels in a preset binarization window corresponding to the restored central pixel, and a second preset threshold to obtain the denoised image; If the mean of the grayscale values ​​of all pixels in the preset binary window corresponding to the restored center pixel is greater than the second preset threshold, the grayscale value of the restored center pixel becomes 1; if the mean of the grayscale values ​​of all pixels in the preset binary window corresponding to the restored center pixel is not greater than the second preset threshold, the grayscale value of the restored center pixel becomes 0.

2. The depth image generation method according to claim 1, wherein: The step of generating a depth image according to the disparity image comprises: generating an initial depth image according to the disparity image and the distance between the emission point of the speckle structured light and the acquisition point of the target speckle pattern; and A filtering algorithm is used to perform denoising on the initial depth image to obtain a final depth image.

3. A depth image generation system, characterized in that: include: A transmitting module, used for transmitting speckle structured light toward a target; An acquisition module, used to obtain a target speckle pattern; a denoising module, connected to the acquisition module, for performing denoising processing on the target speckle pattern based on a binarization algorithm to obtain a denoised image; a matching module, connected to the denoising module, for performing a matching operation on the denoised image and a reference image to obtain a disparity image; as well as An image generation module, connected to the matching module, for generating a depth image of the target based on the disparity image; The denoising module includes: a first binarization unit, connected to the acquisition module, and configured to perform binarization processing on the target speckle pattern to obtain a binarized image; a restoration unit connected to the first binarization unit, configured to perform restoration processing on the binarized image to obtain a restored speckle pattern; and a second binarization unit, connected to the restoration unit and the matching module respectively, for performing binarization processing on the restored speckle image to obtain the denoised image; The first binarization unit is specifically configured to calculate, based on a preset binarization window, the mean grayscale value of all pixels in a corresponding preset binarization window centered at each pixel in the target speckle pattern; and the pixel in the target speckle pattern that is the center is the target center pixel; and Assigning 0 and 1 to the target center pixel according to the grayscale value of the target center pixel, the average grayscale value of all pixels in a preset binarization window corresponding to the target center pixel, and a first preset threshold to obtain the binarized image; If the mean of the grayscale values ​​of all pixels in the preset binary window corresponding to the target center pixel is greater than the first preset threshold, the grayscale value of the target center pixel becomes 1; if the mean of the grayscale values ​​of all pixels in the preset binary window corresponding to the target center pixel is not greater than the first preset threshold, the grayscale value of the target center pixel becomes 0; The restoration unit is specifically configured to select pixels with a grayscale value of 1 in the binary image as pixels to be restored; and restore the grayscale value of the pixels to be restored from 1 to 255 to obtain the restored speckle image; The second binarization unit is specifically configured to calculate, based on the preset binarization window, the mean grayscale value of all pixels in the corresponding preset binarization window centered at each pixel in the restored speckle image; and the pixel in the restored speckle image that is the center is the restored center pixel; and Assigning 0 and 1 to the restored central pixel according to the grayscale value of the restored central pixel, the average grayscale value of all pixels in a preset binarization window corresponding to the restored central pixel, and a second preset threshold to obtain the denoised image; If the mean of the grayscale values ​​of all pixels in the preset binary window corresponding to the restored center pixel is greater than the second preset threshold, the grayscale value of the restored center pixel becomes 1; if the mean of the grayscale values ​​of all pixels in the preset binary window corresponding to the restored center pixel is not greater than the second preset threshold, the grayscale value of the restored center pixel becomes 0.

4. The depth image generation system according to claim 3, wherein: The image generation module is specifically configured to generate an initial depth image according to the disparity image and the distance between the transmitting module and the acquiring module; as well as A filtering algorithm is used to perform denoising on the initial depth image to obtain a final depth image.

Images