Deep recovery method, device, electronic device and computer-readable storage medium

Abstract

Embodiments of the present application relate to the field of machine vision technology and disclose a depth recovery method, apparatus, electronic device, and computer-readable storage medium. The method comprises: determining a target infrared region in an acquired infrared image, and determining a target speckle region in a speckle pattern corresponding to the infrared image based on the target infrared region; generating several candidate speckle patterns based on the valid data bits of the target speckle region and a preset target data bit selection strategy, wherein different candidate speckle patterns select different target data bits; performing quality scoring on the candidate speckle patterns based on a preset quality scoring algorithm, and generating a depth map corresponding to the infrared image based on the candidate speckle pattern with the highest score. The depth recovery method provided by the embodiments of the present application can ensure that all speckles involved in depth recovery are high-quality speckles, thereby improving the accuracy of depth recovery and enhancing the stability and universality of depth cameras used under complex lighting conditions.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of machine vision technology, and in particular to a depth recovery method, device, electronic device, and computer-readable storage medium. Background Art

[0002] Depth cameras can obtain depth information of the target scene in real time, and provide technical support for applications such as motion capture, 3D modeling, indoor navigation and positioning, face recognition, parts scanning, detection and sorting, security monitoring and people counting, etc. The depth recovery and depth imaging of depth cameras on the market are mainly divided into time of flight (TOF), binocular vision technology and monocular structured light technology. Among them, monocular structured light technology is the current mainstream solution for consumer-grade face recognition and payment scenarios. The monocular structured light depth camera projects an irregular speckle pattern to the target scene through a speckle projector, and uses an infrared lens to capture the projection of the speckle pattern on the target scene. Finally, the speckle feature matching is performed based on the captured infrared image and the pre-calibrated reference image to perform depth recovery.

[0003] However, the inventors of this application discovered that when the industry uses monocular structured light technology for depth recovery, the server operates on the entire image. In fact, many background areas do not require depth recovery. Under limited computing resources, the accuracy of depth recovery will be greatly reduced and cannot meet actual needs. At the same time, the lighting conditions in different scenes are different. Excessive light or too dark light will lead to poor speckle quality in the speckle map taken by the depth camera, which in turn affects the accuracy of depth recovery. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to provide a depth recovery method, apparatus, electronic device, and computer-readable storage medium to ensure that the speckles involved in depth recovery are of high quality, thereby improving the accuracy of depth recovery and enhancing the stability and universality of depth cameras under complex lighting conditions.

[0005] To solve the above technical problems, an embodiment of the present application provides a depth recovery method, comprising the following steps: determining a target infrared area in an acquired infrared image, and determining a target speckle area in a speckle map corresponding to the infrared image based on the target infrared area; wherein the target infrared area includes a preset target object; generating a plurality of candidate speckle patterns based on data valid bits of the target speckle area and a preset target data bit selection strategy; wherein different target data bits are selected for different candidate speckle patterns; performing quality scoring on each of the plurality of candidate speckle patterns based on a preset quality scoring algorithm, and generating a depth map corresponding to the infrared image based on the candidate speckle pattern with the highest score.

[0006] An embodiment of the present application further provides a depth recovery device, comprising: a positioning module for determining a target infrared region in an acquired infrared image, and determining a target speckle region in a speckle map corresponding to the infrared image based on the target infrared region; a grouping module for generating a plurality of candidate speckle patterns based on data valid bits of the target speckle region and a preset target data bit selection strategy, wherein different candidate speckle patterns select different target data bits; a scoring module for performing quality scoring on each of the plurality of candidate speckle patterns based on a preset quality scoring algorithm; and an execution module for generating a depth map corresponding to the infrared image based on the candidate speckle pattern with the highest score.

[0007] An embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above-mentioned deep recovery method.

[0008] An embodiment of the present application further provides a computer-readable storage medium storing a computer program, wherein the computer program implements the above-mentioned deep recovery method when executed by a processor.

[0009] The depth recovery method, device, electronic device, and computer-readable storage medium provided by the embodiments of the present application first perform preset target object detection in the acquired infrared image to determine the target infrared area, and then determine the target speckle area in the speckle map corresponding to the infrared image based on the target infrared area. Then, based on the data valid bits of the target speckle area and the preset target data bit selection strategy, several candidate speckle maps are generated, wherein different candidate speckle maps select different target data bits. Finally, quality scores are respectively performed on the several candidate speckle maps based on a preset quality scoring algorithm, and a depth map corresponding to the infrared image is generated based on the candidate speckle map with the highest score. The present application determines the target infrared area and the target speckle area in the infrared image and the speckle map, and subsequently only the target speckle area, i.e., the target speckle area, is selected. The operation is performed on the foreground area of ​​the target object, which can save computing resources. Considering that the industry will normalize the data in the speckle map obtained by the depth camera in order to reduce the amount of data, and the data bit conversion based on the maximum and minimum grayscale values ​​of each pixel in the whole image is seriously affected by the light intensity, it is easy to cause the normalized speckle to be too bright or too dark. The embodiment of the present application directly groups the data valid bits of the target speckle area according to the preset target data bit selection strategy, obtains several candidate speckle maps with different selected target data bits, and finally selects the candidate speckle map with the best quality for depth recovery, ensuring that the speckles involved in depth recovery are all high-quality speckles, thereby improving the accuracy of depth recovery and enhancing the stability and universality of the depth camera under complex lighting conditions.

[0010] In addition, the target speckle area has K valid bits of data, where K is an integer greater than 2, the preset target data bit selection strategy includes a high-bit selection strategy and a low-bit selection strategy, and the candidate speckle patterns include a high-bit candidate speckle pattern and a low-bit candidate speckle pattern; generating a plurality of candidate speckle patterns according to the target speckle area's valid bits of data and the preset target data bit selection strategy includes: obtaining a grayscale value of each pixel in the target speckle area, where the grayscale value is K-bit data; selecting, according to the high-bit selection strategy, the first L bits of the grayscale value of each pixel as the first grayscale value of each pixel, and generating High-order candidate speckle pattern; wherein, L is an integer less than K; according to the low-order selection strategy, the last L bits of the grayscale value of each pixel are selected as the second grayscale value of each pixel to generate a low-order candidate speckle pattern. In order to reduce the amount of data, the industry will compress the original K-bit grayscale value data into L bits. This process is the grayscale value normalization process. The present application abandons the normalization based on the maximum and minimum grayscale values ​​of each pixel in the entire image, and instead selects L bits from the K-bit data to achieve the purpose of reducing the data amount, greatly reducing the amount of calculation and saving the normalization time, thereby improving the quality and efficiency of depth recovery.

[0011] In addition, the target speckle area has 10 valid bits of data, the preset target data bit selection strategy includes a high 8-bit selection strategy, a middle 8-bit selection strategy and a low 8-bit selection strategy, and the candidate speckle patterns include a high 8-bit candidate speckle pattern, a middle 8-bit candidate speckle pattern and a low 8-bit candidate speckle pattern; generating a plurality of candidate speckle patterns according to the valid bits of data of the target speckle area and the preset target data bit selection strategy includes: obtaining the grayscale value of each pixel in the target speckle area, the grayscale value being 10-bit data; selecting the first 8 bits of the grayscale value of each pixel as the third grayscale value of each pixel according to the high 8-bit selection strategy, to generate a high 8-bit candidate speckle pattern; selecting the second to ninth bits of the grayscale value of each pixel as the fourth grayscale value of each pixel according to the middle 8-bit selection strategy, to generate a middle 8-bit candidate speckle pattern; selecting the ninth bit of the grayscale value of each pixel as the fourth grayscale value of each pixel according to the low 8-bit selection strategy. The last 8 bits of the grayscale value of each pixel are used as the fifth grayscale value of each pixel to generate a low-8-bit candidate speckle pattern. The effective bits of the data corresponding to the speckle pattern captured by the depth camera are generally 10 bits, that is, the grayscale value of each pixel in the speckle pattern is generally 10-bit data. The present application abandons the method of converting 10-bit data into 8-bit data based on the maximum and minimum grayscale values ​​of each pixel in the entire image, and instead generates three different groups of 8-bit data according to the preset high 8-bit selection strategy, middle 8-bit selection strategy and low 8-bit selection strategy. For scenes with strong illumination, the low 8 bits of the 10-bit data are basically 1, and the use of the high 8-bit data will not cause the normalized speckle pattern to be too bright. For scenes with weak illumination, the high 8 bits of the 10-bit data are basically 0, and the use of the low 8-bit data will not cause the normalized speckle pattern to be too dark. Such a normalization method can better adapt to scenes with different illumination intensities and greatly reduce the amount of calculation.

[0012] In addition, the quality scoring of the several candidate speckle patterns is performed based on a preset quality scoring algorithm, including: traversing the several candidate speckle patterns, and calculating the grayscale value mean of the current candidate speckle pattern according to the grayscale value of each pixel in the current candidate speckle pattern; calculating the signal-to-noise ratio of the current candidate speckle pattern according to the scattered speckles and non-scattered speckles in the current candidate speckle pattern; and calculating the quality score of each candidate speckle pattern according to the grayscale value mean of each candidate speckle pattern, the signal-to-noise ratio of each candidate speckle pattern and the preset quality scoring algorithm. The grayscale value mean of the candidate speckle pattern can represent the average brightness of the candidate speckle pattern, and the signal-to-noise ratio of the candidate speckle pattern can represent whether the features of the speckle pattern are obvious. The present application combines these two aspects to perform quality scoring on each candidate speckle pattern, and can determine and select the candidate speckle pattern with the highest quality, the most appropriate brightness and the most obvious features, thereby further improving the accuracy of depth recovery.

[0013] In addition, the calculating the signal-to-noise ratio of the current candidate speckle pattern based on the speckles and non-speckle in the current candidate speckle pattern includes: upsampling the current candidate speckle pattern to obtain an upsampled current candidate speckle pattern; determining the speckle region and the non-speckle region of the current candidate speckle pattern based on the upsampled current candidate speckle pattern and a preset speckle region extraction method; wherein the preset speckle region extraction method includes Gaussian blur, edge extraction, ellipse fitting, and ellipse detection; respectively calculating the average grayscale value of the speckle region and the average grayscale value of the non-speckle region in the current candidate speckle pattern; calculating the ratio between the average grayscale value of the speckle region and the average grayscale value of the non-speckle region, and using the ratio as the signal-to-noise ratio of the current candidate speckle pattern. Considering that the speckle regions are relatively dispersed and each speckle region is relatively small, the server may upsample the current candidate speckle pattern, that is, amplify the current candidate speckle pattern, so as to more accurately detect the speckle region of the current candidate speckle pattern, thereby improving the accuracy and reliability of the calculated signal-to-noise ratio.

[0014] In addition, the signal-to-noise ratio of the current candidate speckle pattern is calculated based on the speckles and non-speckles in the current candidate speckle pattern, including: judging whether the grayscale value mean of the current candidate speckle pattern is within a preset grayscale value range; if the grayscale value mean of the current candidate speckle pattern is within the preset grayscale value range, then it is within the preset grayscale value range; if the grayscale value mean of the current candidate speckle pattern is outside the preset grayscale value range, then directly discarding the current candidate speckle pattern. The present application sets a trigger condition before quality scoring, that is, the grayscale value mean of the candidate speckle pattern needs to be within the preset grayscale value range. If it is within the preset grayscale value range, it means that the brightness of the candidate speckle pattern is relatively reasonable, neither too bright nor too dark, and directly discarding the too bright or too dark speckle pattern, further saving computing resources and effectively improving the speed of depth recovery.

[0015] In addition, the preset target object is a human face, and the target infrared area is determined in the acquired infrared image, and the target speckle area is determined in the speckle map corresponding to the infrared image based on the target infrared area, including: performing face detection on the acquired infrared image, determining the face contour in the infrared image, and using the circumscribed rectangle of the face contour as the target infrared area; determining the same-name points of each vertex in the speckle map corresponding to the infrared image based on the coordinates of the vertices of the target infrared area; and connecting the same-name points of each vertex to obtain the target speckle area. Considering that the face contour is not a regular shape, the present application selects the circumscribed rectangle of the face contour in the infrared image as the target infrared area when the preset target object is a face, to facilitate subsequent quality evaluation and depth recovery. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] One or more embodiments are exemplarily described by the figures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments.

[0017] Figure 1 This is a process of a deep recovery method according to an embodiment of the present application Figure 1 ;

[0018] Figure 2 According to an embodiment of the present application, a process of generating several candidate speckle patterns according to the data valid bits of the target speckle area and the preset target data bit selection strategy is described. Figure 1 ;

[0019] Figure 3 According to an embodiment of the present application, a process of generating several candidate speckle patterns according to the data valid bits of the target speckle area and the preset target data bit selection strategy is described. Figure 2 ;

[0020] Figure 4 is a flowchart of performing quality scoring on a plurality of candidate speckle patterns based on a preset quality scoring algorithm according to an embodiment of the present application;

[0021] Figure 5 is a flow chart for calculating the signal-to-noise ratio of a current candidate speckle pattern based on speckles and non-speckles in the current candidate speckle pattern according to an embodiment of the present application;

[0022] Figure 6 This is a process of a deep recovery method according to another embodiment of the present application Figure 2 ;

[0023] Figure 7 This is a flow chart of determining a target infrared region in an acquired infrared image and determining a target speckle region in a speckle pattern corresponding to the infrared image based on the target infrared region according to an embodiment of the present application;

[0024] Figure 8 is a schematic diagram of a depth recovery device according to another embodiment of the present application;

[0025] Figure 9 is a structural diagram of an electronic device according to another embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, each embodiment of the present application will be described in detail below with reference to the accompanying drawings. However, it will be understood by those skilled in the art that in each embodiment of the present application, many technical details are proposed to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present application can be implemented. The division of the following embodiments is for convenience of description and should not constitute any limitation on the specific implementation of the present application. The various embodiments can be combined and referenced with each other under the premise of no contradiction.

[0027] To facilitate understanding of the embodiments of the present application, the following first introduces the relevant content of “deep recovery” introduced in the description of the embodiments of the present application.

[0028] The depth recovery methods of depth cameras in the industry are mainly divided into TOF method, binocular vision technology and monocular structured light method. The TOF method measures the distance by measuring the flight time of light. The depth camera continuously emits laser pulses to the target scene and receives the optical fiber reflected back by the target scene through the built-in sensor, thereby obtaining the exact distance by detecting the round-trip flight time of the light pulse. The TOF method places high demands on the hardware of the depth camera, especially the time measurement module, which has a large amount of computing power and consumes a lot of resources, resulting in low accuracy of the depth value of the edge of the recovered target object, which cannot meet the consumer-grade face recognition scenarios with high requirements for depth recovery accuracy.

[0029] The depth recovery method based on binocular vision technology uses the principle of parallax. The depth camera takes multiple images of the object to be measured from different positions, and obtains the depth information of the object to be measured by calculating the position deviation between corresponding points of the images. However, binocular vision technology is very sensitive to the lighting conditions of the application environment. Changes in light can cause excessive deviations between multiple images, which in turn leads to reduced matching accuracy, resulting in relatively low depth recovery accuracy. It cannot well meet the requirements of consumer-grade face recognition scenarios with high requirements for depth recovery accuracy.

[0030] Monocular structured light technology is the mainstream depth recovery solution in the industry for consumer-grade face recognition scenarios. The depth camera is equipped with a structured light projector and an infrared lens. The structured light projector projects an irregular speckle pattern onto the target scene, and the infrared lens captures the projection of the speckle pattern on the target scene. Depth recovery is then performed based on the captured projection image, a pre-calibrated reference image, and a preset matching algorithm to obtain the depth information of the target scene. However, depth recovery in the industry operates on the entire image. In fact, many background areas in the image do not require depth recovery. In the case of limited computing resources, full-image depth recovery will greatly reduce the accuracy of depth recovery. At the same time, different scenes have different lighting conditions. Excessive light or dark light will result in poor speckle quality in the speckle image captured by the depth camera, and the quality of the speckle involved in depth recovery is not high, which in turn affects the accuracy of depth recovery.

[0031] In order to solve the technical problems of low accuracy of the above-mentioned depth recovery and susceptibility to changes in lighting conditions, an embodiment of the present application provides a depth recovery method, which is applied to an electronic device, wherein the electronic device can be a terminal or a server. In this embodiment and the following embodiments, the electronic device is explained using the server as an example. The implementation details of the depth recovery method of this embodiment are described in detail below. The following content is only the implementation details provided for the convenience of understanding and is not necessary for the implementation of this solution.

[0032] The specific process of the deep recovery method of this embodiment can be as follows: Figure 1 Shown, including:

[0033] Step 101 : determining a target infrared region in the acquired infrared image, and determining a target speckle region in the speckle image corresponding to the infrared image according to the target infrared region.

[0034] In a specific implementation, when performing depth recovery, the server can first perform target object detection on the acquired infrared image, that is, the infrared image taken by the depth camera, and determine the target infrared area in the infrared image. The target infrared area contains a preset target object. Then, the server uses the target infrared area as a reference to determine the target speckle area in the speckle pattern corresponding to the infrared image. The infrared image and the speckle pattern corresponding to the infrared image are homologous data taken by the same lens of the depth camera, and the two are naturally aligned. The preset target object can be selected and set by technicians in this field according to the scenario of the depth recovery application.

[0035] It is understandable that in actual applications, the depth camera needs the depth information of the target object, and does not need the depth information of the background area. If depth recovery is performed on the entire image, the depth recovery of the background area will occupy a large amount of computing resources, which indirectly leads to insufficient computing resources in the target object area, thereby affecting the accuracy of depth recovery. The present application first determines the target infrared area and target speckle area containing the preset target object in the infrared image and speckle image, and then only performs depth recovery on the target infrared area and target speckle area, which can improve the utilization of computing resources and improve the accuracy and speed of depth recovery.

[0036] In one example, depth recovery is applied to the scanning and inspection of parts. The inspected parts are bolts, that is, the target objects preset in this application are bolts. The server performs target object detection on the infrared image taken by the depth camera and detects the position of the bolts in the infrared image, that is, the outline of the bolts is used as the target infrared area. The server obtains the coordinates of each pixel point in the target infrared area and finds points with the same coordinates in the speckle pattern corresponding to the infrared image, thereby determining the position of the bolts in the speckle pattern and obtaining the target speckle area.

[0037] Step 102 : generating a plurality of candidate speckle patterns according to the valid data bits of the target speckle area and a preset target data bit selection strategy.

[0038] Specifically, after determining the target speckle area in the speckle pattern corresponding to the infrared image, the server can generate several candidate speckle patterns according to the data valid bits of the target speckle area and the preset target data bit selection strategy, wherein different candidate speckle patterns select different target data bits.

[0039] In a specific implementation, the data corresponding to each pixel in the target speckle area is the grayscale value, and the data valid bits of the target speckle area are the data valid bits of the binary data representing the grayscale value of each pixel captured by the depth camera.

[0040] In one example, the target speckle area has 12 valid bits of data, and the preset target data bit selection strategy is to select 8 consecutive bits. For example, the 12-bit data bits of the grayscale value of a certain pixel are "001010000101". The server selects 8 consecutive bits, that is, selecting the 1st to 8th bits can obtain "00101000", selecting the 2nd to 9th bits can obtain "01010000", selecting the 3rd to 10th bits can obtain "10100001", selecting the 4th to 11th bits can obtain "01000010", and selecting the 5th to 12th bits can obtain "10000101". According to this target data bit selection strategy, the server can generate a total of 5 candidate speckle patterns.

[0041] Step 103 : performing quality scoring on a plurality of candidate speckle patterns based on a preset quality scoring algorithm, and generating a depth map corresponding to the infrared image based on the candidate speckle pattern with the highest score.

[0042] Specifically, after obtaining several candidate speckle patterns, the server can perform quality scores on the candidate speckle patterns based on a preset quality scoring algorithm, and generate a depth map corresponding to the infrared image according to the candidate speckle pattern with the highest score.

[0043] In one example, the preset quality scoring algorithm may be a digital image correlation method, that is, the server may perform a quality score on each candidate speckle pattern by calculating the second-order derivative of the average grayscale value of each candidate speckle pattern.

[0044] In another example, the server may perform a quality score on each candidate speckle pattern by calculating the gradient of the grayscale value gradient of each candidate speckle pattern.

[0045] In this embodiment, the server first performs a preset target object detection in the acquired infrared image to determine the target infrared area, and then determines the target speckle area in the speckle map corresponding to the infrared image based on the target infrared area. Then, based on the data valid bits of the target speckle area and the preset target data bit selection strategy, a number of candidate speckle patterns are generated, wherein different candidate speckle patterns select different target data bits. Finally, the quality of the candidate speckle patterns is respectively scored based on a preset quality scoring algorithm, and a depth map corresponding to the infrared image is generated based on the candidate speckle pattern with the highest score. In this application, the target infrared area and the target speckle area are determined in the infrared image and the speckle map, and subsequent operations are performed only on the target speckle area, that is, the foreground area with the target object. It can save computing resources. Considering that the industry will normalize the data in the speckle map obtained by the depth camera in order to reduce the amount of data, and the data bit conversion based on the maximum and minimum grayscale values ​​of each pixel in the whole image is seriously affected by the light intensity, it is easy to cause the normalized speckle to be too bright or too dark. In the embodiment of the present application, the valid data bits of the target speckle area are directly grouped according to the preset target data bit selection strategy to obtain several candidate speckle maps with different selected target data bits. Finally, the candidate speckle map with the best quality is selected for depth recovery, ensuring that the speckles involved in depth recovery are all high-quality speckles, thereby improving the accuracy of depth recovery and enhancing the stability and universality of the depth camera under complex lighting conditions.

[0046] In one embodiment, the target speckle area has K valid data bits, where K is an integer greater than 2. The preset target data bit selection strategy includes a high-bit selection strategy and a low-bit selection strategy. The candidate speckle patterns include a high-bit candidate speckle pattern and a low-bit candidate speckle pattern. The server generates a number of candidate speckle patterns according to the target speckle area's valid data bits and the preset target data bit selection strategy. Figure 2 The steps shown in the figure are as follows:

[0047] Step 201 : obtaining the grayscale value of each pixel in the target speckle area, where the grayscale value of each pixel is K-bit data.

[0048] Specifically, after determining the target speckle area in the speckle image corresponding to the infrared image, the server can traverse each pixel in the target speckle area to obtain the grayscale value of each pixel in the target speckle area. The grayscale value of each pixel is binary K-bit data.

[0049] Step 202 : According to the high-bit selection strategy, the first L bits of the grayscale value of each pixel are selected as the first grayscale value of each pixel to generate a high-bit candidate speckle pattern, where L is an integer less than K.

[0050] In a specific implementation, according to the high-bit selection strategy, the server selects the first L bits of the grayscale value of each pixel, that is, the 1st to Lth bits of data, as the first grayscale value of each pixel. After obtaining the first grayscale value of each pixel, the server generates a high-bit candidate speckle pattern according to the first grayscale value of each pixel.

[0051] In one example, K is 12 and L is 9. According to the high-bit selection strategy, the server selects the first 9 bits of the grayscale value of each pixel as the first grayscale value of each pixel, thereby generating a high-bit candidate speckle pattern.

[0052] In one example, K is 10 and L is 4. According to the high-bit selection strategy, the server selects the first 4 bits of the grayscale value of each pixel as the first grayscale value of each pixel, thereby generating a high-bit candidate speckle pattern.

[0053] Step 203 : According to the low-bit selection strategy, the last L bits of the grayscale value of each pixel are selected as the second grayscale value of each pixel to generate a low-bit candidate speckle pattern.

[0054] In a specific implementation, according to the low-bit selection strategy, the server selects the last L bits of the grayscale value of each pixel, that is, the K-L+1th to Kth bits of data as the second grayscale value of each pixel. After obtaining the second grayscale value of each pixel, the server generates a low-bit candidate speckle pattern according to the second grayscale value of each pixel.

[0055] In one example, K is 12 and L is 9. According to the low-bit selection strategy, the server selects the last 9 bits of the grayscale value of each pixel as the second grayscale value of each pixel, thereby generating a low-bit candidate speckle pattern.

[0056] In one example, K is 10 and L is 4. According to the low-bit selection strategy, the server selects the last 4 bits of the grayscale value of each pixel as the second grayscale value of each pixel, thereby generating a low-bit candidate speckle pattern.

[0057] In this embodiment, considering that in order to reduce the amount of data in the industry, the original K-bit grayscale value data will be compressed into L bits. This process is the grayscale value normalization process. This application abandons the normalization based on the maximum and minimum grayscale values ​​of each pixel point in the entire image, but selects L bits from the K-bit data to achieve the purpose of reducing the amount of data, which greatly reduces the amount of calculation and saves the normalization time, thereby improving the quality and efficiency of depth recovery.

[0058] In one embodiment, the target speckle area has 10 valid data bits, and the preset target data bit selection strategy includes an upper 8-bit selection strategy, a middle 8-bit selection strategy, and a lower 8-bit selection strategy. The server generates several candidate speckle patterns including an upper 8-bit candidate speckle pattern, a middle 8-bit candidate speckle pattern, and a lower 8-bit candidate speckle pattern. The server generates several candidate speckle patterns according to the valid data bits of the target speckle area and the preset target data bit selection strategy. Figure 3 The steps shown in the figure are as follows:

[0059] Step 301 : obtaining the grayscale value of each pixel in the target speckle area, where the grayscale value of each pixel is 10-bit data.

[0060] Specifically, after determining the target speckle area in the speckle image corresponding to the infrared image, the server can traverse each pixel in the target speckle area to obtain the grayscale value of each pixel in the target speckle area. The grayscale value of each pixel is 10-bit binary data.

[0061] Step 302 : According to the high 8-bit selection strategy, the first 8 bits of the grayscale value of each pixel are selected as the third grayscale value of each pixel to generate a high 8-bit candidate speckle pattern.

[0062] In a specific implementation, according to the high 8-bit selection strategy, the server selects the first 8 bits of the grayscale value of each pixel, that is, the 1st to 8th bits of data, as the third grayscale value of each pixel. After obtaining the third grayscale value of each pixel, the server generates the high 8-bit candidate speckle pattern according to the third grayscale value of each pixel.

[0063] In an example, the grayscale value of the 10-bit data of a pixel point in the target speckle area is "0100101101", and the server obtains the third grayscale value of the 8-bit data of the pixel point as "01001011" according to the high 8-bit selection strategy.

[0064] Step 303 : According to the middle 8-bit selection strategy, the second to ninth bits of the grayscale value of each pixel are selected as the fourth grayscale value of each pixel to generate a middle 8-bit candidate speckle pattern.

[0065] In a specific implementation, according to the middle 8-bit selection strategy, the server selects the 2nd to 9th bit data of the grayscale value of each pixel as the fourth grayscale value of each pixel. After obtaining the fourth grayscale value of each pixel, the server generates the middle 8-bit candidate speckle pattern according to the fourth grayscale value of each pixel.

[0066] In an example, the grayscale value of the 10-bit data of a pixel point in the target speckle area is "0100101101", and the server obtains the fourth grayscale value of the 8-bit data of the pixel point as "10010110" according to the middle 8-bit selection strategy.

[0067] Step 304 : According to the low 8-bit selection strategy, the last 8 bits of the grayscale value of each pixel are selected as the fifth grayscale value of each pixel to generate a low 8-bit candidate speckle pattern.

[0068] In a specific implementation, according to the low 8-bit selection strategy, the server selects the last 8 bits of the grayscale value of each pixel, that is, the 3rd to 10th bits of data, as the fifth grayscale value of each pixel. After obtaining the fifth grayscale value of each pixel, the server generates a low 8-bit candidate speckle pattern according to the fifth grayscale value of each pixel.

[0069] In an example, the grayscale value of the 10-bit data of a pixel point in the target speckle area is "0100101101", and the server obtains the fifth grayscale value of the 8-bit data of the pixel point as "00101101" according to the low 8-bit selection strategy.

[0070] In this embodiment, the effective bits of the data corresponding to the speckle pattern captured by the depth camera are generally 10 bits, that is, the grayscale value of each pixel in the speckle pattern is generally 10-bit data. In order to reduce the amount of data, the industry will convert these 10-bit data into 8-bit data, that is, normalize the grayscale value to 0 to 255. Traditional normalization is to traverse the target speckle area, find the maximum grayscale value and the minimum grayscale value, and use the maximum grayscale value and the minimum grayscale value to normalize the target speckle area. However, when the light intensity of the shooting environment is too strong or too dark, the depth camera method cannot accurately know whether the current scene is indoor or outdoor, and whether the light intensity is strong or weak. The maximum grayscale value of each pixel in the target speckle area will be too large or the minimum grayscale value will be too small, which results in poor quality of the target speckle area after normalization. This greatly reduces the accuracy of depth recovery. The present application abandons the method of converting 10-bit data into 8-bit data based on the maximum and minimum grayscale values ​​of each pixel in the entire image. Instead, three different groups of 8-bit data are generated according to the preset high 8-bit selection strategy, middle 8-bit selection strategy, and low 8-bit selection strategy. For scenes with strong illumination, the low 8 bits of the 10-bit data are basically 1, and the use of high 8-bit data will not cause the normalized speckle pattern to be too bright. For scenes with very weak illumination, the high 8 bits of the 10-bit data are basically 0, and the use of low 8-bit data will not cause the normalized speckle pattern to be too dark. This normalization method can better adapt to scenes with different illumination intensities and greatly reduce the amount of calculation. There is no need to add a light sensing module to the depth camera to detect the current illumination conditions, saving the hardware cost of the depth camera.

[0071] In one embodiment, the server performs quality scoring on several candidate speckle patterns based on a preset quality scoring algorithm, which can be done as follows: Figure 4 The steps shown in the figure are as follows:

[0072] Step 401: traverse a number of candidate speckle patterns, and calculate the mean grayscale value of the current candidate speckle pattern according to the grayscale value of each pixel in the current candidate speckle pattern.

[0073] In a specific implementation, after obtaining several candidate speckle patterns, the server can traverse these candidate speckle patterns, obtain the grayscale value of each pixel in the current candidate speckle pattern, and divide the sum of the grayscale values ​​of each pixel in the current candidate speckle pattern by the total number of pixels in the current candidate speckle pattern to calculate the grayscale value mean of the current candidate speckle pattern.

[0074] Step 402: Calculate the signal-to-noise ratio of the current candidate speckle pattern based on the speckles and non-speckles in the current candidate speckle pattern.

[0075] In one example, after calculating the grayscale value mean of the current candidate speckle pattern, the server can determine scattered speckles and non-scattered speckles in the current candidate speckle pattern, and calculate the signal-to-noise ratio of the current candidate speckle pattern by dividing the number of scattered speckles by the number of non-scattered speckles.

[0076] In an example, the server may execute step 401 first and then execute step 402, or may execute step 402 first and then execute step 401, or may execute step 401 and step 402 simultaneously.

[0077] Step 403 : Calculate the quality score of each candidate speckle pattern according to the grayscale value mean of each candidate speckle pattern, the signal-to-noise ratio of each candidate speckle pattern and a preset quality scoring algorithm.

[0078] Specifically, after calculating the grayscale mean and signal-to-noise ratio of the current candidate speckle pattern, the server can calculate the quality score of each candidate speckle pattern according to the grayscale mean, signal-to-noise ratio and preset quality scoring algorithm.

[0079] In one example, the server calculates the quality score of each candidate speckle pattern according to the grayscale value mean of each candidate speckle pattern, the signal-to-noise ratio of each candidate speckle pattern, and a preset quality scoring algorithm, which can be achieved by the following formula:

[0080] C=α*A+β*B,α+β=1

[0081] Where α and β are preset coefficients, A is the mean grayscale value of the candidate speckle pattern, B is the signal-to-noise ratio of the candidate speckle pattern, and C is the quality score of the candidate speckle pattern. α is generally set to 0.4, and β is generally set to 0.6.

[0082] In this embodiment, considering that the mean grayscale value of the candidate speckle pattern can represent the average brightness of the candidate speckle pattern, and the signal-to-noise ratio of the candidate speckle pattern can represent whether the features of the speckle pattern are obvious, the present application combines these two aspects to perform a quality score on each candidate speckle pattern, and can determine and select the candidate speckle pattern with the highest quality, the most appropriate brightness, and the most obvious features, thereby further improving the accuracy of depth recovery.

[0083] In one embodiment, the server calculates the signal-to-noise ratio of the current candidate speckle pattern based on the speckles and non-speckles in the current candidate speckle pattern, which can be calculated by: Figure 5 The steps shown in the figure are as follows:

[0084] Step 501: upsample the current candidate speckle pattern to obtain an upsampled current candidate speckle pattern.

[0085] In a specific implementation, in order to find speckles and non-speckles more clearly and accurately in the current candidate speckle pattern, the server may upsample the current candidate speckle pattern by a factor of 4, that is, magnify the current candidate speckle pattern by a factor of 4 to obtain an upsampled current candidate speckle pattern.

[0086] Step 502 : Determine the speckle region and the non-speckle region of the current candidate speckle pattern according to the upsampled current candidate speckle pattern and a preset speckle region extraction method.

[0087] In a specific implementation, the preset speckle area extraction method includes Gaussian blur, edge extraction, ellipse fitting and ellipse detection. That is, the server performs Gaussian blur, edge extraction, ellipse fitting and ellipse detection on the upsampled current candidate speckle pattern, so as to find elliptical "speckles" in the upsampled current candidate speckle pattern. The inside of the "speckle" is the speckle area, and the outside of the "speckle" is the non-speckle area.

[0088] Step 503 : Calculate the average grayscale value of the speckled area and the average grayscale value of the non-speckle area in the current candidate speckle pattern respectively.

[0089] Step 504: Calculate the ratio between the average grayscale value of the speckle area and the average grayscale value of the non-speckle area, and use the ratio as the signal-to-noise ratio of the current candidate speckle pattern.

[0090] In a specific implementation, after determining the speckle area and the non-speckle area of ​​the current candidate speckle pattern, the server can respectively calculate the average grayscale value of the speckle area and the average grayscale value of the non-speckle area in the current candidate speckle pattern, and then calculate the ratio between the average grayscale value of the speckle area and the average grayscale value of the non-speckle area, and use the ratio as the signal-to-noise ratio of the current candidate speckle pattern.

[0091] In this embodiment, considering that the speckle areas are relatively dispersed and each speckle area is relatively small, the server can upsample the current candidate speckle pattern, that is, amplify the current candidate speckle pattern, so as to more accurately detect the speckle areas of the current candidate speckle pattern, thereby improving the accuracy and reliability of the calculated signal-to-noise ratio.

[0092] Another embodiment of the present application relates to a deep recovery method. The implementation details of the deep recovery method of this embodiment are described in detail below. The following content is only for the convenience of understanding the implementation details and is not necessary for the implementation of this solution. The specific process of the deep recovery method of this embodiment can be as follows: Figure 6 Shown, including:

[0093] Step 601 : determining a target infrared region in the acquired infrared image, and determining a target speckle region in the speckle image corresponding to the infrared image according to the target infrared region.

[0094] Step 602: Generate a number of candidate speckle patterns according to the valid data bits of the target speckle area and a preset target data bit selection strategy.

[0095] Among them, steps 601 to 602 are substantially the same as steps 101 to 102 and are not described again here.

[0096] Step 603: traverse a number of candidate speckle patterns, and calculate the mean grayscale value of the current candidate speckle pattern according to the grayscale value of each pixel in the current candidate speckle pattern.

[0097] Among them, step 603 is substantially the same as step 401 and will not be described again here.

[0098] Step 604 , determining whether the mean grayscale value of the current candidate speckle pattern is within a preset grayscale value range; if so, executing step 605 ; otherwise, directly executing step 608 .

[0099] In a specific implementation, the present application sets a trigger condition before quality scoring, that is, the grayscale value mean of the candidate speckle pattern needs to be within a preset grayscale value range. Within the preset grayscale value range, it means that the brightness of the candidate speckle pattern is relatively reasonable, neither too bright nor too dark. After calculating the grayscale value mean of the current candidate speckle pattern, the server can determine whether the grayscale value mean of the current candidate speckle pattern is within the preset grayscale value range. If so, the server is allowed to continue to perform quality scoring on the current candidate speckle pattern. If the grayscale value mean of the current candidate speckle pattern is outside the preset grayscale value range, the current candidate speckle pattern is directly discarded, which further saves computing resources and can effectively improve the speed of depth recovery.

[0100] In one example, the preset grayscale value range may be [80, 150], that is, candidate speckle patterns with grayscale mean values ​​greater than or equal to 80 and less than or equal to 150 are retained, and candidate speckle patterns with grayscale mean values ​​less than 80 or greater than 150 are discarded.

[0101] Step 605: Calculate the signal-to-noise ratio of the current candidate speckle pattern based on the speckles and non-speckles in the current candidate speckle pattern.

[0102] Step 606 : Calculate the quality score of each candidate speckle pattern according to the grayscale value mean of each candidate speckle pattern, the signal-to-noise ratio of each candidate speckle pattern, and a preset quality scoring algorithm.

[0103] Among them, steps 605 to 606 are substantially the same as steps 402 to 403 and are not described again here.

[0104] Step 607: Generate a depth map corresponding to the infrared image based on the candidate speckle pattern with the highest score.

[0105] Among them, step 607 is substantially the same as step 103 and will not be described again here.

[0106] Step 608: directly discard the current candidate speckle pattern.

[0107] In one embodiment, the preset target object is a human face. The server determines the target infrared area in the acquired infrared image, and determines the target speckle area in the speckle image corresponding to the infrared image according to the target infrared area. Figure 7 The steps shown in the figure are as follows:

[0108] Step 701 : performing face detection on the acquired infrared image, determining the face outline in the infrared image, and taking the circumscribed rectangle of the face outline in the infrared image as the target infrared area.

[0109] In the specific implementation, the server first performs face detection on the acquired infrared image to determine the face contour in the infrared image. Since the face contour is not a regular shape and is not easy to operate, this application selects the circumscribed rectangle of the face contour in the infrared image as the target infrared area.

[0110] Step 702 : determining the same-named points of the vertices of the target infrared region in the speckle pattern corresponding to the infrared image according to the coordinates of the vertices of the target infrared region.

[0111] Step 703: Connect the points with the same name at the vertices of each target infrared region to obtain the target speckle region.

[0112] In a specific implementation, the target infrared area is a regular rectangle. As long as the coordinates of the four vertices are determined, the position of the rectangle in the image can be determined without determining the coordinates of each point inside the rectangle. The server determines the same-name points of each vertex of the target infrared area in the speckle pattern corresponding to the infrared image based on the coordinates of each vertex of the target infrared area. By connecting the same-name points of each vertex of the target infrared area, a rectangle is also determined in the speckle pattern corresponding to the infrared image. The rectangular area in this speckle pattern is the target speckle area.

[0113] In this embodiment, considering that the human face contour is not a regular shape, when the preset target object is a human face, the application selects the circumscribed rectangle of the human face contour in the infrared image as the target infrared area to facilitate subsequent quality evaluation and depth recovery.

[0114] The steps of the various methods above are divided only for the purpose of clear description. During implementation, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process without changing the core design of the algorithm and process are all within the scope of protection of this patent.

[0115] Another embodiment of the present application relates to a deep recovery device. The implementation details of the deep recovery device of this embodiment are described in detail below. The following content is only for the convenience of understanding the implementation details and is not necessary for the implementation of this solution. The schematic diagram of the deep recovery device of this embodiment can be as follows: Figure 8 Shown, including:

[0116] A positioning module 801 is configured to determine a target infrared region in the acquired infrared image, and determine a target speckle region in the speckle pattern corresponding to the infrared image based on the target infrared region, wherein the target infrared region includes a preset target object;

[0117] The grouping module 802 is configured to generate a plurality of candidate speckle patterns according to the valid data bits of the target speckle area and a preset target data bit selection strategy, wherein different candidate speckle patterns select different target data bits;

[0118] A scoring module 803 is configured to perform quality scoring on a plurality of candidate speckle patterns based on a preset quality scoring algorithm;

[0119] The execution module 804 is configured to generate a depth map corresponding to the infrared image according to the candidate speckle pattern with the highest score.

[0120] It is worth mentioning that all modules involved in this embodiment are logical modules. In actual applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, to highlight the innovation of this application, this embodiment does not include units that are not closely related to solving the technical problem proposed by this application. However, this does not mean that other units do not exist in this embodiment.

[0121] Another embodiment of the present application relates to an electronic device, such as Figure 9 As shown, it includes: at least one processor 901; and a memory 902 that is communicatively connected to the at least one processor 901; wherein the memory 902 stores instructions that can be executed by the at least one processor 901, and the instructions are executed by the at least one processor 901 to enable the at least one processor 901 to execute the deep recovery method in the above-mentioned embodiments.

[0122] The memory and processor are connected using a bus, which can include any number of interconnected buses and bridges. The bus connects various circuits of one or more processors and memories. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits. These are all well known in the art and are therefore not described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over a wireless medium via an antenna. Furthermore, the antenna receives data and transmits it to the processor.

[0123] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0124] Another embodiment of the present application relates to a computer-readable storage medium storing a computer program, which implements the above method embodiment when executed by a processor.

[0125] That is, those skilled in the art will understand that all or part of the steps in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a program, which is stored in a storage medium and includes a number of instructions for causing a device (which may be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0126] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and that in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A deep recovery method, characterized in that: include: Determining a target infrared area in the acquired infrared image, and determining a target speckle area in a speckle image corresponding to the infrared image according to the target infrared area; wherein the target infrared area includes a preset target object; According to the data valid bits of the target speckle area and a preset target data bit selection strategy, a plurality of candidate speckle patterns are generated; wherein different candidate speckle patterns select different target data bits; the data valid bits of the target speckle area are the data valid bits of binary data representing the grayscale value of each pixel captured by the depth camera, and the preset target data bit selection strategy is to select a plurality of consecutive data bits from the data valid bits of the grayscale value of each pixel in the target speckle area; The quality scores of the candidate speckle patterns are respectively performed based on a preset quality scoring algorithm, and a depth map corresponding to the infrared image is generated according to the candidate speckle pattern with the highest score.

2. The deep recovery method according to claim 1, characterized in that: The target speckle area has K valid data bits, where K is an integer greater than 2, the preset target data bit selection strategy includes a high-bit selection strategy and a low-bit selection strategy, and the candidate speckle patterns include a high-bit candidate speckle pattern and a low-bit candidate speckle pattern; The generating of a plurality of candidate speckle patterns according to the data valid bits of the target speckle area and a preset target data bit selection strategy includes: Obtaining a grayscale value of each pixel in the target speckle area, wherein the grayscale value is K-bit data; According to the high-bit selection strategy, the first L bits of the grayscale value of each pixel are selected as the first grayscale value of each pixel to generate a high-bit candidate speckle pattern; wherein L is an integer less than K; According to the low-bit selection strategy, the last L bits of the grayscale value of each pixel are selected as the second grayscale value of each pixel to generate a low-bit candidate speckle pattern.

3. The deep recovery method according to any one of claims 1 to 2, characterized in that: The target speckle area has 10 valid data bits, the preset target data bit selection strategy includes an upper 8-bit selection strategy, a middle 8-bit selection strategy and a lower 8-bit selection strategy, and the candidate speckle patterns include an upper 8-bit candidate speckle pattern, a middle 8-bit candidate speckle pattern and a lower 8-bit candidate speckle pattern; The generating of a plurality of candidate speckle patterns according to the data valid bits of the target speckle area and a preset target data bit selection strategy includes: Obtaining a grayscale value of each pixel in the target speckle area, wherein the grayscale value is 10-bit data; According to the high 8-bit selection strategy, the first 8 bits of the grayscale value of each pixel are selected as the third grayscale value of each pixel to generate a high 8-bit candidate speckle pattern; According to the middle 8-bit selection strategy, the second to ninth bits of the grayscale value of each pixel are selected as the fourth grayscale value of each pixel to generate a middle 8-bit candidate speckle pattern; According to the lower 8-bit selection strategy, the last 8 bits of the grayscale value of each pixel are selected as the fifth grayscale value of each pixel to generate a lower 8-bit candidate speckle pattern.

4. The deep recovery method according to any one of claims 1 to 2, characterized in that: The performing quality scoring on the plurality of candidate speckle patterns based on a preset quality scoring algorithm includes: Traversing the plurality of candidate speckle patterns, and calculating the mean grayscale value of the current candidate speckle pattern according to the grayscale value of each pixel point in the current candidate speckle pattern; Calculating a signal-to-noise ratio of the current candidate speckle pattern according to the speckles and non-speckles in the current candidate speckle pattern; The quality score of each candidate speckle pattern is calculated according to the grayscale value mean of each candidate speckle pattern, the signal-to-noise ratio of each candidate speckle pattern and a preset quality scoring algorithm.

5. The deep recovery method according to claim 4, characterized in that: The calculating the signal-to-noise ratio of the current candidate speckle pattern according to the speckles and non-speckles in the current candidate speckle pattern includes: Upsampling the current candidate speckle pattern to obtain an upsampled current candidate speckle pattern; determining a speckled area and a non-speckled area of ​​the current candidate speckle pattern according to the upsampled current candidate speckle pattern and a preset speckle area extraction method; wherein the preset speckle area extraction method includes Gaussian blur, edge extraction, ellipse fitting, and ellipse detection; respectively calculating an average grayscale value of the speckle area and an average grayscale value of the non-speckle area in the current candidate speckle image; A ratio between an average grayscale value of the speckle area and an average grayscale value of the non-speckle area is calculated, and the ratio is used as a signal-to-noise ratio of the current candidate speckle pattern.

6. The deep recovery method according to claim 4, characterized in that: The calculating the signal-to-noise ratio of the current candidate speckle pattern according to the speckles and non-speckles in the current candidate speckle pattern includes: Determining whether the grayscale value mean of the current candidate speckle pattern is within a preset grayscale value range; If the grayscale value mean of the current candidate speckle pattern is within the preset grayscale value range, then it is within the preset grayscale value range; If the mean grayscale value of the current candidate speckle pattern is outside the preset grayscale value range, the current candidate speckle pattern is directly discarded.

7. The deep recovery method according to claim 4, characterized in that: The quality score of each candidate speckle pattern is calculated according to the grayscale value mean of each candidate speckle pattern, the signal-to-noise ratio of each candidate speckle pattern and a preset quality scoring algorithm using the following formula: C=α*A+β*B,α+β=1 Wherein, α and β are preset coefficients, A is the mean grayscale value of the candidate speckle pattern, B is the signal-to-noise ratio of the candidate speckle pattern, and C is the quality score of the candidate speckle pattern.

8. The deep recovery method according to any one of claims 1 to 2, characterized in that: The preset target object is a human face, and determining a target infrared area in the acquired infrared image, and determining a target speckle area in a speckle image corresponding to the infrared image according to the target infrared area, comprises: Performing face detection on the acquired infrared image to determine the face outline in the infrared image, and using the circumscribed rectangle of the face outline as the target infrared area; determining, according to the coordinates of each vertex of the target infrared area, points of the same name as the vertices in the speckle pattern corresponding to the infrared image; The target speckle area is obtained by connecting the points of the same name at each vertex.

9. A deep recovery device, characterized in that: The device comprises: a positioning module, configured to determine a target infrared area in the acquired infrared image, and determine a target speckle area in a speckle image corresponding to the infrared image based on the target infrared area, wherein the target infrared area includes a preset target object; a grouping module, configured to generate a plurality of candidate speckle patterns based on the data valid bits of the target speckle area and a preset target data bit selection strategy, wherein different candidate speckle patterns select different target data bits; the data valid bits of the target speckle area are the data valid bits of binary data representing the grayscale value of each pixel captured by the depth camera, and the preset target data bit selection strategy is to select a plurality of consecutive data bits from the data valid bits of the grayscale value of each pixel in the target speckle area; A scoring module, configured to perform quality scoring on each of the plurality of candidate speckle patterns based on a preset quality scoring algorithm; An execution module is configured to generate a depth map corresponding to the infrared image according to the candidate speckle pattern with the highest score.

10. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the deep recovery method according to any one of claims 1 to 8.

11. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the deep recovery method according to any one of claims 1 to 8 is implemented.

Images