Depth map correction method and apparatus, computer device, and storage medium

Abstract

The application relates to a computer vision technology, in particular to a depth map correction method and device, a computer device and a storage medium. The method comprises the following steps: obtaining a depth map obtained based on structured light projected by a projector, and a projection temperature of the projector when the structured light is projected; determining a center wavelength of the structured light projected by the projector based on the projection temperature; for each pixel point in the depth map, determining a depth correction factor of a current pixel point in the depth map based on a pixel value of the current pixel point, pixel values of pixel points adjacent to the current pixel point and the center wavelength, until the depth correction factors of all pixel points in the depth map are obtained; correcting pixel values of all pixel points in the depth map according to the obtained depth correction factors; and the corrected depth map is used for living body detection. The method can shorten the living body detection time.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method, apparatus, computer device, and storage medium for correcting depth maps. Background Technology

[0002] With the continuous development of smart devices and image processing technology, more and more users are using smart devices to perform business involving sensitive personal information. Because of the sensitive personal information involved, before performing facial recognition on the user who initiates the business, a liveness detection is usually performed on the user to prevent the user from using pre-obtained images of others to pass facial recognition.

[0003] Since depth maps contain depth information, they can be used for liveness detection. In traditional liveness detection schemes, depth maps are typically acquired using structured light projected by the stabilized projector once its temperature has reached a stable state, and liveness detection is then performed based on these maps. However, the stabilization time required for the projector's temperature extends the liveness detection process. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer equipment, and storage medium for depth map correction to address the aforementioned technical problems, which can shorten the time for liveness detection.

[0005] A method for correcting a depth map, the method comprising:

[0006] Obtain a depth map based on structured light projected by a projector, and the projection temperature of the projector when projecting the structured light;

[0007] The center wavelength of the structured light projected by the projector is determined based on the projection temperature;

[0008] For each pixel in the depth map, a depth correction factor is determined based on the pixel value of the current pixel in the depth map, the pixel values ​​of the pixels adjacent to the current pixel, and the center wavelength, until the depth correction factor of each pixel in the depth map is obtained;

[0009] The pixel values ​​of each pixel in the depth map are corrected according to the obtained depth correction factor; the corrected depth map is used for liveness detection.

[0010] A depth map correction device, the device comprising:

[0011] The acquisition module is used to acquire a depth map based on the structured light projected by the projector, and the projection temperature of the projector when projecting the structured light.

[0012] The first determining module is used to determine the center wavelength of the structured light projected by the projector based on the projection temperature;

[0013] The second determining module is used to determine the depth correction factor of each pixel in the depth map based on the pixel value of the current pixel in the depth map, the pixel values ​​of the pixels adjacent to the current pixel, and the center wavelength, until the depth correction factor of each pixel in the depth map is obtained.

[0014] The correction module is used to correct the pixel values ​​of each pixel in the depth map according to the obtained depth correction factor; the corrected depth map is used for liveness detection.

[0015] In one embodiment, the acquisition module is further configured to acquire speckle structured light reflected back during the process of the projector projecting structured light onto the target object; generate a speckle structured light map based on the speckle structured light; perform speckle matching calculations on the speckle structured light map and a reference structured light map of the target object to obtain the disparity between the speckle structured light map and the reference structured light map; determine the distance between the projector and each point in the target object based on the disparity; and generate a depth map for the target object based on the distance.

[0016] In one embodiment, the parallax is the parallax between each target speckle in the speckle structured light map and a matching speckle in the reference structured light map; the distance includes at least two target distances; the at least two target distances are determined based on the parallax between at least two adjacent target speckles and the matching speckle;

[0017] The acquisition module is further configured to determine a distance difference for each target distance corresponding to each target speckle and each matched speckle; adjust the parallax between each target speckle and each matched speckle according to the distance difference; and determine the distance between the projector and the point in the target object that reflects the structured light based on the adjusted parallax.

[0018] In one embodiment, the acquisition module is further configured to determine image depth information based on the distance; perform image rendering based on the image depth information to obtain an initial depth map; and perform denoising and filtering processing on the initial depth map to obtain a depth map of the target object.

[0019] In one embodiment, the first determining module is further configured to obtain the conversion relationship between the projection temperature and the wavelength; and determine the center wavelength of the structured light projected by the projector based on the conversion relationship and the projection temperature.

[0020] In one embodiment, the device further includes:

[0021] The first detection module is used to detect the wavelength of the structured light projected by the projector at different calibration temperatures;

[0022] The fitting module is used to fit the wavelength of the structured light at different calibration temperatures to obtain a wavelength variation curve; the wavelength variation curve is used as the conversion relationship between the projection temperature and the wavelength.

[0023] In one embodiment, the fitting module is further configured to use different calibration temperatures and corresponding wavelengths as data nodes; determine the temperature step size between adjacent data nodes; determine wavelength adjustment parameters based on different data nodes, the temperature step size, and node coefficients; and construct wavelength variation curves for different data nodes according to the wavelength adjustment parameters and the calibration temperatures in different data nodes.

[0024] In one embodiment, the second determining module is further configured to perform an exponentiation operation on the difference between the pixel value of the current pixel in the depth map and the pixel value of the pixel adjacent to the current pixel to obtain at least two pixel power components; determine a correction parameter corresponding to the at least two pixel power components based on the center wavelength and a preset conversion coefficient; and determine the depth correction factor of the current pixel according to the correction parameter, the at least two pixel power components and the reference parameter.

[0025] In one embodiment, the depth map is an image obtained by the projector projecting structured light onto a target object; the apparatus further includes:

[0026] The second detection module is used to perform liveness detection on the target object based on the corrected depth map to obtain a first detection result; acquire an infrared image corresponding to the corrected depth map, and perform liveness detection on the target object based on the infrared image to obtain a second detection result; and determine that the target object is a live object based on the first detection result and the second detection result.

[0027] In one embodiment, the device further includes:

[0028] The processing module is configured to, when the target object is a living object, acquire a color image corresponding to the corrected depth map; extract image features of the target object from the color image; if the image features match image features stored in a feature library and a resource transfer operation is detected, transfer a specified amount of resources out of the target object's resource account; or, transfer the specified amount of resources into the target object's resource account.

[0029] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0030] Obtain a depth map based on structured light projected by a projector, and the projection temperature of the projector when projecting the structured light;

[0031] The center wavelength of the structured light projected by the projector is determined based on the projection temperature;

[0032] For each pixel in the depth map, a depth correction factor is determined based on the pixel value of the current pixel in the depth map, the pixel values ​​of the pixels adjacent to the current pixel, and the center wavelength, until the depth correction factor of each pixel in the depth map is obtained;

[0033] The pixel values ​​of each pixel in the depth map are corrected according to the obtained depth correction factor; the corrected depth map is used for liveness detection.

[0034] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0035] Obtain a depth map based on structured light projected by a projector, and the projection temperature of the projector when projecting the structured light;

[0036] The center wavelength of the structured light projected by the projector is determined based on the projection temperature;

[0037] For each pixel in the depth map, a depth correction factor is determined based on the pixel value of the current pixel in the depth map, the pixel values ​​of the pixels adjacent to the current pixel, and the center wavelength, until the depth correction factor of each pixel in the depth map is obtained;

[0038] The pixel values ​​of each pixel in the depth map are corrected according to the obtained depth correction factor; the corrected depth map is used for liveness detection.

[0039] The aforementioned depth map correction method, apparatus, computer equipment, and storage medium determine the center wavelength of the structured light based on the projection temperature of the projector when it projects the structured light. Then, it calculates the depth correction factor of the current pixel based on the pixel value of the current pixel, the pixel values ​​of the adjacent pixels, and the center wavelength. This yields the depth correction factor for each pixel in the depth map. The pixel values ​​of each pixel in the depth map are then corrected based on the obtained depth correction factors. This eliminates the need to wait for the projector temperature to stabilize; only the first few frames of the depth map need to be corrected. Liveness detection can then be performed using the corrected depth map, thus avoiding the waiting time during liveness detection, shortening the liveness detection time, and improving the efficiency of liveness detection. Attached Figure Description

[0040] Figure 1 This is an application environment diagram of a depth map correction method in one embodiment;

[0041] Figure 2 This is a flowchart illustrating a depth map correction method in one embodiment;

[0042] Figure 3 This is a schematic diagram of obtaining a depth map based on projected structured light in one embodiment;

[0043] Figure 4 This is a schematic diagram illustrating how wavelength changes with temperature in one embodiment;

[0044] Figure 5 This is a schematic diagram illustrating coordinate transformation between different cameras in one embodiment;

[0045] Figure 6 This is a schematic diagram of facial feature points in one embodiment;

[0046] Figure 7 This is a schematic diagram of the depth map correction architecture in one embodiment;

[0047] Figure 8 This is a flowchart illustrating a depth map correction method applied to an account registration scenario in one embodiment;

[0048] Figure 9 This is a schematic diagram of an account registration page in one embodiment;

[0049] Figure 10 This is a flowchart illustrating a depth map correction method applied to a payment scenario in one embodiment;

[0050] Figure 11 This is a schematic diagram of a payment page in one embodiment;

[0051] Figure 12This is a schematic diagram illustrating an application of the system in an access control scenario in one embodiment;

[0052] Figure 13 This is a structural block diagram of a depth map correction device in one embodiment;

[0053] Figure 14 This is a structural block diagram of a depth map correction device in another embodiment;

[0054] Figure 15 This is an internal structural diagram of a computer device in one embodiment;

[0055] Figure 16 This is a diagram of the internal structure of a computer device in another embodiment. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0057] Artificial Intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0058] The solutions provided in this application involve technologies such as computer vision and machine learning in artificial intelligence, and are specifically illustrated through the following embodiments:

[0059] The depth map correction method provided in this application can be applied to, for example... Figure 1 The application environment shown includes a terminal 102 and a server 104. The depth map correction method can be executed by the terminal 102, the server 104, or a coordinated execution by both the terminal 102 and the server 104.

[0060] Taking the depth map correction method executed by terminal 102 as an example, terminal 102 initiates liveness detection, acquires the depth map based on the structured light projected by the projector, and the projection temperature of the projector when projecting the structured light; determines the center wavelength of the structured light projected by the projector based on the projection temperature; for each pixel in the depth map, determines the depth correction factor of the current pixel based on the pixel value of the current pixel, the pixel values ​​of the pixels adjacent to the current pixel, and the center wavelength, until the depth correction factor of each pixel in the depth map is obtained; corrects the pixel values ​​of each pixel in the depth map according to the obtained depth correction factor, and then performs liveness detection based on the corrected depth map. After obtaining the detection result, if the target is alive, a color image is acquired, and face recognition is performed on the target object based on the color image. If the target object is determined to be the user, a resource transfer is initiated to server 104, such as requesting server 104 to transfer resources to the target object's resource account.

[0061] Taking the depth map correction method executed by server 104 as an example, server 104 obtains the depth map obtained based on the structured light projected by the projector in terminal 102, and the projection temperature of the projector when projecting the structured light; determines the center wavelength of the structured light projected by the projector based on the projection temperature; for each pixel in the depth map, determines the depth correction factor of the current pixel based on the pixel value of the current pixel, the pixel values ​​of the pixels adjacent to the current pixel, and the center wavelength, until the depth correction factor of each pixel in the depth map is obtained; corrects the pixel value of each pixel in the depth map according to the obtained depth correction factor, then performs liveness detection based on the corrected depth map, and then sends the detection result to terminal 102 so that terminal 102 can perform corresponding operations based on the detection result.

[0062] The terminal 102 can be a camera, smartphone, tablet, laptop, desktop computer, or smartwatch, but is not limited to these. It should be noted that the terminal 102 integrates a projector (such as a laser) and a sensor. The projector projects structured light onto the surface of the target object, and the sensor performs imaging based on the reflected structured light, then reconstructs the 3D coordinate information of the target object's surface to obtain a depth map.

[0063] Server 104 can be an independent physical server or a service node in a blockchain system. The service nodes in the blockchain system form a peer-to-peer (P2P) network. The P2P protocol is an application layer protocol that runs on top of the Transmission Control Protocol (TCP).

[0064] In addition, server 104 can also be a server cluster consisting of multiple physical servers, which can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0065] Terminal 102 and server 104 can be connected via Bluetooth, USB (Universal Serial Bus) or network, etc., and this application does not impose any restrictions.

[0066] In one embodiment, such as Figure 2 As shown, a depth map correction method is provided, which can be applied to... Figure 1 Taking terminal 102 as an example, the explanation includes the following steps:

[0067] S202, Obtain the depth map based on the structured light projected by the projector, and the projection temperature of the projector when projecting the structured light.

[0068] The projector can be a laser integrated into the terminal for projecting structured light, such as an infrared projector or a laser projector (i.e., a laser). Structured light can refer to a matrix of light arranged according to a certain structural pattern, such as structured lasers (e.g., matrix lasers) and matrix infrared light. Projection temperature can refer to the temperature of the projector when projecting structured light.

[0069] A depth map can be an image used to represent the distance between a projector and a target object. For example, the pixel value of each pixel in the depth map can serve as image depth information, representing the vertical distance between the projector and various points on the surface of the target object.

[0070] In one embodiment, when the terminal receives a liveness detection request or a resource transfer request, it projects structured light onto the target object via a projector. Then, the reflected structured light is collected by sensors in the terminal to obtain speckle structured light. This speckle structured light is then analyzed to obtain a depth map. Figure 3 As shown.

[0071] The steps described above for parsing the speckle structured light to obtain a depth map may specifically include: the terminal generating a speckle structured light map based on the speckle structured light; then performing speckle matching calculations between the speckle structured light map and a reference structured light map of the target object to obtain the disparity between the speckle structured light map and the reference structured light map, and generating a depth map for the target object based on the disparity.

[0072] Specifically, speckle matching calculation can include: the terminal extracting texture features from the speckle structured light map and the reference structured light map respectively, obtaining the first texture feature and the second texture feature in sequence, and then matching based on the first texture feature map and the second texture feature map in the speckle structured light map and the reference structured light map, and then calculating the disparity based on the matched texture features.

[0073] The generation of the depth map can specifically include: after obtaining the parallax between the speckle structured light map and the reference structured light map, the terminal obtains the camera internal parameters configured in the terminal, determines the distance between the projector and each point in the target object based on the parallax and the camera internal parameters, and then generates a depth map for the target object based on the distance.

[0074] The terminal can determine image depth information based on distance, and then perform image rendering based on this depth information to obtain a depth map. Furthermore, after image rendering, the terminal can perform denoising and filtering on the resulting depth map (which can be called the initial depth map) to obtain the depth map of the target object. This image depth information can be the pixel values ​​of the depth map.

[0075] To obtain the projection temperature, the projector collects the temperature of the structured light each time it projects the structured light, thus obtaining the projection temperature.

[0076] S204, the center wavelength of the structured light projected by the projector is determined based on the projection temperature.

[0077] The center wavelength can refer to the center point of the band where the energy of structured light is concentrated. For example, for lasers, which typically have a linewidth of tens of nanometers, if the energy of the laser is mainly concentrated in the range of 795–805 nm, then the center wavelength of the laser is 800 nm.

[0078] In one embodiment, the terminal acquires the conversion relationship between projection temperature and wavelength; based on the conversion relationship and the projection temperature, it determines the center wavelength of the structured light projected by the projector. This conversion relationship can be a curve used to fit the wavelength at different temperatures.

[0079] The specific process for obtaining the conversion relationship includes: the terminal detects the wavelength of the structured light projected by the projector at different calibration temperatures; the wavelengths at different calibration temperatures are fitted to obtain wavelength variation curves; and the wavelength variation curves are used as the conversion relationship between projection temperature and wavelength.

[0080] The calibration temperature refers to the projection temperature recorded by the projector when projecting structured light. This calibration temperature can be between 0 and 60 degrees Celsius. Figure 4 As shown.

[0081] In one embodiment, the terminal detects the wavelength of structured light projected by the projector at different calibration temperatures; fits the wavelengths at different calibration temperatures to obtain wavelength variation curves; and uses the wavelength variation curves as the conversion relationship between projection temperature and wavelength.

[0082] When the projector projects structured light at temperatures between 0 and 60 degrees Celsius, the terminal can measure the wavelength of the projected structured light using an integrated spectrometer. These wavelengths are then fitted to record the trend of wavelength variation with a calibration temperature. Figure 4 As shown. During fitting, the wavelength can be fitted using spline fitting. It should be noted that wavelength fitting refers to fitting wavelength values ​​at different temperatures.

[0083] In one embodiment, the step of fitting the wavelength of structured light at different calibration temperatures to obtain wavelength variation curves may specifically include: the terminal taking different calibration temperatures and corresponding wavelengths as data nodes; determining the temperature step size between adjacent data nodes; determining wavelength adjustment parameters based on different data nodes, temperature step sizes, and node coefficients; and constructing wavelength variation curves for different data nodes according to the wavelength adjustment parameters and the calibration temperatures in different data nodes.

[0084] The nodal coefficients can be second-order derivatives. For example... Figure 4 As shown, temperature and wavelength are used as data nodes. Assume there are n+1 data nodes (x0, y0), (x1, y1), (x2, y2), ..., (x...). n ,y n Each data node is input into the matrix equation y = a. i +b i x+c i x 2 +d i x 3 In the middle, by solving the matrix equation, the second differential value m can be obtained. i .

[0085] S206, for each pixel in the depth map, determine the depth correction factor of the current pixel based on the pixel value of the current pixel in the depth map, the pixel values ​​of the pixels adjacent to the current pixel, and the center wavelength, until the depth correction factor of each pixel in the depth map is obtained.

[0086] Here, "current pixel" can refer to the pixel currently being calculated during the process of calculating the depth correction factor for each pixel in the depth map. For example, assuming the depth map has a total of m×n (if m×n=P) pixels, when calculating the depth correction factor for the i×i (if i×i=K)th pixel, then the Kth pixel is the current pixel. Correspondingly, adjacent pixels can be pixels near the Kth pixel, such as the (i-1)×i, (i+1)×i, i×(i-1), and i×(i+1)th pixels, or other nearby pixels.

[0087] In one embodiment, the terminal inputs the pixel value of the current pixel, the pixel values ​​of the pixels adjacent to the current pixel, and the center wavelength into a depth correction function, and calculates the depth correction factor for the current pixel based on the depth correction function. The depth correction function is as follows:

[0088] S(x i )=a+b(x i -t)+c(x i -t) 2 +d(x i -t) 3

[0089] Where, x i Let represent the current pixel i in the depth map, and t represent the pixels adjacent to the current pixel (which can be one or more adjacent pixels). S i (x) represents the depth correction factor for the current pixel i in the depth map, a represents the baseline parameter, and b, c, and d are correction parameters for different power components. In addition, b = 0.3(λ-940), c = 0.13(λ-940), and d = 0.024(λ-940).

[0090] Specifically, the terminal performs an exponentiation operation on the pixel value of the current pixel in the depth map and the difference between the pixel values ​​of the pixels adjacent to the current pixel to obtain at least two pixel power components; based on the center wavelength and a preset conversion coefficient, it determines the correction parameters corresponding to the at least two pixel power components; and based on the correction parameters, the at least two pixel power components and the reference parameters, it determines the depth correction factor of the current pixel.

[0091] Here, pixel power components can refer to the results of exponentiation operations performed on the differences between the pixel values ​​of adjacent pixels of the current pixel, and can include first-order pixel power components such as (x... i -t), second-order pixel power components such as (x i -t) 2 And third-order pixel power components such as (x i -t) 3wait.

[0092] For example, assuming the correction parameters are b, c, and d, then b = 0.3(λ-940), c = 0.13(λ-940), and d = 0.024(λ-940). Then, based on these correction parameters, the different pixel power components are weighted, and the weighted result is summed with the reference parameters to obtain the depth correction factor of the current pixel.

[0093] S208, Correct the pixel values ​​of each pixel in the depth map according to the obtained depth correction factor.

[0094] The corrected depth map is used for liveness detection, such as performing liveness detection on a target object based on the corrected depth map.

[0095] In one embodiment, the terminal can weight the pixel values ​​of each pixel in the depth map according to the obtained depth correction factor, thereby obtaining a depth map composed of weighted pixel values, thus realizing the correction of the depth map.

[0096] After correcting the depth map, the terminal can perform liveness detection based on the corrected depth map to determine whether the target object is alive. Then, based on the result, it performs corresponding operations, such as facial recognition. For example, in facial payment, the terminal corrects the pixel values ​​of each pixel in the depth map of the first few frames, then performs liveness detection using the corrected depth map. If the target object is alive, it continues to identify the target object using the color image corresponding to the corrected depth map to determine if the target object is the user. If so, payment is processed. This method allows for correction of the first 5-10 frames of the depth map. Since the depth map is acquired at a rate of 12.5 frames per second, each frame takes 80ms. Correcting the first 5-10 frames improves the response time by 5×80ms to 10×80ms (i.e., 400ms to 800ms), significantly shortening the liveness detection time and improving the user experience.

[0097] In the above embodiments, the center wavelength of the structured light is determined based on the projection temperature of the projector when projecting the structured light. Then, the depth correction factor of the current pixel is calculated based on the pixel value of the current pixel in the depth map, the pixel values ​​of the pixels adjacent to the current pixel, and the center wavelength. This yields the depth correction factor for each pixel in the depth map. The pixel values ​​of each pixel in the depth map are then corrected based on the obtained depth correction factor. This eliminates the need to wait for the projector temperature to reach a stable state. Liveness detection can be performed using only the corrected depth map, thus avoiding the waiting time in the liveness detection process, shortening the liveness detection time, and improving the efficiency of liveness detection.

[0098] In one embodiment, obtaining the depth map based on the structured light projected by the projector in S202 may specifically include: during the process of the terminal projecting structured light onto the target object through the projector, collecting the reflected speckle structured light; generating a speckle structured light map based on the speckle structured light; performing speckle matching calculations on the speckle structured light map and a reference structured light map of the target object to obtain the disparity between the speckle structured light map and the reference structured light map; determining the distance between the projector and each point in the target object based on the disparity; and generating a depth map for the target object based on the distance.

[0099] Here, speckle structured light refers to structured light reflected back from the surface of a target object. A reference structured light pattern can be defined as: a structured light pattern formed by projecting structured light onto a target object from a fixed position, and then analyzing the reflected speckle structured light.

[0100] When performing speckle matching calculations on the speckle structured light map and the reference structured light map, speckle matching calculations can be performed based on local matching algorithms (Block Matching), global matching algorithms (Global Matching), or semi-global matching algorithms (Semi-Global Matching).

[0101] The specific steps for speckle matching calculation can include: the terminal can search for speckles that match the target speckle in the reference structured light image, and then determine the disparity between the speckle structured light image and the reference structured light image based on the target speckle and the matching speckle. Alternatively, the terminal can determine a matching region in the speckle structured light image to obtain the target speckle within that region, and then search for speckles that match the target speckle in the reference structured light image. This allows for the rapid identification of matching speckles in the two images, and the disparity between the speckle structured light image and the reference structured light image can then be determined based on the target speckle and the matching speckle.

[0102] The target speckle can be any speckle in the speckle structured light pattern, or it can be a speckle within a matching region of the speckle structured light pattern. The speckle corresponds to the speckle structured light; for example, when receiving a beam of structured infrared light, a speckle can be generated in the speckle structured light pattern based on that beam of infrared light.

[0103] In one embodiment, the parallax mentioned above is the parallax between each target speckle in the speckle structured light map and the matching speckle in the reference structured light map; the distance mentioned above includes at least two target distances; the at least two target distances are determined based on the parallax between at least two adjacent target speckles and the matching speckles. Therefore, the step of determining the distance between the projector and each point in the target object based on the parallax can specifically include: the terminal determining a distance difference for each target distance corresponding to each target speckle and each matching speckle; adjusting the parallax between each target speckle and each matching speckle according to the distance difference; and determining the distance between the projector and the point in the target object that reflects structured light based on the adjusted parallax. The reference structured light map can be a speckle structured light map acquired by the projector at a known distance (e.g., 0.5 meters) from the target object.

[0104] In the above embodiments, depth maps can be generated based on speckle structured light to correct the depth maps of the previous few frames and perform liveness detection, which helps improve the time efficiency of liveness detection. Furthermore, in the process of generating depth maps based on speckle structured light, error factors are considered and error compensation is performed, i.e., parallax is adjusted, which helps improve the accuracy of depth information in the depth map.

[0105] In one embodiment, the step of generating a depth map for a target object based on distance may specifically include: the terminal determining image depth information based on distance; rendering an image based on the image depth information to obtain an initial depth map; and performing denoising and filtering processing on the initial depth map to obtain a depth map of the target object.

[0106] The denoising process can remove information in the initial depth map that does not belong to the target object, treating it as noise. The filtering process can effectively filter the edges in the depth map, enhancing their sharpness. Additionally, the initial depth map can be subjected to contrast enhancement.

[0107] When a terminal has multiple depth map acquisition devices (such as cameras) and these depth maps need to be aligned, the coordinates of the target object in the first coordinate system (i.e., the coordinate system corresponding to the first camera) can be determined. Then, the coordinates in the first coordinate system can be transformed to the second coordinate system (i.e., the coordinate system corresponding to the second camera) using camera extrinsic parameters. Figure 5 As shown, this achieves alignment of different depth maps. The camera extrinsic parameters include the rotation matrix R and the translation matrix t.

[0108] In the above embodiments, by denoising and filtering the initial depth map, background noise can be effectively removed and the edges of the depth map can be enhanced, thereby making the obtained depth map more accurate and improving the accuracy of liveness detection based on the depth map.

[0109] In one embodiment, after correcting the depth map, a liveness detection operation can be performed, specifically as follows: liveness detection is performed on the target object based on the corrected depth map to obtain a first detection result; wherein, the depth map is an image obtained by projecting structured light onto the target object; an infrared image corresponding to the corrected depth map is acquired, and liveness detection is performed on the target object based on the infrared image to obtain a second detection result; based on the first detection result and the second detection result, the target object is determined to be a live object.

[0110] The target object can be a real person, a paper photograph, a digital photograph, a mannequin, or a mask. It should be noted that the depth map and the corresponding infrared image are aligned in time and space; that is, the depth map and the infrared image can be acquired at the same time, and their resolution, field of view (FOV), and number of pixels are identical.

[0111] Based on depth maps, liveness detection can determine the distance between the terminal and various points of the target object. If the target object is a live person, its surface should not be horizontal, and the distance between the terminal and various points of the target object should be different. Therefore, it can be determined whether the target object is a person in a paper or digital photograph.

[0112] Liveness detection based on infrared images can determine whether a target object emits infrared light. If the target object is a live object, it will emit infrared light, thus determining whether the target object is a human model or a mask worn by the user.

[0113] In the above embodiments, liveness detection based on depth maps and infrared images can accurately determine whether the target object is a live object, thus improving the accuracy of liveness detection.

[0114] In one embodiment, after performing liveness detection, image recognition can continue to be performed, that is, to identify whether the target object is the user himself / herself. Specifically, when the target object is a live object, the terminal acquires a color image corresponding to the corrected depth map; extracts the image features of the target object from the color image; if the image features match the image features stored in the feature library and a resource transfer operation is detected, then a specified amount of resources is transferred out of the target object's resource account; or, a specified amount of resources is transferred into the target object's resource account.

[0115] In this context, a color image can refer to an RGB (red, green, and blue) image obtained by capturing an image of a target object. When determining that the target object is a living person, the image features can be facial features or key facial points, such as at least one of the following: facial edge features, left eyebrow features, right eyebrow features, nose features, left eye features, right eye features, and lip features. Figure 6 As shown, Figure 6 In the diagram, 1-17 represent facial edge feature points, 18-22 and 23-27 represent the user's left eyebrow feature points and right eyebrow feature points respectively, 28-36 represent the user's nose feature points, 37-42 represent the user's left eye feature points, 43-48 represent the user's right eye feature points, and 49-68 represent the user's lip feature points.

[0116] Before extracting image features, the terminal can first determine the face region, and then extract the image features of the target object in that face region.

[0117] When performing image feature matching, the terminal can retrieve corresponding image features from a feature library based on the user's account. It then compares the extracted image features with those retrieved from the feature library to determine the similarity. Based on the similarity score, it determines whether the target object belongs to the user. Alternatively, the terminal can compare the extracted image features with each image feature in the feature library one by one to determine the similarity score. Based on the similarity score, it determines whether the target object is a registered user, thus identifying the target object.

[0118] In the above embodiments, when the target object is determined to be a live object by performing liveness detection based on depth map and infrared image, face recognition is performed by color image to identify whether the target object is the user. Only when the target object is the user will the resource transfer operation be performed to avoid security risks to resources.

[0119] As an example, the above method is described in conjunction with the correction architecture of the depth map, such as... Figure 7 As shown, Figure 7 This is a schematic diagram of the depth map correction architecture. We will first introduce the depth map correction architecture, and then describe the depth map correction method based on this architecture, specifically including the following:

[0120] (1) Description of the correction architecture of the depth map.

[0121] Basic principle: First, temperature calibration is performed, and the relationship between temperature and wavelength is characterized by a temperature-corrected wavelength model. Then, the depth correction factor is calculated using the center wavelength of the structured laser and the depth correction function. Finally, the depth correction factor is used to correct the depth map.

[0122] Projector: refers to a laser that can emit structured laser light in a regular pattern, used for distance measurement.

[0123] Wavelength measurement: refers to the use of a spectrometer to measure the center wavelength of a structured laser.

[0124] Temperature-corrected wavelength model: This refers to a model that characterizes the wavelength change with temperature by fitting the center wavelength of a structured laser measured at different temperatures to obtain a wavelength change curve.

[0125] Depth map correction refers to calculating a depth correction factor based on the relationship between wavelength and depth, and then correcting the depth map according to this depth correction factor.

[0126] (2) Steps for correcting the depth map.

[0127] First, wavelength measurements are performed. Under different temperature conditions (selectable from -10°C to 60°C), a spectrometer is used to measure the wavelength of the structured laser emitted by the projector, and the trend of wavelength change with temperature is recorded, such as... Figure 4 As shown.

[0128] Then, a temperature-corrected wavelength model is constructed, and the construction steps are as follows:

[0129] like Figure 4 As shown, temperature and wavelength are used as data nodes. Assume there are n+1 data nodes (x0, y0), (x1, y1), (x2, y2), ..., (x...). n ,y n (where x0, x1, x2, ..., x) n These represent different temperatures, and y0, y1, y2, ..., y n These represent the wavelengths of the structured laser projected by the projector at different temperatures.

[0130] 1. Calculate the temperature step size h i =x i+1 -x i ;

[0131] 2. Substitute the data nodes and the specified first and second endpoint conditions into the matrix equation y = a i +b i x+c i x 2 +d i x 3 ;

[0132] 3. Solve the matrix equation to obtain the second differential value m. i The coefficient matrix in this matrix equation is a tridiagonal matrix, which can be decomposed using LU decomposition into a unit lower triangular matrix and an upper triangular matrix. That is...

[0133] B = Ax = (LU)x = L(Ux) = Ly

[0134] 4. Calculate the coefficients of the matrix equation.

[0135] a i =y i

[0136]

[0137]

[0138]

[0139] Among them, a i b i c i and d i All are coefficients of the matrix equation, m i It is the second derivative value.

[0140] 5. In each subinterval x i ≤x≤x i+1 Create the following temperature-corrected wavelength model:

[0141] g(x) = a i +b i (xx i )+c i (xx i ) 2 +d i (xx i ) 3

[0142] Where x represents the projection temperature of the projector when it is currently projecting structured laser light, x i The calibrated temperature.

[0143] Finally, the center wavelength of the structured laser is calculated based on the temperature-corrected wavelength model and the projection temperature of the projector, and a depth correction factor is calculated using the depth correction function. This depth correction factor is then used to correct the depth map.

[0144] S(x i )=a+b(x i -t)+c(x i -t) 2 +d(x i -t) 3

[0145] Where, x iLet represent the current pixel i in the depth map, and t represent the pixels adjacent to the current pixel (which can be one or more adjacent pixels). S i (x) represents the depth correction factor for the current pixel i in the depth map, a represents the baseline parameter, and b, c, and d are correction parameters for different power components. In addition, b = 0.3(λ-940), c = 0.13(λ-940), and d = 0.024(λ-940).

[0146] The depth map of the first 5 to 10 frames can be corrected using the above method. Since the depth map is acquired at a rate of 12.5 frames per second, the time for each depth map frame is 80ms. By correcting the depth map of the first 5 to 10 frames, the response time can be improved by 5×80ms to 10×80ms (i.e., 400ms to 800ms), which greatly shortens the liveness detection time and improves the user experience.

[0147] To provide a more intuitive and clear understanding of the solution presented in this application, a resource account registration application scenario is also provided, which utilizes the aforementioned depth map correction method. Specifically, as... Figure 8 As shown, the application of this depth map correction method in this application scenario is as follows:

[0148] S802, Terminal detection resource account registration operation.

[0149] like Figure 9 As shown, the terminal uses... Figure 9 (b) The registration initiation page 90A displays the user information entered by the user, including the user's name, ID number and email address, and then detects the user's action of clicking the registration button 902 in real time.

[0150] S804, depth map acquired by the terminal.

[0151] Specifically, the terminal can project structured light onto the target object via a projector, then generate a depth map using sensors, and obtain the projection temperature of the projector when projecting the structured light. Furthermore, during the depth map acquisition process, the terminal can also acquire color and infrared images of the target object, such as... Figure 9 As shown in (b), a color image of the target object is displayed in the image acquisition frame 904 of the image acquisition page 90B.

[0152] S806, the terminal corrects the depth map.

[0153] Specifically, the terminal determines the center wavelength of the structured light projected by the projector based on the projection temperature; for each pixel in the depth map, the depth correction factor of the current pixel is determined based on the pixel value of the current pixel in the depth map, the pixel values ​​of the pixels adjacent to the current pixel, and the center wavelength, until the depth correction factor of each pixel in the depth map is obtained; and the pixel value of each pixel in the depth map is corrected according to the obtained depth correction factor.

[0154] S808, the terminal performs liveness detection based on the corrected depth map.

[0155] The S810 terminal uses color images for facial recognition.

[0156] The terminal extracts facial features from the color image, then compares the extracted facial features with the facial features stored in the feature library corresponding to the user information. If they match, facial recognition is passed, and then S812 is executed.

[0157] S812, the terminal sends a registration request to the server.

[0158] The registration request includes user information.

[0159] S814, the terminal receives resource account information sent by the server.

[0160] Upon receiving a registration request from the terminal, the server generates resource account information and registers the user information and resource account information with the resource account management system, while simultaneously sending the resource account information back to the terminal. After receiving the resource account information, the terminal displays the registration result and resource account information on the results display page, such as... Figure 9 As shown in (c).

[0161] In addition, the server can also perform facial recognition based on color images captured by the terminal. For example, if the terminal sends a color image and user information to the server, the server performs facial recognition on the color image and generates resource account information upon successful recognition. The server then registers the user information and resource account information with the resource account management system and simultaneously sends the resource account information back to the terminal.

[0162] In the above embodiments, the terminal uses the calibrated depth map to perform liveness detection. Only when it determines that the target object is a live object and that the target object is the user himself / herself will it initiate a registration request to the server. In this way, the server generates resource account information and feeds the resource account information back to the client. This can effectively prevent unauthorized users from impersonating legitimate users to register resource accounts, ensure the legitimate use of resource accounts, and improve the registration efficiency of resource accounts.

[0163] This application also provides a payment application scenario that applies the aforementioned depth map correction method. Specifically, as... Figure 10 As shown, the application of this depth map correction method in this application scenario is as follows:

[0164] S1002, Operation to verify product and price.

[0165] like Figure 11 As shown, the terminal uses... Figure 11 (b) The product and price display page 1102 displays the purchased product and its corresponding price, and then the user clicks the confirmation button 1104 in real time.

[0166] S1004, Depth map acquired by the terminal.

[0167] Specifically, the terminal can project structured light onto the target object via a projector, then generate a depth map using sensors, and obtain the projection temperature of the projector during structured light projection. Furthermore, during the depth map acquisition process, the terminal can also acquire color and infrared images of the target object, such as... Figure 11 As shown in (b), a color image of the target object is displayed on the image acquisition page 1106.

[0168] S1006, the terminal corrects the depth map.

[0169] Specifically, the terminal determines the center wavelength of the structured light projected by the projector based on the projection temperature; for each pixel in the depth map, the depth correction factor of the current pixel is determined based on the pixel value of the current pixel in the depth map, the pixel values ​​of the pixels adjacent to the current pixel, and the center wavelength, until the depth correction factor of each pixel in the depth map is obtained; and the pixel value of each pixel in the depth map is corrected according to the obtained depth correction factor.

[0170] S1008, the terminal performs liveness detection based on the corrected depth map.

[0171] S1010: The terminal uses color images for facial recognition.

[0172] The terminal extracts facial features from the color image, then compares the extracted facial features with the facial features stored in the feature database corresponding to the user information. If they match, facial recognition is passed, and then step S1012 is executed.

[0173] S1012, the terminal makes payments to the merchant through the server.

[0174] It should be noted that when the target is identified as the user, the payment is made to the merchant through the target's account, i.e., the funds are transferred to the merchant's account; or, the funds are transferred to a third-party platform through the target's account, and the third-party platform then transfers the funds to the merchant's account.

[0175] S1014, the terminal receives a payment success message.

[0176] After receiving the payment success message, the terminal displays the words "Payment Successful" on the result display page 1108, such as... Figure 11 As shown in (c).

[0177] In the above embodiments, the terminal uses the calibrated depth map to perform liveness detection. Only when the target object is determined to be a live object and is the user himself / herself will the payment for the product be made, thus ensuring the security of the user's funds.

[0178] This application also provides an application scenario for opening access control. In this scenario, the terminal can be an access control device integrated with an access control system. This access control device includes a data acquisition terminal (integrating a display and a camera) 120 and a liveness detection terminal (including a gate) 122. Specifically, as... Figure 12 As shown, the depth map correction method described above is applied to this application scenario as follows:

[0179] When a target object approaches the access control device that integrates the access control system, the liveness detection terminal 122 in the access control device is triggered to perform liveness detection on the target object. The data acquisition terminal 120 in the access control device displays an image acquisition page, so that the data acquisition terminal 120 can acquire the depth map of the target object through the image acquisition page. In addition, it can also acquire the color image and infrared image of the target object.

[0180] When a depth map is acquired, the data acquisition terminal 120 sends the depth map to the liveness detection terminal 122. The liveness detection terminal 122 determines the center wavelength of the structured light projected by the projector based on the projection temperature; for each pixel in the depth map, it determines the depth correction factor of the current pixel based on the pixel value of the current pixel, the pixel values ​​of the adjacent pixels, and the center wavelength, until the depth correction factor of each pixel in the depth map is obtained; and corrects the pixel value of each pixel in the depth map according to the obtained depth correction factor.

[0181] After correcting the depth map, the terminal performs liveness detection based on the corrected depth map, and then performs face recognition using the color image. The face recognition process may include: the terminal extracting facial features from the color image, and then comparing the extracted facial features with facial features stored in a feature database corresponding to the user information. If they match, the liveness detection terminal 122 opens the gate to allow the target object to enter via face recognition.

[0182] In the above embodiments, a liveness detection is performed on the target object requesting the gate to verify whether the target object is alive and its identity. The gate is opened when the target object is determined to be alive, which ensures the security of the access control system. Compared with manual verification of the target object, it saves costs and improves passage efficiency.

[0183] It should be understood that, although Figure 2 , 8 The steps in flowchart 10 are shown sequentially as indicated by the arrows; however, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise explicitly stated herein, there is no strict order in which these steps are performed, and they can be executed in other orders. Furthermore, Figure 2 , 8 At least some of the steps in 10 may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0184] In one embodiment, such as Figure 13 As shown, a depth map correction device is provided. This device can be a software module, a hardware module, or a combination of both integrated into a computer device. Specifically, the device includes: an acquisition module 1302, a first determination module 1304, a second determination module 1306, and a correction module 1308, wherein:

[0185] The acquisition module 1302 is used to acquire a depth map based on the structured light projected by the projector, and the projection temperature of the projector when projecting the structured light.

[0186] The first determining module 1304 is used to determine the center wavelength of the structured light projected by the projector based on the projection temperature;

[0187] The second determining module 1306 is used to determine the depth correction factor of each pixel in the depth map based on the pixel value of the current pixel in the depth map, the pixel values ​​of the pixels adjacent to the current pixel, and the center wavelength, until the depth correction factor of each pixel in the depth map is obtained.

[0188] The correction module 1308 is used to correct the pixel values ​​of each pixel in the depth map according to the obtained depth correction factor; the corrected depth map is used for liveness detection.

[0189] In the above embodiments, the center wavelength of the structured light is determined based on the projection temperature of the projector when projecting the structured light. Then, the depth correction factor of the current pixel is calculated based on the pixel value of the current pixel in the depth map, the pixel values ​​of the pixels adjacent to the current pixel, and the center wavelength. This yields the depth correction factor for each pixel in the depth map. The pixel values ​​of each pixel in the depth map are then corrected based on the obtained depth correction factor. This eliminates the need to wait for the projector temperature to reach a stable state. Liveness detection can be performed using only the corrected depth map, thus avoiding the waiting time in the liveness detection process, shortening the liveness detection time, and improving the efficiency of liveness detection.

[0190] In one embodiment, the acquisition module 1302 is further configured to acquire speckle structured light reflected back during the process of the projector projecting structured light onto the target object; generate a speckle structured light map based on the speckle structured light; perform speckle matching calculation on the speckle structured light map and the reference structured light map of the target object to obtain the disparity between the speckle structured light map and the reference structured light map; determine the distance between the projector and each point in the target object based on the disparity; and generate a depth map for the target object based on the distance.

[0191] In one embodiment, the parallax is the parallax between each target speckle in the speckle structured light map and a matching speckle in the reference structured light map; the distance includes at least two target distances; the at least two target distances are determined based on the parallax between at least two adjacent target speckles and the matching speckle.

[0192] The acquisition module 1302 is also used to determine the distance difference of each target distance for each target speckle and each matching speckle; adjust the parallax between each target speckle and each matching speckle according to the distance difference; and determine the distance between the projector and the point in the target object that reflects the structured light based on the adjusted parallax.

[0193] In the above embodiments, depth maps can be generated based on speckle structured light to correct the depth maps of the previous few frames and perform liveness detection, which helps improve the time efficiency of liveness detection. Furthermore, in the process of generating depth maps based on speckle structured light, error factors are considered and error compensation is performed, i.e., parallax is adjusted, which helps improve the accuracy of depth information in the depth map.

[0194] In one embodiment, the acquisition module 1302 is further configured to determine image depth information based on distance; perform image rendering based on the image depth information to obtain an initial depth map; and perform denoising and filtering processing on the initial depth map to obtain a depth map of the target object.

[0195] In the above embodiments, by denoising and filtering the initial depth map, background noise can be effectively removed and the edges of the depth map can be enhanced, thereby making the obtained depth map more accurate and improving the accuracy of liveness detection based on the depth map.

[0196] In one embodiment, the first determining module 1304 is further configured to obtain the conversion relationship between projection temperature and wavelength; and determine the center wavelength of the structured light projected by the projector based on the conversion relationship and the projection temperature.

[0197] In one embodiment, such as Figure 14 As shown, the device also includes:

[0198] The first detection module 1310 is used to detect the wavelength of structured light projected by the projector at different calibration temperatures;

[0199] The fitting module 1312 is used to fit the wavelength of structured light at different calibration temperatures to obtain a wavelength variation curve; the wavelength variation curve is used as the conversion relationship between projection temperature and wavelength.

[0200] In one embodiment, the fitting module 1312 is further configured to use different calibration temperatures and corresponding wavelengths as data nodes respectively; determine the temperature step size between adjacent data nodes; determine wavelength adjustment parameters based on different data nodes, temperature step sizes and node coefficients; and construct wavelength change curves for different data nodes according to the wavelength adjustment parameters and the calibration temperatures in different data nodes.

[0201] In one embodiment, the second determining module 1306 is further configured to perform an exponentiation operation on the difference between the pixel value of the current pixel and the pixel value of the pixel adjacent to the current pixel in the depth map to obtain at least two pixel power components; determine the correction parameter corresponding to the at least two pixel power components based on the center wavelength and a preset conversion coefficient; and determine the depth correction factor of the current pixel according to the correction parameter, the at least two pixel power components and the reference parameter.

[0202] In one embodiment, the depth map is an image obtained by projecting structured light onto a target object; such as Figure 14 As shown, the device also includes:

[0203] The second detection module 1314 is used to perform liveness detection on the target object based on the corrected depth map to obtain a first detection result; acquire an infrared image corresponding to the corrected depth map, and perform liveness detection on the target object based on the infrared image to obtain a second detection result; and determine that the target object is a live object based on the first detection result and the second detection result.

[0204] In the above embodiments, liveness detection based on depth maps and infrared images can accurately determine whether the target object is a live object, thus improving the accuracy of liveness detection.

[0205] In one embodiment, such as Figure 14 As shown, the device also includes:

[0206] The processing module 1316 is used to acquire a color image corresponding to the corrected depth map when the target object is a living object; extract image features of the target object from the color image; if the image features match the image features stored in the feature library and a resource transfer operation is detected, transfer a specified amount of resources out of the target object's resource account; or transfer a specified amount of resources into the target object's resource account.

[0207] In the above embodiments, when the target object is determined to be a live object by performing liveness detection based on depth map and infrared image, face recognition is performed by color image to identify whether the target object is the user. Only when the target object is the user will the resource transfer operation be performed to avoid security risks to resources.

[0208] Specific limitations regarding the depth map correction device can be found in the limitations of the depth map correction method described above, and will not be repeated here. Each module in the aforementioned depth map correction device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in a computer device, or stored in software in the memory of a computer device, so that the processor can call and execute the corresponding operations of each module.

[0209] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 15 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores depth maps. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a depth map correction method.

[0210] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 16As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a depth map correction method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0211] Those skilled in the art will understand that Figure 15 , 16 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0212] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0213] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0214] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the steps in the above method embodiments.

[0215] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0216] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0217] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for correcting a depth map, characterized in that, The method includes: Obtain a depth map based on structured light projected by a projector, and the projection temperature of the projector when projecting the structured light; The center wavelength of the structured light projected by the projector is determined based on the projection temperature and the conversion relationship; the conversion relationship is a wavelength variation curve obtained by fitting the wavelength of the structured light at different calibration temperatures; For each pixel in the depth map, a depth correction factor for the current pixel is determined based on the pixel value of the current pixel in the depth map, the pixel values ​​of the pixels adjacent to the current pixel, and the center wavelength. This includes: performing an exponentiation operation on the difference between the pixel value of the current pixel in the depth map and the pixel values ​​of the pixels adjacent to the current pixel to obtain at least two pixel power components; determining correction parameters corresponding to the at least two pixel power components based on the center wavelength and a preset conversion coefficient; and determining the depth correction factor for the current pixel based on the correction parameters, the at least two pixel power components, and a reference parameter, until the depth correction factor for each pixel in the depth map is obtained. The pixel values ​​of each pixel in the depth map are corrected according to the obtained depth correction factor; the corrected depth map is used for liveness detection.

2. The method according to claim 1, characterized in that, The process of obtaining a depth map based on structured light projected by a projector includes: Collect the speckle structured light reflected back during the process of the projector projecting structured light onto the target object; A speckle structured light pattern is generated based on the aforementioned speckle structured light; The speckle structured light map is compared with the reference structured light map of the target object by speckle matching calculation to obtain the disparity between the speckle structured light map and the reference structured light map; The distance between the projector and each point in the target object is determined based on the parallax. A depth map for the target object is generated based on the distance.

3. The method according to claim 2, characterized in that, The parallax is the parallax between each target speckle in the speckle structured light map and the matching speckle in the reference structured light map; the distance includes at least two target distances; the at least two target distances are determined based on the parallax between at least two adjacent target speckles and the matching speckle; Determining the distance between the projector and each point in the target object based on the parallax includes: For each target speckle and the target distance corresponding to each matched speckle, determine the distance difference for each target distance; Adjust the parallax between each target speckle and each matched speckle based on the distance difference; Based on the adjusted parallax, the distance between the projector and the point in the target object that reflects the structured light is determined.

4. The method according to claim 2, characterized in that, The step of generating a depth map for the target object based on the distance includes: Determine image depth information based on the distance; Based on the image depth information, image rendering is performed to obtain an initial depth map; The initial depth map is denoised and filtered to obtain the depth map of the target object.

5. The method according to claim 1, characterized in that, The determination of the center wavelength of the structured light projected by the projector based on the projection temperature and conversion relationship includes: Obtain the conversion relationship between the projection temperature and the wavelength; The center wavelength of the structured light projected by the projector is determined based on the conversion relationship and the projection temperature.

6. The method according to claim 5, characterized in that, The method further includes: The wavelength of structured light projected by the projector at different calibration temperatures was detected; The wavelength of the structured light at different calibration temperatures is fitted to obtain wavelength variation curves; The wavelength variation curve is used as the conversion relationship between the projection temperature and the wavelength.

7. The method according to claim 6, characterized in that, The process of fitting the wavelength of the structured light at different calibration temperatures to obtain wavelength variation curves includes: Each of the different calibration temperatures and the corresponding wavelengths is used as a data node; Determine the temperature step size between adjacent data nodes; Based on the different data nodes, the temperature step size, and the node coefficient, the wavelength adjustment parameters are determined; Based on the wavelength adjustment parameters and the calibration temperature in different data nodes, wavelength variation curves for different data nodes are constructed respectively.

8. The method according to claim 2, characterized in that, The step of performing speckle matching calculations between the speckle structured light map and the reference structured light map of the target object to obtain the disparity between the speckle structured light map and the reference structured light map includes: Texture features are extracted from the speckle structured light map and the reference structured light map of the target object to obtain the first texture feature and the second texture feature; In the speckle structured light map and the reference structured light map, matching is performed based on the first texture feature map and the second texture feature map to obtain the matched texture features; The disparity is calculated based on the matched texture features.

9. The method according to any one of claims 1 to 8, characterized in that, The depth map is an image obtained by the projector projecting structured light onto the target object; the method further includes: Based on the corrected depth map, liveness detection is performed on the target object to obtain a first detection result; An infrared image corresponding to the corrected depth map is acquired, and a liveness detection is performed on the target object based on the infrared image to obtain a second detection result; Based on the first detection result and the second detection result, the target object is determined to be a living organism.

10. The method according to claim 9, characterized in that, The method further includes: When the target object is a living body, acquire a color image corresponding to the corrected depth map; Extract image features of the target object from the color image; If the image features match the image features stored in the feature library and a resource transfer operation is detected, then a specified amount of resources is transferred out of the target object's resource account; or, the specified amount of resources is transferred into the target object's resource account.

11. A depth map correction device, characterized in that, The device includes: The acquisition module is used to acquire a depth map based on the structured light projected by the projector, and the projection temperature of the projector when projecting the structured light. The first determining module is used to determine the center wavelength of the structured light projected by the projector based on the projection temperature and the conversion relationship; the conversion relationship is a wavelength variation curve obtained by fitting the wavelength of the structured light at different calibration temperatures; The second determining module is configured to, for each pixel in the depth map, determine a depth correction factor for the current pixel based on the pixel value of the current pixel in the depth map, the pixel values ​​of the pixels adjacent to the current pixel, and the center wavelength. This includes: performing an exponentiation operation on the difference between the pixel value of the current pixel and the pixel values ​​of the pixels adjacent to the current pixel in the depth map to obtain at least two pixel power components; determining correction parameters corresponding to the at least two pixel power components based on the center wavelength and a preset conversion coefficient; and determining the depth correction factor for the current pixel based on the correction parameters, the at least two pixel power components, and a reference parameter, until the depth correction factor for each pixel in the depth map is obtained. The correction module is used to correct the pixel values ​​of each pixel in the depth map according to the obtained depth correction factor; the corrected depth map is used for liveness detection.

12. The apparatus according to claim 11, characterized in that, The acquisition module is also used to collect the speckle structured light reflected back during the process of the projector projecting structured light onto the target object; and to generate a speckle structured light map based on the speckle structured light. The speckle structured light map is compared with the reference structured light map of the target object by speckle matching calculation to obtain the disparity between the speckle structured light map and the reference structured light map; The distance between the projector and each point in the target object is determined based on the parallax; a depth map of the target object is generated based on the distance.

13. The apparatus according to claim 12, characterized in that, The parallax is the parallax between each target speckle in the speckle structured light map and the matching speckle in the reference structured light map; the distance includes at least two target distances; the at least two target distances are determined based on the parallax between at least two adjacent target speckles and the matching speckle; The acquisition module is further configured to determine a distance difference for each target distance corresponding to each target speckle and each matched speckle; and adjust the parallax between each target speckle and each matched speckle based on the distance difference. Based on the adjusted parallax, the distance between the projector and the point in the target object that reflects the structured light is determined.

14. The apparatus according to claim 12, characterized in that, The acquisition module is further configured to determine image depth information based on the distance; perform image rendering based on the image depth information to obtain an initial depth map; and perform denoising and filtering processing on the initial depth map to obtain the depth map of the target object.

15. The apparatus according to claim 11, characterized in that, The first determining module is further configured to obtain the conversion relationship between the projection temperature and the wavelength; and to determine the center wavelength of the structured light projected by the projector based on the conversion relationship and the projection temperature.

16. The apparatus according to claim 15, characterized in that, The device further includes: The first detection module is used to detect the wavelength of the structured light projected by the projector at different calibration temperatures; The fitting module is used to fit the wavelength of the structured light at different calibration temperatures to obtain a wavelength variation curve; the wavelength variation curve is used as the conversion relationship between the projection temperature and the wavelength.

17. The apparatus according to claim 16, characterized in that, The fitting module is further configured to use different calibration temperatures and corresponding wavelengths as data nodes; determine the temperature step size between adjacent data nodes; determine wavelength adjustment parameters based on different data nodes, temperature step sizes, and node coefficients; and construct wavelength variation curves for different data nodes according to the wavelength adjustment parameters and the calibration temperatures in different data nodes.

18. The apparatus according to claim 12, characterized in that, The acquisition module is further configured to extract texture features from the speckle structured light map and the reference structured light map of the target object to obtain a first texture feature and a second texture feature; In the speckle structured light map and the reference structured light map, matching is performed based on the first texture feature map and the second texture feature map to obtain the matched texture features; The disparity is calculated based on the matched texture features.

19. The apparatus according to any one of claims 11 to 18, characterized in that, The depth map is an image obtained by the projector projecting structured light onto the target object; the device further includes: The second detection module is used to perform liveness detection on the target object based on the corrected depth map to obtain a first detection result; acquire an infrared image corresponding to the corrected depth map, and perform liveness detection on the target object based on the infrared image to obtain a second detection result; and determine that the target object is a live object based on the first detection result and the second detection result.

20. The apparatus according to claim 19, characterized in that, The device further includes: The processing module is configured to, when the target object is a living object, acquire a color image corresponding to the corrected depth map; extract image features of the target object from the color image; if the image features match image features stored in a feature library and a resource transfer operation is detected, transfer a specified amount of resources out of the target object's resource account; or, transfer the specified amount of resources into the target object's resource account.

21. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 10.

22. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

23. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

Images