Deep reconstruction method, system, device, and storage medium

Abstract

The application provides a depth reconstruction method, system, device and storage medium, comprising the following steps: acquiring a preset frame frequency value, continuously and sequentially collecting a background image, an infrared structured light image or an infrared image of a same target according to the frame frequency value, or continuously and sequentially collecting an infrared image, an infrared structured light image and a background image of the same target, generating a target structured light image according to the difference between the gray values of corresponding pixels of the background image and the infrared structured light image, and generating a depth image through depth reconstruction or three-dimensional reconstruction according to the target structured light image. The application can improve the depth reconstruction result of a measured target under the condition that the background light is strong, shorten the time interval between the collection of adjacent two frames of images, improve the difficulty of attack on the depth camera, and make the depth camera more secure.

Description

TECHNICAL FIELD

[0001] The present application relates to structured light three-dimensional reconstruction, in particular, to a depth reconstruction method, system, device and storage medium. BACKGROUND

[0002] Mobile payment has become a mainstream payment method in China. As of now, the number of mobile payment users has reached 1.05 billion, and the annual mobile payment transaction volume is 3 trillion yuan, becoming a pillar of the financial industry. Mobile payment has exploded with the popularity of smart phones. With the maturity of 4G / 5G, artificial intelligence, big data, and biometric technology, 2019 ushered in the face payment year representing payment 4.0. Face payment terminals are gradually expanding in offline business scenarios and are about to be used on a large scale. It is estimated that the market size will reach 185 trillion yuan in 2022.

[0003] As the core device of the face payment terminal, the face recognition camera module plays a very key role. The current mature face recognition camera module adopts a structured light scheme.

[0004] The structured light three scheme is based on the optical triangulation measurement principle. The optical projector projects a certain pattern of structured light onto the surface of an object, forming a three-dimensional image of the light bar modulated by the surface shape of the measured object on the surface. The three-dimensional image is detected by another camera at another position, thereby obtaining a two-dimensional distorted image of the light bar. The degree of distortion of the light bar depends on the relative position between the optical projector and the camera and the surface profile (height) of the object. Intuitively, the displacement (or offset) shown along the light bar is proportional to the height of the object surface, the kink represents the change of the plane, and the discontinuity shows the physical gap of the surface. When the relative position between the optical projector and the camera is fixed, the two-dimensional distorted light bar image coordinates can reproduce the three-dimensional profile of the object surface.

[0005] The depth camera module broadens the dimension of front-end perception, and can well solve the problems of anti-prosthesis attack and reduced recognition accuracy in extreme conditions encountered by 2D face recognition. The effect has been recognized by the market, and the demand is strong. It can be applied to scenarios such as door locks, access control, and payment based on 3D face recognition. When face recognition and other needs are required, not only the depth image of the target is needed, but also the grayscale image. Generally, the grayscale image is divided into two types: one is the grayscale image captured by the same camera, or the grayscale image captured by different cameras, or both. In depth reconstruction applications, the texture image collected contains the texture pattern projected by the projector and the background light. The texture pattern projected by the projector is equivalent to the effective signal, and the background light is equivalent to the noise interference. In some cases, such as when the light intensity is strong, the background light is relatively strong, and the interference is relatively serious, resulting in a low signal-to-noise ratio. SUMMARY

[0006] In view of the defects in the prior art, the present application aims to provide a depth reconstruction method, system, device and storage medium.

[0007] The depth reconstruction method provided by the present application comprises the following steps

[0008] Step S1: Obtain a preset frame frequency value, and sequentially acquire a background image, an infrared structured light image and an infrared image of a same target according to the frame frequency value, or sequentially acquire an infrared image, an infrared structured light image and a background image of a same target.

[0009] Step S2: Generate a target structured light image according to the difference between the gray values of corresponding pixels of the background image and the infrared structured light image.

[0010] Step S3: Perform depth reconstruction or three-dimensional reconstruction according to the target structured light image to generate a depth image.

[0011] Preferably, the step S1 comprises the following steps:

[0012] Step S101: Obtain a preset frame frequency threshold value and an original frame frequency value, wherein the original frame frequency value is the frame frequency value of an infrared camera used for acquiring the infrared structured light image and the infrared image.

[0013] Step S102: Determine the multiple value between the original frame frequency value and the frame frequency threshold value, and when the multiple value is less than or equal to a preset multiple threshold value, determine the preset frame frequency value as the product of the multiple threshold value and the frame frequency threshold value, and when the multiple value is greater than or equal to the preset multiple threshold value, determine the preset frame frequency value as the original frame frequency value.

[0014] Step S103: Sequentially acquire a background image, an infrared structured light image and an infrared image of a same target according to the frame frequency value within a frame frequency threshold number of sampling periods, or sequentially acquire an infrared image, an infrared structured light image and a background image of a same target.

[0015] Preferably, the step S103 comprises the following steps:

[0016] Step S1031: Project structured light and flood light to the target through the light projector of the depth camera.

[0017] Step S1032: When the multiple value is less than or equal to the preset multiple threshold value, sequentially acquire a background image, an infrared structured light image and an infrared image of a same target according to the frame frequency value within a frame frequency threshold number of sampling periods, or sequentially acquire an infrared image, an infrared structured light image and a background image of a same target.

[0018] Step S1033: When the multiple value is greater than a preset multiple threshold, sequentially capturing the background image, the infrared structured light image, the infrared image of the same target or sequentially capturing the infrared image, the infrared structured light image, the background image of the same target in any three continuous frames in each sampling period within a plurality of sampling periods determined according to a frame frequency threshold according to the frame frequency value.

[0019] Preferably, the step S2 comprises the following steps:

[0020] Step S201: determining the pixel value of each pixel in the background image and the infrared structured light image;

[0021] Step S202: pixel-level alignment of the background image and the infrared structured light image;

[0022] Step S203: subtracting the gray value of the corresponding pixel of the background image from each pixel in the infrared structured light image to generate a target structured light image.

[0023] Preferably, the step S3 comprises the following steps:

[0024] Step S301: calculating the target structured light image with known calibration information to obtain a parallax image of the target structured light image;

[0025] Step S302: determining the distance between the optical center of the infrared camera and each parallax value in the parallax image according to the principle of triangulation to generate depth information of each pixel;

[0026] Step S303: depth reconstruction or three-dimensional reconstruction according to the depth information of each pixel to generate a depth image.

[0027] Preferably, the infrared structured light image is an image containing coded texture, including any one of the following structured light images:

[0028] - speckle point image;

[0029] - stripe image;

[0030] - coded image;

[0031] - grating image.

[0032] Preferably, the preset frame frequency threshold is 15 FPS, and the preset multiple threshold is 3.

[0033] The depth reconstruction system provided by the present application is used to implement the depth reconstruction method, comprising:

[0034] An image acquisition module is configured to acquire a preset frame frequency value, and sequentially acquire a background image, an infrared structured light image, or an infrared image of a same target according to the frame frequency value, or sequentially acquire an infrared image, an infrared structured light image, and a background image of the same target.

[0035] An image enhancement module is configured to generate a target structured light image by subtracting the gray value of corresponding pixels of the background image and the infrared structured light image.

[0036] A depth reconstruction module is configured to generate a depth image by depth reconstruction or three-dimensional reconstruction according to the target structured light image.

[0037] The depth reconstruction device provided by the application comprises:

[0038] A processor;

[0039] A memory in which executable instructions of the processor are stored;

[0040] The processor is configured to execute the steps of the depth reconstruction method by executing the executable instructions.

[0041] The computer readable storage medium provided by the application is used for storing a program, and the program is executed to realize the steps of the depth reconstruction method.

[0042] Compared with the prior art, the application has the following beneficial effects:

[0043] In the application, the background image, the infrared structured light image, or the infrared image of the same target is sequentially acquired according to a preset frame frequency value, or the infrared image, the infrared structured light image, and the background image of the same target are sequentially acquired, so that the continuous acquisition of three images is realized, the time interval between the acquisition of adjacent two images is shortened, the difficulty of attack on the depth camera is improved, and the safety of the depth camera is higher.

[0044] In the application, the target structured light image is generated by subtracting the gray value of corresponding pixels of the background image and the infrared structured light image, and then the depth image is generated by depth reconstruction or three-dimensional reconstruction according to the target structured light image, so that the interference of background light is reduced, and the depth camera can be applied to an environment with high light intensity. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to make the technical solutions in the embodiments of the present application or the prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description only need to explain the embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative effort based on the provided drawings. Other features, objects and advantages of the present application will become more apparent through reading the following detailed description of the non-limiting embodiments with reference to the accompanying drawings:

[0046] Figure 1 Flow chart of steps for the depth reconstruction method in the embodiments of the present application;

[0047] Figure 2 Flow chart of steps for determining the frame frequency value in the embodiments of the present application;

[0048] Figure 3 Flow chart of steps for collecting images according to the frame frequency value in the embodiments of the present application;

[0049] Figure 4 Flow chart of steps for generating the target structured light image in the embodiments of the present application;

[0050] Figure 5 Flow chart of steps for generating the depth image by performing the depth reconstruction in the embodiments of the present application;

[0051] Figure 6 Schematic diagram of the collected images of the depth camera in the embodiments of the present application;

[0052] Figure 7 Module schematic diagram of the depth reconstruction system in the embodiments of the present application;

[0053] Figure 8 Structure schematic diagram of the depth reconstruction device in the embodiments of the present application; and

[0054] Figure 9 Structure schematic diagram of the computer readable storage medium in the embodiments of the present application. DETAILED DESCRIPTION

[0055] The present application will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made. These all belong to the protection scope of the present application.

[0056] The terms "first", "second", "third", "fourth" and the like in the description and in the claims of the present application, and above-described drawings, if any, are used to distinguish between similar objects and are not necessarily used to describe a particular sequential or chronological order. It is to be understood that the use of the terms so construed herein can be interchanged, under appropriate circumstances, and that the embodiments of the application described herein can be capable of accomplishing functionalities other than those specifically described herein without departing from the scope of the present application. Furthermore, the terms "comprise", "comprising", "include", "including", and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, product, or apparatus that comprises a list of steps or elements does not necessarily comprise only those steps or elements but can include other steps or elements not expressly listed or inherent to such process, method, product, or apparatus.

[0057] The technical solutions of the present application will be described in detail below with specific examples. The following specific examples can be combined with each other, and the same or similar concepts or processes can not be described in detail in some examples.

[0058] The depth reconstruction method provided by the present application aims to solve the problems in the prior art.

[0059] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific examples. The following specific examples can be combined with each other, and the same or similar concepts or processes can not be described in detail in some examples. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0060] Figure 1 For the step flow chart of the depth reconstruction method in the embodiments of the present application, as shown in Figure 1 The depth reconstruction method provided by the present application comprises the following steps

[0061] Step S1: obtaining a preset frame frequency value, and sequentially collecting a background image, an infrared structured light image, or an infrared image of the same target according to the frame frequency value, or sequentially collecting an infrared image, an infrared structured light image, and a background image of the same target;

[0062] Figure 2 For the step flow chart of determining the frame frequency value in the embodiments of the present application, as shown in Figure 2 The step S1 comprises the following steps:

[0063] Step S101: obtaining a preset frame frequency threshold value and an original frame frequency value, the original frame frequency value being a frame frequency value of an infrared camera used for collecting an infrared structured light image and an infrared image;

[0064] Step S102: determining a multiple value between the original frame frequency value and the frame frequency threshold value, when the multiple value is less than or equal to a preset multiple threshold value, then determining the preset frame frequency value as a product of the multiple threshold value and the frame frequency threshold value, when the multiple value is greater than or equal to the preset multiple threshold value, then determining the preset frame frequency value as the original frame frequency value;

[0065] Step S103: according to the frame frequency value, continuously and sequentially collecting the background image, the infrared structured light image, or the infrared image of the same target within the frame frequency threshold value sampling periods, or continuously and sequentially collecting the infrared image, the infrared structured light image, and the background image of the same target.

[0066] In the embodiment of the present application, the preset frame frequency threshold value is 15 FPS, the preset multiple threshold value is 3, when the original frame frequency value is 30 FPS, the multiple value is 2, since the multiple value 2 is less than the preset multiple threshold value 3, the preset frame frequency value is determined as the product of the preset multiple threshold value 3 and the preset frame frequency threshold value 15, and the frame frequency value is determined as 45; when the original frame frequency value is 60 FPS, the multiple value is 4, since the multiple value 4 is greater than the preset multiple threshold value 3, the preset frame frequency value is determined as the original frame frequency value 60 FPS.

[0067] Figure 3 The step flow chart of collecting the image according to the frame frequency value in the embodiment of the present application is shown in the following figure: Figure 3 As shown in the figure, the step S103 includes the following steps:

[0068] Step S1031: projecting the structured light and the flood light to the target through the light projector of the depth camera;

[0069] Step S1032: when the multiple value is less than or equal to the preset multiple threshold value, according to the frame frequency value, continuously and sequentially collecting the background image, the infrared structured light image, or the infrared image of the same target within the frame frequency threshold value sampling periods, or continuously and sequentially collecting the infrared image, the infrared structured light image, and the background image of the same target;

[0070] Step S1033: when the multiple value is greater than the preset multiple threshold value, according to the frame frequency value, continuously and sequentially collecting the background image, the infrared structured light image, or the infrared image of the same target in any three consecutive frames in each sampling period within the multiple sampling periods determined according to the frame frequency threshold value, or continuously and sequentially collecting the infrared image, the infrared structured light image, and the background image of the same target.

[0071] In the embodiment of the present application, when the multiple value is 2, and the multiple value is less than or equal to the preset multiple threshold value, then the background image, the infrared structured light image, the infrared image of the same target are sequentially collected in the frame frequency threshold value 15 sampling periods according to the frame frequency value 45, or the infrared image, the infrared structured light image, the background image of the same target are sequentially collected.

[0072] By analogy, for example, when the multiple value is 4, and the multiple value is greater than the preset multiple threshold value, then the background image, the infrared structured light image, the infrared image of the same target are sequentially collected every 4 frames in the frame frequency threshold value 15 sampling periods according to the frame frequency value 60, or the infrared image, the infrared structured light image, the background image of the same target are sequentially collected.

[0073] In the embodiment of the present application, the infrared structured light image is an image containing coded texture, including any one of the following structured light images:

[0074] - speckle image;

[0075] - stripe image;

[0076] - coded image;

[0077] - grating image.

[0078] Step S2: generating a target structured light image according to the difference between the gray values of the corresponding pixels of the background image and the infrared structured light image;

[0079] Figure 4 The step flow chart for generating a target structured light image in the embodiment of the present application is shown in Figure 4 The step S2 includes the following steps:

[0080] Step S201: determining the gray value of each pixel in the background image and the infrared structured light image;

[0081] Step S202: aligning the background image and the infrared structured light image at the pixel level;

[0082] Step S203: subtracting the gray value of the corresponding pixel of the background image from each pixel in the infrared structured light image to generate a target structured light image.

[0083] Step S3: generating a depth image according to the depth reconstruction or three-dimensional reconstruction of the target structured light image.

[0084] Figure 5 The step flow chart for generating a depth image according to the depth reconstruction in the embodiment of the present application is shown in Figure 5 The step S3 includes the following steps:

[0085] Step S301: calculating the target structured light image and known calibration information to obtain a parallax image of the target structured light image;

[0086] Step S302: determining the distance between the optical center of the infrared camera and each parallax value in the parallax image according to the principle of triangulation to generate depth information of each pixel;

[0087] Step S303: generating a depth image according to the depth information of each pixel. Figure 6 The schematic diagram of the image collected by the depth camera in the embodiment of the present application is shown in Figure 6 As shown, when the depth camera provided by the present application is used, the background image can be first collected by the infrared camera, then the infrared structured light image is collected after the structured light is projected to the target by the structured light projector, and finally the infrared image is collected after the floodlight is projected to the target by the floodlight projector. The infrared camera uses a 940nm infrared camera. The floodlight projector uses an LED light source.

[0088] Figure 7 The module schematic diagram of the depth reconstruction system in the embodiment of the present application is shown in Figure 7 The depth reconstruction system provided by the present application is used to realize the depth reconstruction method, and comprises:

[0089] The image acquisition module is used to obtain a preset frame frequency value, and sequentially collect the background image, the infrared structured light image and the infrared image of the same target according to the frame frequency value, or sequentially collect the infrared image, the infrared structured light image and the background image of the same target.

[0090] The image enhancement module is used to generate a target structured light image by subtracting the gray value of the corresponding pixels of the background image and the infrared structured light image.

[0091] The depth reconstruction module is used to generate a depth image by depth reconstruction or three-dimensional reconstruction according to the target structured light image.

[0092] The depth reconstruction device in the embodiment of the present application comprises a processor and a memory having executable instructions of the processor. The processor is configured to execute the steps of the depth reconstruction method by executing the executable instructions.

[0093] As above, the embodiment can continuously and sequentially collect the background image, the infrared structured light image, the infrared image of the same target according to the preset frame frequency value, or continuously and sequentially collect the infrared image, the infrared structured light image, the background image of the same target, realize the continuous collection of three frames of images, shorten the time interval of the collection between adjacent two frames of images, improve the difficulty of the attack on the depth camera, and make the security of the depth camera higher.

[0094] Those skilled in the art can understand that each aspect of the present application can be implemented as a system, a method or a program product. Therefore, each aspect of the present application can be specifically implemented as follows: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "platform" here.

[0095] Figure 8 is a structural schematic diagram of the depth reconstruction device in the embodiment of the present application. The electronic device 600 according to this embodiment of the present application will be described below with reference to Figure 8 . Figure 8 The electronic device 600 shown is merely an example, and should not bring any limitation to the functions and use range of the embodiment of the present application.

[0096] As shown in Figure 8 , the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 can include but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0097] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 executes the steps according to various exemplary embodiments of the present application described in the depth reconstruction method part of the present specification. For example, the processing unit 610 can execute the steps as shown in Figure 1 .

[0098] The storage unit 620 can include a readable medium in the form of a volatile storage unit, such as a random access memory (RAM) 6201 and / or a cache memory unit 6202, and can further include a read-only memory (ROM) 6203.

[0099] The storage unit 620 can further include a program / utility 6204 having a set of (at least one) program modules 6205, which include but are not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination thereof can include implementation of a network environment.

[0100] Bus 630 can be one or more of several types of bus structures including a memory bus or memory controller, a peripheral bus, a graphics bus, a processor or local bus using any of a variety of bus architectures.

[0101] Electronic device 600 can also communicate with one or more external devices 700 such as a keyboard or pointing device, a Bluetooth device, etc.; other devices that enable a user to interact with electronic device 600; and / or any devices (e.g., a router, a modem, a printer, etc.) that enable electronic device 600 to communicate with one or more other computing devices. Such communication can occur via Input / Output (I / O) interface 650. Still yet, electronic device 600 can communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g., the Internet) via network adapter 660. As depicted, network adapter 660 can communicate with the other components of electronic device 600 via bus 630 although it is understood that many other architectures can be implemented to enable communication between network adapter 660 and other components of electronic device 600. Figure 8 It should be appreciated that the software modules described herein can include one or more instructions that, when executed by a processor, can cause the processor to perform one or more steps described herein. These software modules can be implemented in a variety of ways. For example, software modules can be implemented in hardware, software, or a combination of both. In some embodiments, software modules can be implemented in software, which can be stored in memory and executed by a processor. In some embodiments, software modules can be implemented in a high-level programming language, such as C, using an interpreter. In some embodiments, software modules can be implemented in assembly or machine language using an assembler or compiler. In some embodiments, software modules can be stored in memory 620 and / or memory 640. In some embodiments, software modules can be stored on a computer readable medium, such as a floppy disk, a hard drive, a CD, a DVD, a memory stick, a memory card, a RAM, a ROM, or a flash drive. In some embodiments, software modules can be downloaded into electronic device 600 over the Internet or another network.

[0102] The embodiment also provides a computer readable storage medium for storing a program, the program being executed to implement the steps of the depth reconstruction method. In some possible implementation manners, various aspects of the present application can also be implemented in the form of a program product, which includes program codes for causing terminal equipment to perform the steps described in the depth reconstruction method part of the present specification according to various exemplary embodiments of the present application when the program product is run on the terminal equipment.

[0103] As shown above, the program of the computer readable storage medium of the embodiment, when executed, continuously and sequentially acquires the background image, the infrared structured light image, or the infrared image of the same target according to the pre-set frame frequency value, or continuously and sequentially acquires the infrared image, the infrared structured light image, and the background image of the same target, to realize the continuous acquisition of three frames of images, shorten the time interval of the acquisition between adjacent two frames of images, improve the difficulty of the attack on the depth camera, and make the security of the depth camera higher.

[0104] Figure 9 is a structural schematic diagram of the computer readable storage medium in the embodiment of the present application. Referring to Figure 9As shown, a program product 800 for implementing the above-described method according to an embodiment of the present application is described, which can take the form of a portable compact disc read-only memory (CD-ROM) and includes a program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present application is not limited thereto, and in the present document, the readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device.

[0105] The program product can take any combination of one or more readable media. The readable media can be a readable signal medium or a readable storage medium. The readable storage medium, for example, can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0106] The computer readable storage medium can include a data signal transported, propagated or transmitted, in baseband or as part of a carrier wave, in which readable program code is embodied. Such a propagated signal can take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. The readable storage medium can also be any readable medium that is capable of storing, transmitting or transferring a program for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the readable storage medium can be transmitted using any suitable medium, including, but not limited to, wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the above.

[0107] The program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++, etc., or conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider.

[0108] The embodiment of the present application realizes the continuous acquisition of three frames of images by continuously and sequentially acquiring the background image, the infrared structured light image, the infrared image of the same target according to the preset frame frequency value, or continuously and sequentially acquiring the infrared image, the infrared structured light image and the background image of the same target, shortens the time interval of the acquisition between adjacent two frames of images, improves the difficulty of the attack on the depth camera, and makes the security of the depth camera higher; in the embodiment of the present application, the target structured light image is generated by subtracting the gray value of the corresponding pixels of the background image and the infrared structured light image, and then the depth image is generated by depth reconstruction or three-dimensional reconstruction according to the target structured light image, which reduces the interference of the background light and makes the depth camera applicable to the environment with strong light intensity.

[0109] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts of each embodiment can be referred to each other. The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0110] The specific embodiments of the present application are described above. It should be understood that the present application is not limited to the above specific embodiments, and various modifications or changes can be made by those skilled in the art within the scope of the claims, which does not affect the essential content of the present application.

Claims

1. A depth reconstruction method, characterized in that, The method comprises the following steps: Step S1: obtaining a preset frame frequency value, and continuously and sequentially collecting a background image, an infrared structured light image, or an infrared image of a same target according to the frame frequency value, or continuously and sequentially collecting an infrared image, an infrared structured light image, or a background image of a same target; Step S2: generating a target structured light image according to a difference between gray values of corresponding pixels of the background image and the infrared structured light image; Step S3: generating a depth image according to depth reconstruction or three-dimensional reconstruction of the target structured light image; The step S1 comprises the following steps: Step S101: obtaining a preset frame frequency threshold value and an original frame frequency value, the original frame frequency value being a frame frequency value of an infrared camera used for collecting an infrared structured light image and an infrared image; Step S102: determining a multiple value between the original frame frequency value and the frame frequency threshold value, when the multiple value is less than or equal to a preset multiple threshold value, determining the preset frame frequency value as a product of the multiple threshold value and the frame frequency threshold value, and when the multiple value is greater than or equal to the preset multiple threshold value, determining the preset frame frequency value as the original frame frequency value; Step S103: continuously and sequentially collecting a background image, an infrared structured light image, or an infrared image of a same target according to the preset frame frequency value within a frame frequency threshold value number of sampling periods, or continuously and sequentially collecting an infrared image, an infrared structured light image, or a background image of a same target.

2. The depth reconstruction method of claim 1, wherein, The step S103 comprises the following steps: Step S1031: projecting structured light and flood light to a target through a light projector of a depth camera; Step S1032: when the multiple value is less than or equal to the preset multiple threshold value, continuously and sequentially collecting a background image, an infrared structured light image, or an infrared image of a same target according to the preset frame frequency value within a frame frequency threshold value number of sampling periods, or continuously and sequentially collecting an infrared image, an infrared structured light image, or a background image of a same target; Step S1033: when the multiple value is greater than the preset multiple threshold value, continuously and sequentially collecting a background image, an infrared structured light image, or an infrared image of a same target according to the preset frame frequency value in any three consecutive frames in each sampling period within a plurality of sampling periods determined according to the frame frequency threshold value, or continuously and sequentially collecting an infrared image, an infrared structured light image, or a background image of a same target.

3. The depth reconstruction method of claim 1, wherein, The step S2 comprises the following steps: Step S201: determining a pixel value of each pixel in the background image and the infrared structured light image; Step S202: performing pixel-level alignment on the background image and the infrared structured light image; Step S203: generating a target structured light image by subtracting a gray value of a corresponding pixel of the background image from each pixel in the infrared structured light image.

4. The depth reconstruction method of claim 1, wherein, The step S3 comprises the following steps: Step S301: calculating a parallax image of the target structured light image according to known calibration information; Step S302: determining a distance between an optical center of the infrared camera and each parallax value in the parallax image according to a triangulation principle, and generating depth information of each pixel; Step S303: generating a depth image according to depth reconstruction or three-dimensional reconstruction of each pixel.

5. The depth reconstruction method of claim 1, wherein, The infrared structured light image is an image containing coded textures, including any one of the following structured light images: - a speckle image; - a stripe image; - a coded image; - a grating image.

6. The depth reconstruction method of claim 1, wherein, The preset frame frequency threshold is 15 FPS, and the preset multiple threshold is 3.

7. A depth reconstruction system for implementing the depth reconstruction method of any one of claims 1 to 6, characterized by The method comprises the steps of: an image acquisition module, configured to acquire a preset frame frequency value, and sequentially acquire a background image, an infrared structured light image, or an infrared image of a same target according to the frame frequency value, or sequentially acquire an infrared image, an infrared structured light image, and a background image of the same target; an image enhancement module, configured to generate a target structured light image by subtracting the gray value of corresponding pixels of the background image and the infrared structured light image; a depth reconstruction module, configured to perform depth reconstruction or three-dimensional reconstruction according to the target structured light image to generate a depth image.

8. A deep reconstruction device, characterized by, The method comprises the steps of: a processor; a memory, in which executable instructions of the processor are stored; wherein the processor is configured to execute the steps of the depth reconstruction method according to any one of claims 1 to 6 by executing the executable instructions.

9. A computer readable storage medium for storing a program, characterized in that, The program is executed to implement the steps of the depth reconstruction method according to any one of claims 1 to 6.

Images