Image exposure method and device based on depth camera and electronic equipment

Abstract

The invention discloses an image exposure method and device based on a depth camera and electronic equipment. The method comprises the following steps: acquiring a first image acquired by the depth camera; determining a first exposure index of the first image according to the depth value of each pixel point in the first image; second exposure time of a second image collected by the depth camera is determined according to the first exposure index, the second image is the Nth frame image after the first image, and N is a positive integer.

Description

Technical Field

[0001] The present invention relates to the field of image exposure technology, and more specifically, to an image exposure method, device and electronic device based on a depth camera. Background Art

[0002] Time of flight (TOF) depth cameras emit a floodlight beam of a specific wavelength, then use sensors to receive the reflected beam of the object in the measured space and measure the flight time of the beam in space to calculate the distance, thereby acquiring an image of the measured space. TOF depth cameras can obtain grayscale images and images at the same time, and are widely used in 3D depth vision-related gesture recognition, face recognition, 3D modeling, somatosensory games, machine vision, assisted focus, security, autonomous driving and other technical fields.

[0003] As a device that can output depth maps, TOF cameras have also been widely studied. The principle of TOF cameras is to calculate the distance of the photographed object from the camera by calculating the phase difference between the emitted infrared light and the received infrared light. In order to improve the accuracy, four sine waves with different phases are usually used to correlate with the received light to calculate the depth value. The correlation operation is essentially an integral operation in the circuit, and the integration time is called the exposure time.

[0004] Currently, most TOF cameras on the market do not have the function of automatic exposure, which results in a large range of overexposure in the picture when the ambient light is abnormally strong or the imaging is close-range, causing abnormal depth; and when imaging at a long distance, the picture is underexposed, resulting in a serious decrease in the signal-to-noise ratio, or even the direct loss of depth values. Summary of the invention

[0005] An objective of embodiments of the present invention is to provide an image exposure method, device and electronic device based on a depth camera.

[0006] According to a first aspect of the present disclosure, there is provided an image exposure method based on a depth camera, comprising:

[0007] Acquire a first image captured by the depth camera;

[0008] determining a first exposure index of the first image according to a depth value of each pixel in the first image;

[0009] A second exposure time of a second image captured by the depth camera is determined according to the first exposure index, wherein the second image is an N-th frame image after the first image, and N is a positive integer.

[0010] Optionally, determining a first exposure index of the first image according to a depth value of each pixel in the first image includes:

[0011] Dividing the first image into a plurality of image blocks;

[0012] Obtaining a weight of each image block according to a position of each image block in the first image;

[0013] A first exposure index of the first image is determined according to a weight of each image block and a depth value of each pixel in each image block.

[0014] Optionally, the method further includes:

[0015] Acquire a target area in the first image that contains the photographed target object according to a depth value of each pixel in the first image;

[0016] Determining a first number of pixels belonging to the target area in each image block;

[0017] A first exposure index of the first image is also determined based on the first quantity.

[0018] Optionally, the step of determining the first exposure index includes:

[0019] Determine the sum of the depth values ​​of each pixel in each image block;

[0020] Determine a first average depth value of each image block according to the sum of the depth values ​​of each pixel in each image block and the first number of each image block;

[0021] The first exposure index is obtained by performing a weighted summation on the first average depth value of each image block according to the weight of each image block.

[0022] Optionally, when the first image is a first frame image captured by the depth camera during this operation, determining a second exposure time of a second image captured by the depth camera according to the first exposure index includes:

[0023] Acquire a first exposure time of the first image;

[0024] When the first exposure index is greater than a first maximum threshold, determining a difference between the first exposure time and a set step length as the second exposure time;

[0025] When the first exposure index is less than a first minimum threshold, the sum of the first exposure time and the set step length is determined as the second exposure time.

[0026] Optionally, when the first image is not the first frame image captured by the depth camera during this operation, determining the second exposure time of the second image captured by the depth camera according to the first exposure index includes:

[0027] When the first exposure index is less than a first minimum threshold or the first exposure index is greater than a first maximum threshold, obtaining a third exposure index and a third exposure time of a third image captured by the depth camera, where the third image is an Nth frame image before the first image;

[0028] When the first exposure index and the third exposure index are both smaller than the first minimum threshold, or the first exposure index and the third exposure index are both larger than the first maximum threshold, the second exposure time is determined according to the third exposure time, the first exposure index, and the third exposure index.

[0029] Optionally, when the first image is not the first frame image captured by the depth camera during this operation, determining the second exposure time of the second image captured by the depth camera according to the first exposure index includes:

[0030] Acquire a first exposure time of the first image;

[0031] When the first exposure index is greater than a first minimum threshold and less than a second minimum threshold, determining the second exposure time according to the first exposure time, the first exposure index, the first minimum threshold and a set target exposure index; or

[0032] When the first exposure index is less than a first maximum threshold and greater than a second maximum threshold, the second exposure time is determined according to the first exposure time, the first exposure index, the first maximum threshold and a set target exposure index.

[0033] Optionally, the method further includes:

[0034] When the second exposure time is greater than the maximum exposure time of the depth camera, updating the second exposure time to the maximum exposure time;

[0035] When the second exposure time is less than a minimum exposure time of the depth camera, the second exposure time is updated to the minimum exposure time.

[0036] According to a second aspect of the present disclosure, there is provided an image exposure device based on a depth camera, comprising:

[0037] An image acquisition module, used to acquire a first image captured by the depth camera;

[0038] an index determination module, configured to determine a first exposure index of the first image according to a depth value of each pixel in the first image;

[0039] A time determination module is used to determine a second exposure time of a second image captured by the depth camera according to the first exposure index, wherein the second image is the Nth image after the first image, and N is a positive integer.

[0040] According to a third aspect of the present disclosure, an electronic device is provided, comprising a memory and a processor, wherein the memory is used to store a computer program, and the computer program is used to control the processor to execute the method described in the first aspect of the present disclosure.

[0041] Through the embodiments of the present disclosure, the second exposure time of the second image is determined according to the first exposure index of the first image, so that the depth camera can be adjusted to the target exposure level more quickly, thereby improving the image quality. Moreover, by adjusting the exposure time in a coarse adjustment followed by a fine adjustment, the image flicker problem can be effectively avoided.

[0042] Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0044] Figure 1 is a block diagram of an example of a hardware configuration of an electronic device that can be used to implement an embodiment of the present invention;

[0045] Figure 2 is a flowchart of an image exposure method based on a depth camera provided according to an embodiment of the present disclosure;

[0046] Figure 3 is a flowchart of an example of an image exposure method based on a depth camera provided according to an embodiment of the present disclosure;

[0047] Figure 4 is a block diagram of an image exposure device based on a depth camera according to an embodiment of the present disclosure;

[0048] Figure 5 is a block diagram of an electronic device provided according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0049] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.

[0050] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0051] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0052] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0053] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0054] Figure 1 1 is a block diagram showing a hardware configuration of an electronic device 1000 that can implement an embodiment of the present disclosure.

[0055] The electronic device 1000 may be, for example, a mobile phone, a tablet computer, a camera, or other electronic product equipped with a depth camera. Figure 1 As shown, the electronic device 1000 may include a processor 1100, a memory 1200, an interface device 1300, a communication device 1400, a display device 1500, an input device 1600, a speaker 1700, a microphone 1800, and the like. Among them, the processor 1100 may be a processor CPU, a microprocessor MCU, and the like. The memory 1200 includes, for example, a ROM (read-only memory), a RAM (random access memory), a non-volatile memory such as a hard disk, and the like. The interface device 1300 includes, for example, a USB interface, a headphone interface, and the like. The communication device 1400 is, for example, capable of wired or wireless communication, and may specifically include Wifi communication, Bluetooth communication, 2G / 3G / 4G / 5G communication, and the like. The display device 1500 is, for example, a liquid crystal display screen, a touch display screen, and the like. The input device 1600 may include, for example, a touch screen, a keyboard, a somatosensory input, and the like. The user may input / output voice information through the speaker 1700 and the microphone 1800.

[0056] Figure 1The electronic device shown is merely illustrative and does not in any way imply any limitation on the present disclosure, its application or use. In the embodiments of the present disclosure, the memory 1200 of the electronic device 1000 is used to store instructions, and the instructions are used to control the processor 1100 to operate to perform any one of the methods provided in the embodiments of the present disclosure. It should be understood by those skilled in the art that although Figure 1 In the electronic device 1000, multiple devices are shown, but the present disclosure may only involve some of the devices, for example, the electronic device 1000 only involves the processor 1100 and the storage device 1200. A technician can design instructions according to the scheme disclosed in the present disclosure. How instructions control the processor to operate is well known in the art, so it will not be described in detail here.

[0057] This embodiment provides an image exposure method based on a depth camera. The image exposure method based on a depth camera can be implemented by an electronic device having a depth camera, and the electronic device can be such as Figure 1 The electronic device 1000 is shown. Figure 2 As shown, it includes steps S2100 to S2300 as shown below:

[0058] Step S2100: Acquire a first image captured by a depth camera.

[0059] In this embodiment, the depth camera may be a time of flight (TOF) depth camera.

[0060] The image captured by the depth camera includes an aligned grayscale image and a depth image. Therefore, the image captured by the depth camera records the grayscale value and depth value of each pixel.

[0061] Step S2200: determining a first exposure index of the first image according to the depth value of each pixel in the first image.

[0062] In one embodiment of the present disclosure, determining the first exposure index of the first image according to the depth value of each pixel in the first image may include steps S2211 to S2213 as shown below:

[0063] Step S2211, dividing the first image into multiple image blocks.

[0064] In this embodiment, the first image may be divided into a set number to obtain a plurality of image blocks of equal size.

[0065] The set number may be set according to the application scenario or specific requirements. If the set number is n, then the first image may be divided into n×n image blocks, and the size of each image block is a*b, that is, each image block contains a*b pixels. For example, the set n may be 5.

[0066] Step S2212: Obtain the weight of each image block according to the position of each image block in the first image.

[0067] In this embodiment, the weight of each image block may be calculated using a two-dimensional Gaussian function.

[0068] Specifically, the weight of each image block can be calculated by the following formula:

[0069]

[0070] Among them, G(i,j) represents the weight of the image block in the i-th row and j-th column, σ represents the standard deviation of the Gaussian distribution, and is generally taken as σ=(n-1) / 2, where n is the set number.

[0071] In one example, the number n is set to 5, then σ=(5-1) / 2=2. According to the Gaussian weight formula The weights of the 5*5 image blocks are calculated and shown in Table 1 below.

[0072] Table 1

[0073] 1.000 1.083 1.127 1.127 1.083 1.083 1.174 1.221 1.221 1.174 1.127 1.221 1.271 1.271 1.221 1.127 1.221 1.271 1.271 1.221 1.083 1.174 1.221 1.221 1.174

[0074] In one embodiment, when the weight of each image block is obtained, the method may further include normalizing the weight of each image block so that the sum of the weights of all image blocks in the first image is 1.

[0075] Specifically, the weight of each image block may be normalized by the following formula:

[0076]

[0077] Among them, g(i,j) represents the weight of the image block in the i-th row and j-th column after normalization.

[0078] In one example, the weights obtained after normalizing the weights in Table 1 may be as shown in Table 2.

[0079] Table 2

[0080] 0.034 0.037 0.038 0.038 0.037 0.037 0.040 0.042 0.042 0.040 0.038 0.042 0.043 0.043 0.042 0.038 0.042 0.043 0.043 0.042 0.037 0.040 0.042 0.042 0.040

[0081] Step S2213: determining a first exposure index of the first image according to the weight of each image block and the depth value of each pixel in each image block.

[0082] In this embodiment, determining a first exposure index of a first image according to a weight of each image block and a depth value of each pixel in each image block may include: determining a sum of the depth values ​​of each pixel in each image block; determining a second average depth value of each image block according to the sum of the depth values ​​of each pixel in each image block and a second number of pixels in each image block; and performing weighted summation of the second average depth value of each image block according to the weight of each image block to obtain the first exposure index.

[0083] Specifically, the first exposure index can be determined by the following formula:

[0084]

[0085] Wherein, EIR1 represents the first exposure index of the first image, IR p,q Indicates the depth value of the pixel at the pth row and qth column, m2 i,j represents the second number of image blocks in the i-th row and j-th column, the second number m2 i,j =a*b.

[0086] In one embodiment of the present disclosure, determining the first exposure index of the first image according to the depth value of each pixel in the first image may include steps S2221 to S2223 as shown below:

[0087] Step S2221: acquiring a target area in the first image that includes the photographed target object according to the depth value of each pixel in the first image.

[0088] For a depth camera, the target object to be photographed is not necessarily all objects. Sometimes, only objects within a specific range need to be photographed. For example, for passenger flow detection, assuming that the depth camera is suspended vertically, the shooting range is the ground and people on the ground, and the target object is the head and shoulders of a person. Assuming that the minimum depth value of a person's shoulder from the ground is D thresh , the height H of the depth camera from the ground, the target area can be filtered out by the following formula:

[0089] 0≤depth≤HD thresh

[0090] Among them, depth represents the depth value of the pixel in the target area.

[0091] Step S2222: divide the first image into multiple image blocks.

[0092] In this embodiment, the first image may be divided into a set number to obtain a plurality of image blocks of equal size.

[0093] The set number may be set according to the application scenario or specific requirements. If the set number is n, then the first image may be divided into n×n image blocks, and the size of each image block is a*b, that is, each image block contains a*b pixels. For example, the set n may be 5.

[0094] Step S2223, determining a first number of pixels belonging to the target area in each image block.

[0095] Step S2224: Obtain the weight of each image block according to the position of each image block in the first image.

[0096] In this embodiment, the weight of each image block may be calculated using a two-dimensional Gaussian function.

[0097] Specifically, the weight of each image block can be calculated by the following formula:

[0098]

[0099] Among them, G(i,j) represents the weight of the image block in the i-th row and j-th column, σ represents the standard deviation of the Gaussian distribution, and is generally taken as σ=(n-1) / 2, where n is the set number.

[0100] In one embodiment, when the weight of each image block is obtained, the method may further include normalizing the weight of each image block so that the sum of the weights of all image blocks in the first image is 1.

[0101] Specifically, the weight of each image block may be normalized by the following formula:

[0102]

[0103] Among them, g(i,j) represents the weight of the image block in the i-th row and j-th column after normalization.

[0104] Step S2225 , determining a first exposure index of the first image according to the weight of each image block, the depth value of each pixel in each image block, and the first number of each image block.

[0105] In this embodiment, determining a first exposure index of a first image based on a weight of each image block, a depth value of each pixel in each image block, and a first number of each image block may include: determining a sum of the depth values ​​of each pixel in each image block; determining a first average depth value of each image block based on the sum of the depth values ​​of each pixel in each image block and the first number of each image block; and performing weighted summation of the first average depth value of each image block based on the weight of each image block to obtain a first exposure index.

[0106] Specifically, the first exposure index can be determined by the following formula:

[0107]

[0108] Wherein, EIR1 represents the first exposure index of the first image, IR p,q Indicates the depth value of the pixel at the pth row and qth column, m1 i,j Represents the first number of image blocks in the i-th row and j-th column.

[0109] In an example, the size of the first image is 640*480, the number n is set to 5, each resulting size is 128*96, and the first number of each image block is shown in Table 3 below.

[0110] Table 3

[0111] 2982 0 42 248 444 4332 0 444 5846 5804 4500 0 1354 4148 6036 4658 874 3696 370 3674 4754 4418 4854 600 246

[0112] According to the first quantity shown in Table 3, the obtained first exposure index is 367.

[0113] Step S2300: determining a second exposure time of a second image captured by the depth camera according to the first exposure index, wherein the second image is the Nth frame image after the first image, and N is a positive integer.

[0114] In this embodiment, N is pre-set according to the application scenario or specific requirements. For example, N can be, but is not limited to, 1.

[0115] For example, the first avatar is the Kth frame image captured by the depth camera, and the second image is the K+Nth frame image captured by the depth camera.

[0116] In this embodiment, the first minimum threshold EIR may be preset. min1 , the second minimum threshold EIR min2 , the second maximum threshold EIR max2 , the first maximum threshold EIR max1 and set the target exposure index EIR dst , where EIR min1 <EIR min2 <EIRdst <EIR max2 <EIR max1 These values ​​can be set through experience or determined through existing image statistics.

[0117] In one example, the EIR can be set dst =1124, the first minimum threshold EIR min1 =482, the first maximum threshold EIR max1 =1766, the second minimum threshold EIR min2 =803, the second maximum threshold EIR max2 =1445.

[0118] In the case where the first image is the first frame image captured by the depth camera during this operation, determining the second exposure time of the second image captured by the depth camera according to the first exposure time and the first exposure index, including: obtaining the first exposure time of the first image; in the case where the first exposure index is greater than a first maximum threshold, determining the difference between the first exposure time and the set step length as the second exposure time; in the case where the first exposure index is less than a first minimum threshold, determining the sum of the first exposure time and the set step length as the second exposure time.

[0119] The setting step size may be set in advance according to an application scenario or specific requirements. For example, the setting step size may be 200 μs.

[0120] In one embodiment of the present disclosure, the first exposure time of the first image may be a preset value.

[0121] In another embodiment of the present disclosure, the first exposure time of the first image may be determined according to the exposure index and exposure time of the third image captured by the depth camera, wherein the third image is the Nth frame image before the first image. For example, the first avatar is the Kth frame image captured by the depth camera, and the third image is the KNth frame image captured by the depth camera.

[0122] Specifically, the method of determining the first exposure time of the first image according to the exposure index and exposure time of the third image can refer to the method of determining the second exposure time of the second image according to the first exposure index in step S2300, which is not repeated here.

[0123] In the case where the first image is not the first frame image collected by the depth camera during this operation, determining the second exposure time of the second image collected by the depth camera according to the first exposure time and the first exposure index, including: when the first exposure index is less than the first minimum threshold value, or the first exposure index is greater than the first maximum threshold value, obtaining the third exposure index and the third exposure time of the third image collected by the depth camera, the third image being the Nth frame image before the first image; when the first exposure index and the third exposure index are both less than the first minimum threshold value, or the first exposure index and the third exposure index are both greater than the first maximum threshold value, determining the second exposure time according to the first exposure time, the first exposure index, and the third exposure index.

[0124] Specifically, the second exposure time may be determined by the following formula:

[0125]

[0126] Among them, EIR1 represents the first exposure index, EIR3 represents the third exposure index, E3 represents the third exposure time, and E2 represents the second exposure time.

[0127] Through this embodiment, the second exposure index of the second image can be quickly adjusted to [EIR min1 ,EIR max1 ] range.

[0128] In one embodiment of the present disclosure, when the first image is not the first frame image captured by the depth camera during this operation, determining the second exposure time of the second image captured by the depth camera according to the first exposure time and the first exposure index includes: obtaining the first exposure time of the first image; when the first exposure index is greater than the first minimum threshold and less than the second minimum threshold, determining the second exposure time according to the first exposure time, the first exposure index, the first minimum threshold and the set target exposure index; or, when the first exposure index is less than the first maximum threshold and greater than the second maximum threshold, determining the second exposure time according to the first exposure time, the first exposure index, the first maximum threshold and the set target exposure index.

[0129] When the first exposure index is greater than the first minimum threshold and less than the second minimum threshold, the second exposure time may be determined according to the following formula:

[0130]

[0131] Where E2 represents the second exposure time, E1 represents the first exposure time, and EIR min1 Indicates the first minimum threshold EIR min1 , EIR dstIt represents the set target exposure index, and EIR1 represents the first exposure index.

[0132] When the first exposure index is less than the first maximum threshold and greater than the second maximum threshold, the second exposure time may be determined according to the following formula:

[0133]

[0134] Where E2 represents the second exposure time, E1 represents the first exposure time, and EIR min1 Indicates the first maximum threshold EIR max1 , EIR dst It represents the set target exposure index, and EIR1 represents the first exposure index.

[0135] In one embodiment of the present disclosure, when the first image is not the first frame image captured by the depth camera during this operation, determining the second exposure time of the second image captured by the depth camera according to the first exposure time and the first exposure index includes: when the first exposure index is greater than or equal to the second minimum threshold and less than or equal to the second maximum threshold, using the first exposure time as the second exposure time.

[0136] Through this embodiment, the second exposure index of the second image can be fine-tuned in small steps to [EIR min2 , EIR max2 ] range.

[0137] Through the embodiments of the present disclosure, the second exposure time of the second image is determined according to the first exposure index of the first image, so that the depth camera can be adjusted to the target exposure level more quickly, thereby improving the image quality. Moreover, by adjusting the exposure time in a coarse adjustment followed by a fine adjustment, the image flicker problem can be effectively avoided.

[0138] In one embodiment of the present disclosure, the method may further include: when the second exposure time is greater than the maximum exposure time of the depth camera, updating the second exposure time to the maximum exposure time; when the second exposure time is less than the minimum exposure time of the depth camera, updating the second exposure time to the minimum exposure time.

[0139] In this embodiment, the maximum exposure time E of the depth camera can be obtained first. max and minimum exposure time E min For example, you can set the maximum exposure time E max =6400μs, minimum exposure time E min =1000μs.

[0140] The second exposure time is greater than the maximum exposure time E of the depth camera. maxIn the case of, the second exposure time is updated to the maximum exposure time; when the second exposure time is less than the minimum exposure time E of the depth camera min In the case of, the second exposure time is updated to the minimum exposure time; when the second exposure time is less than or equal to the maximum exposure time E max , and greater than or equal to the minimum exposure time E min In this case, the second exposure time is not adjusted, and the second image is exposed according to the second exposure time.

[0141] Through this embodiment, it can be ensured that the second exposure time is within the exposure capability range of the depth camera.

[0142] Figure 3 The flowchart is an example of an image exposure method based on a depth camera provided according to an embodiment of the present disclosure.

[0143] like Figure 3 As shown, the method may include the following steps:

[0144] Step S3001: Acquire a first image captured by a depth camera.

[0145] Step S3002: acquiring a target area in the first image that includes the photographed target object according to the depth value of each pixel in the first image.

[0146] Step S3003: divide the first image into multiple image blocks.

[0147] Step S3004: determine a first number of pixels belonging to the target area in each image block.

[0148] Step S3005: Obtain the weight of each image block according to the position of each image block in the first image.

[0149] Specifically, the weight of each image block can be calculated by the following formula:

[0150]

[0151] Wherein, G(i, j) represents the weight of the image block in the i-th row and j-th column, σ represents the standard deviation of the Gaussian distribution, and is generally taken as σ=(n-1) / 2, where n is the set number.

[0152] Furthermore, the weight of each image block is normalized by the following formula so that the sum of the weights of all image blocks in the first image is 1:

[0153]

[0154] Among them, g(i, j) represents the weight of the image block in the i-th row and j-th column after normalization.

[0155] Step S3006: determining a first exposure index of the first image according to the weight of each image block, the depth value of each pixel in each image block, and the first number of each image block.

[0156] Specifically, the first exposure index can be determined by the following formula:

[0157]

[0158] Wherein, EIR1 represents the first exposure index of the first image, IR p，q Indicates the depth value of the pixel at the pth row and qth column, m1 i，j Represents the first number of image blocks in the i-th row and j-th column.

[0159] Step S3007, determining a second exposure time of a second image captured by the depth camera according to the first exposure index, wherein the second image is the Nth frame image after the first image, and N is a positive integer.

[0160] Specifically, when the first image is the nth frame image captured by the depth camera, the second exposure time can be determined by the following formula:

[0161]

[0162] E n+1 =min(E n+1 , E max )or max(E n+1 , E min )

[0163] Among them, E n+1 Indicates the second exposure time, E n Indicates the first exposure time, E n-1 Indicates the third exposure time, EIR n Indicates the first exposure index, EIR n-1 Represents the third exposure index, EIR min1 Indicates the first minimum threshold, EIR min2 Indicates the second minimum threshold, EIR max2 Indicates the second maximum threshold, EIR max1 Indicates the first maximum threshold, EIR dst Indicates the set target exposure index, E max Indicates the maximum exposure time of the depth camera, E min Indicates the minimum exposure time of the depth camera.

[0164] Furthermore, n=1 indicates that the first image is the first frame image captured by the depth camera during this operation, and n>1 indicates that the first image is not the first frame image captured by the depth camera during this operation.

[0165] This embodiment also provides an image exposure device based on a depth camera, such as Figure 4 As shown, the depth camera-based image exposure device 4000 may include an image acquisition module 4100 , an indicator determination module 4200 , and a time determination module 4300 .

[0166] The image acquisition module 4100 is used to acquire the first image captured by the depth camera.

[0167] The index determination module 4200 is used to determine a first exposure index of the first image according to a depth value of each pixel in the first image.

[0168] The time determination module 4300 is used to determine a second exposure time of a second image captured by the depth camera according to the first exposure index, wherein the second image is the Nth frame image after the first image, and N is a positive integer.

[0169] In one embodiment of the present disclosure, the indicator determination module 4200 is further used to:

[0170] Dividing the first image into a plurality of image blocks;

[0171] Obtaining a weight of each image block according to a position of each image block in the first image;

[0172] A first exposure index of the first image is determined according to a weight of each image block and a depth value of each pixel in each image block.

[0173] In one embodiment of the present disclosure, the image exposure device 4000 further includes a region acquisition module and a quantity determination module.

[0174] The region acquisition module is used to acquire a target region containing the photographed target object in the first image according to the depth value of each pixel point in the first image.

[0175] The quantity determination module is used to determine a first quantity of pixel points belonging to the target area in each image block.

[0176] The index determination module 4200 is further configured to determine a first exposure index of the first image according to the first quantity.

[0177] In one embodiment of the present disclosure, the indicator determination module 4200 is further used to:

[0178] Determine the sum of the depth values ​​of each pixel in each image block;

[0179] Determine a first average depth value of each image block according to the sum of the depth values ​​of each pixel in each image block and the first number of each image block;

[0180] The first exposure index is obtained by performing a weighted summation on the first average depth value of each image block according to the weight of each image block.

[0181] In one embodiment of the present disclosure, the time determination module 4300 is further configured to:

[0182] Acquire a first exposure time of the first image;

[0183] When the first exposure index is greater than a first maximum threshold, determining a difference between the first exposure time and a set step length as the second exposure time;

[0184] When the first exposure index is less than a first minimum threshold, the sum of the first exposure time and the set step length is determined as the second exposure time.

[0185] In one embodiment of the present disclosure, when the first image is not the first frame image captured by the depth camera during this operation, the time determination module 4300 is used to: when the first exposure index is less than a first minimum threshold value, or the first exposure index is greater than a first maximum threshold value, obtain a third exposure index and a third exposure time of a third image captured by the depth camera, where the third image is the Nth frame image before the first image; when the first exposure index and the third exposure index are both less than the first minimum threshold value, or the first exposure index and the third exposure index are both greater than the first maximum threshold value, determine the second exposure time according to the third exposure time, the first exposure index, and the third exposure index.

[0186] In one embodiment of the present disclosure, when the first image is not the first frame image captured by the depth camera during this operation, the time determination module 4300 is used to: obtain a first exposure time of the first image; when the first exposure index is greater than a first minimum threshold and less than a second minimum threshold, determine the second exposure time according to the first exposure time, the first exposure index, the first minimum threshold and a set target exposure index; or, when the first exposure index is less than a first maximum threshold and greater than a second maximum threshold, determine the second exposure time according to the first exposure time, the first exposure index, the first maximum threshold and a set target exposure index.

[0187] In one embodiment of the present disclosure, the image exposure device 4000 further includes:

[0188] A module for updating the second exposure time to the maximum exposure time when the second exposure time is greater than the maximum exposure time of the depth camera;

[0189] A module for updating the second exposure time to the minimum exposure time when the second exposure time is less than the minimum exposure time of the depth camera.

[0190] This embodiment provides an electronic device 5000, such as Figure 5 As shown, it includes a processor 5100 and a memory 5200, the memory 5200 is used to store a computer program, and the processor 5100 is used to run the computer program so that the method described in any embodiment of the present disclosure is executed.

[0191] The above embodiments mainly focus on describing the differences from other embodiments, but it should be clear to those skilled in the art that the above embodiments can be used alone or in combination with each other as needed.

[0192] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments, but it should be clear to those skilled in the art that the above embodiments can be used alone or in combination with each other as needed. In addition, for the device embodiment, since it corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the corresponding part of the method embodiment. The system embodiment described above is merely illustrative, and the modules described as separate components may or may not be physically separated.

[0193] The present invention may be an apparatus, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0194] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical compression device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0195] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0196] The computer program instructions for performing the operation of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present invention.

[0197] Various aspects of the present invention are described herein with reference to the flow charts and / or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each box of the flow chart and / or block diagram and the combination of each box in the flow chart and / or block diagram can be implemented by computer-readable program instructions.

[0198] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0199] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0200] The flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of an instruction, and the module, a program segment or a part of an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

[0201] Embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the marketplace, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present invention is defined by the appended claims.

Claims

1. An image exposure method based on a depth camera, characterized in that: include: Acquire a first image captured by the depth camera; determining a first exposure index of the first image according to a depth value of each pixel in the first image; A second exposure time of a second image captured by the depth camera is determined according to the first exposure index, wherein the second image is an N-th frame image after the first image, and N is a positive integer.

2. The method according to claim 1, characterized in that: The determining, according to the depth value of each pixel in the first image, a first exposure index of the first image includes: dividing the first image into a plurality of image blocks; Obtaining a weight of each image block according to a position of each image block in the first image; A first exposure index of the first image is determined according to a weight of each image block and a depth value of each pixel in each image block.

3. The method according to claim 2, characterized in that The method further comprises: Acquire a target area in the first image that contains the photographed target object according to a depth value of each pixel in the first image; Determining a first number of pixels belonging to the target area in each image block; A first exposure index of the first image is also determined based on the first quantity.

4. The method according to claim 3, characterized in that: The step of determining the first exposure index comprises: Determine the sum of the depth values ​​of each pixel in each image block; Determine a first average depth value of each image block according to the sum of the depth values ​​of each pixel in each image block and the first number of each image block; The first exposure index is obtained by performing a weighted summation on the first average depth value of each image block according to the weight of each image block.

5. The method according to claim 1, characterized in that In a case where the first image is a first frame image captured by the depth camera during this operation, determining a second exposure time of a second image captured by the depth camera according to the first exposure index includes: Acquire a first exposure time of the first image; When the first exposure index is greater than a first maximum threshold, determining a difference between the first exposure time and a set step length as the second exposure time; When the first exposure index is less than a first minimum threshold, the sum of the first exposure time and the set step length is determined as the second exposure time.

6. The method according to claim 1, characterized in that In a case where the first image is not a first frame image captured by the depth camera during this operation, determining a second exposure time of a second image captured by the depth camera according to the first exposure index includes: When the first exposure index is less than a first minimum threshold or the first exposure index is greater than a first maximum threshold, obtaining a third exposure index and a third exposure time of a third image captured by the depth camera, where the third image is an Nth frame image before the first image; When the first exposure index and the third exposure index are both smaller than the first minimum threshold, or the first exposure index and the third exposure index are both larger than the first maximum threshold, the second exposure time is determined according to the third exposure time, the first exposure index, and the third exposure index.

7. The method according to claim 1, characterized in that In a case where the first image is not a first frame image captured by the depth camera during this operation, determining a second exposure time of a second image captured by the depth camera according to the first exposure index includes: Acquire a first exposure time of the first image; When the first exposure index is greater than a first minimum threshold and less than a second minimum threshold, determining the second exposure time according to the first exposure time, the first exposure index, the first minimum threshold and a set target exposure index; or When the first exposure index is less than a first maximum threshold and greater than a second maximum threshold, the second exposure time is determined according to the first exposure time, the first exposure index, the first maximum threshold and a set target exposure index.

8. The method according to claim 1, characterized in that: The method further comprises: When the second exposure time is greater than the maximum exposure time of the depth camera, updating the second exposure time to the maximum exposure time; When the second exposure time is less than a minimum exposure time of the depth camera, the second exposure time is updated to the minimum exposure time.

9. An image exposure device based on a depth camera, characterized in that: include: An image acquisition module, used to acquire a first image captured by the depth camera; an index determination module, configured to determine a first exposure index of the first image according to a depth value of each pixel in the first image; A time determination module is used to determine a second exposure time of a second image captured by the depth camera according to the first exposure index, wherein the second image is the Nth frame image after the first image, and N is a positive integer.

10. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory is used to store a computer program, and the computer program is used to control the processor to execute the method according to any one of claims 1 to 8.

Images