Method, apparatus, storage medium, and electronic device for determining brightness

Abstract

An embodiment of the present invention provides a method, apparatus, storage medium, and electronic device for determining brightness. The method includes: analyzing a first infrared image obtained by a first imaging device to determine a first moving speed of a target object, where the first infrared image is an image including the target object obtained after a target projector projects a target pattern onto the target object at a first distance from the target device; determining a target distance between the target object and the target device when calculating the distance between the target object and the target device for the current time based on the first moving speed and a first delay time; determining a target brightness when the target projector projects the target pattern onto the target object based on a target mapping table, the current ambient infrared brightness, and the target distance. By using the method of the present invention, the problem that the projection intensity of structured light cannot be adaptively adjusted according to diverse application scenarios in the related art is solved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computers, and more particularly, to a method, apparatus, storage medium, and electronic device for determining brightness. Background Art

[0002] With the rapid development of digital technology and image processing technology, face recognition technology has been widely used. For example, face unlocking functions on mobile phones, face payment devices in supermarkets and on transportation vehicles, and so on.

[0003] In the related art, in the current structured light projection field, structured light projection usually projects an optical pattern (for example, optical information in the form of dots, lines, or stripes with a wavelength of about 940 nm) onto a detection object, thereby imposing optical feature information on the detection object. During the projection process, in order to ensure recognition accuracy, it is necessary to project with a relatively high intensity and a large amount of light. However, since the detection object may be damaged under high-intensity structured light projection, especially when the optical pattern needs to be projected onto the facial area of the detection object, high-intensity projection will cause certain damage to people's eyes. In addition, in the related art, a common infrared camera is usually used to collect the structured light speckles after projecting onto the detection object. This method is only applicable to indoor environments, and in outdoor environments, the quality of the collected structured light speckles is poor.

[0004] Regarding the problem that the projection intensity of structured light in the related art cannot be adaptively adjusted according to diverse application scenarios, no effective solution has been proposed yet. Summary of the Invention

[0005] Embodiments of the present invention provide a method, apparatus, storage medium, and electronic device for determining brightness, so as to at least solve the problem that the projection intensity of structured light in the related art cannot be adaptively adjusted according to diverse application scenarios.

[0006] According to an embodiment of the present invention, a method for determining brightness is provided, including: parsing a first infrared image obtained by a first imaging device to determine a first moving speed of a target object, where the first infrared image is an image including the target object obtained after a target projector projects a target pattern onto the target object at a first distance from a target device, and the target device includes the first imaging device and the target projector; determining a target distance between the target object and the target device when calculating the distance between the target object and the target device for the current time based on the first moving speed and a first delay time, where the first delay time is the time required to calculate the distance between the target object and the target device; determining a target brightness when the target projector projects the target pattern onto the target object based on a target mapping table, the current ambient infrared brightness, and the target distance, where the target mapping table includes a pre-determined correspondence between each ambient infrared brightness and the brightness respectively projected by the target projector when projecting the target pattern onto a reference object at a preset distance from the target device.

[0007] According to another embodiment of the present invention, a device for determining brightness is provided, including: a first determination module configured to parse a first infrared image obtained by a first imaging device to determine a first moving speed of a target object, where the first infrared image is an image including the target object obtained after a target projector projects a target pattern onto the target object at a first distance from a target device, and the target device includes the first imaging device and the target projector; a second determination module configured to determine a target distance between the target object and the target device when calculating the distance between the target object and the target device for the current time based on the first moving speed and a first delay time, where the first delay time is the time required to calculate the distance between the target object and the target device; a third determination module configured to determine a target brightness when the target projector projects the target pattern onto the target object based on a target mapping table, the current ambient infrared brightness, and the target distance, where the target mapping table includes a pre-determined correspondence between each ambient infrared brightness and the brightness respectively projected by the target projector when projecting the target pattern onto a reference object at a preset distance from the target device.

[0008] According to still another embodiment of the present invention, a computer-readable storage medium is further provided, where a computer program is stored in the computer-readable storage medium, and the computer program is configured to execute the steps in any one of the above method embodiments when running.

[0009] According to another embodiment of the present invention, an electronic device is further provided, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0010] Through the present invention, by analyzing the first infrared image including the target object obtained by the first imaging device after the target projector projects a target pattern onto the target object at a first distance from the target device, the first moving speed of the target object is determined. Then, based on the first moving speed and the first delay time, when calculating the distance between the target object and the target device for the current time, the target distance between the target object and the target device is determined. Thus, further based on the target mapping table, the current ambient infrared brightness, and the target distance, the target brightness when the target projector projects the target pattern onto the target object can be determined, ensuring that the brightness projected by the target pattern projector can meet diverse requirements, improving the quality of pattern projection, solving the problem in the related art that the projection intensity of structured light cannot be adaptively adjusted according to diverse application scenarios, and thus achieving the effect of being able to adaptively adjust the projection intensity of structured light according to diverse application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a hardware structure block diagram of a mobile terminal for a method of determining brightness according to an embodiment of the present invention;

[0012] Figure 2 is a flowchart of a method of determining brightness according to an embodiment of the present invention;

[0013] Figure 3 is a flowchart of a method of determining the brightness of a pattern projector according to a specific embodiment of the present invention;

[0014] Figure 4 is an example diagram of a mapping table of ambient infrared brightness - projector brightness according to a specific embodiment of the present invention;

[0015] Figure 5 is a schematic diagram of a pattern projector projecting a pattern according to a specific embodiment of the present invention;

[0016] Figure 6 is a schematic diagram of the pattern projection effect in an indoor completely dark scene according to a specific embodiment of the present invention;

[0017] Figure 7 is a schematic diagram of the pattern projection effect when there is a large amount of background noise outdoors according to a specific embodiment of the present invention;

[0018] Figure 8It is a flowchart for generating a mapping table of ambient infrared brightness - projector brightness according to a specific embodiment of the present invention;

[0019] Figure 9 It is a processing flowchart of a color camera according to a specific embodiment of the present invention;

[0020] Figure 10 It is a processing flowchart of an infrared camera according to a specific embodiment of the present invention;

[0021] Figure 11 It is a flowchart for running a fast AE mode according to a specific embodiment of the present invention;

[0022] Figure 12 It is a structural block diagram of a brightness determination device according to an embodiment of the present invention. Detailed implementation manners

[0023] In the following, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in conjunction with the embodiments.

[0024] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above - mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence.

[0025] First, the related technologies involved in the present invention will be described:

[0026] 1. Optical structure system:

[0027] Main applicable projector types:

[0028] Structured - light laser pattern projection method, structured - light speckle pattern projection method, etc.

[0029] Common usage scenarios of structured - light devices:

[0030] The color camera acquires a white - light image, the infrared camera acquires an infrared image, the projector emits an infrared pattern, the infrared camera acquires relevant information such as the pattern and saves it, and the software algorithm analyzes it to obtain an anti - forgery image.

[0031] 2. Principle of structured - light depth acquisition:

[0032] The projector emits an infrared pattern, and the infrared camera can accurately acquire parameters such as the pattern, shape, and deformation degree emitted by the pattern projector. The software algorithm calculates the spatial information of the surface and the texture information of the object surface according to the deformation degree and contrast.

[0033] The method embodiments provided in the embodiments of the present application can be executed on a mobile terminal, a computer terminal, or a similar computing device. Taking running on a mobile terminal as an example, Figure 1It is a hardware block diagram of a mobile terminal for a method of determining brightness according to an embodiment of the present invention. As Figure 1 shown, the mobile terminal may include one or more ( Figure 1 only one is shown in the figure) processors 102 (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Among them, the above-mentioned mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only illustrative and does not limit the structure of the above-mentioned mobile terminal. For example, the mobile terminal may further include more or fewer components than Figure 1 shown in the figure, or have a different configuration from Figure 1 shown in the figure.

[0034] The memory 104 can be used to store computer programs. For example, software programs and modules of application software, such as the computer program corresponding to the method of determining brightness in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implements the above-mentioned method. The memory 104 may include a high-speed random access memory, and may further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely set relative to the processor 102, and these remote memories can be connected to the mobile terminal through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0035] The transmission device 106 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wireless network provided by a communication provider of the mobile terminal. In one instance, the transmission device 106 includes a network adapter (abbreviated as NIC), which can be connected to other network devices through a base station and thus can communicate with the Internet. In one instance, the transmission device 106 may be a radio frequency (abbreviated as RF) module, which is used to communicate with the Internet wirelessly.

[0036] In this embodiment, a method of determining brightness is provided. Figure 2 It is a flowchart of the method of determining brightness according to an embodiment of the present invention. As Figure 2 shown, the process includes the following steps:

[0037] Step S202: Analyze the first infrared image obtained by the first imaging device to determine the first moving speed of the target object. The first infrared image is an image including the target object obtained after the target projector projects a target pattern onto the target object at a first distance from the target device. The target device includes the first imaging device and the target projector.

[0038] Step S204: Based on the first moving speed and the first delay time, determine the target distance between the target object and the target device when calculating the distance between the target object and the target device for the current time. The first delay time is the time required to calculate the distance between the target object and the target device.

[0039] Step S206: Based on the target mapping table, the current ambient infrared brightness, and the target distance, determine the target brightness when the target projector projects the target pattern onto the target object. The target mapping table includes the pre-determined corresponding relationships between each ambient infrared brightness and the brightness projected by the target projector when projecting the target pattern onto a reference object at a preset distance from the target device.

[0040] Among them, the above operations can be performed by a platform with computing capabilities, such as an algorithm platform, etc., or a chip, device, system with computing capabilities, or a controller or processor in a device or system, or a separate controller or processor, or other processing devices or processing units with similar processing capabilities.

[0041] In the above embodiment, the first imaging device can be a camera with infrared imaging function, or other devices with infrared imaging and shooting functions, etc. The target pattern projector includes, but is not limited to, a structured light speckle projector, a structured light laser projector, etc. The target device includes, but is not limited to, a face payment device, a face unlocking device, etc.

[0042] In the above embodiments, a color camera can be used to analyze the visible light image including the target object obtained after the target pattern projector projects a target pattern onto the target object at a first distance from the target device, so as to determine the face position of the target object. For example, when the color camera determines that dynamic detection of the target object is required, the color camera can analyze the visible light image including the target object to determine the face coordinates of the target object, and then obtain the face coordinate position determined by the color camera. The infrared camera corrects the face position to obtain the corrected face coordinate position. Then, when the face of the target object needs to be exposed subsequently, the face corresponding to the corrected face coordinate position can be exposed, further improving the accuracy of face exposure. In addition, the color camera saves the received visible light image.

[0043] In the above embodiments, by analyzing the first infrared image including the target object obtained after the target projector projects a target pattern onto the target object at a first distance from the target device by the first imaging device, the first moving speed of the target object is determined. Then, based on the first moving speed and the first delay time, when calculating the distance between the target object and the target device for the current time, the target distance between the target object and the target device is determined. Thus, further based on the target mapping table, the current ambient infrared brightness, and the target distance, the target brightness when the target projector projects a target pattern onto the target object can be determined, ensuring that the brightness projected by the target pattern projector can meet diverse requirements, improving the quality of pattern projection, and solving the problem in the related art that the projection intensity of structured light cannot be adaptively adjusted according to diverse application scenarios. Furthermore, the effect of being able to adaptively adjust the projection intensity of structured light according to diverse application scenarios is achieved.

[0044] In an exemplary embodiment, the method further includes: sequentially performing the following steps based on each received ambient infrared brightness: at the ambient infrared brightness, adjusting the brightness projected by the target projector when projecting a target pattern onto the reference object at a preset distance from the target device in a preset adjustment manner to determine the first brightness projected by the target projector. Wherein, when the target projector projects the first brightness, the clarity of the image including the reference object obtained by the first imaging device is greater than other clarities, and the other clarities are the clarities of the images including the reference object obtained by the first imaging device when the target projector projects other brightnesses except the first brightness; generating the target mapping table based on each ambient infrared brightness and the corresponding determined first brightness.

[0045] In this embodiment, the preset adjustment method can be set artificially in advance, or the device can automatically set the corresponding adjustment method according to historical adjustment records. Among them, the preset adjustment method can be to adjust the brightness projected by the target projector when projecting a target pattern onto a reference object at a preset distance from the target device in ascending order. For example, assuming that the minimum brightness that the target projector can project is 10 cd / m 2 , the brightness projected by the target projector can be adjusted sequentially to 10 cd / m 2 , 20 cd / m 2 , 30 cd / m 2 , etc. It can also be to adjust the brightness projected by the target projector when projecting a target pattern onto a reference object at a preset distance from the target device in descending order. For example, assuming that the maximum brightness that the target projector can project is 200 cd / m 2 , the brightness projected by the target projector can be adjusted sequentially to 200 cd / m 2 , 180 cd / m 2 , 160 cd / m 2 , etc. It can also be to adjust the brightness projected by the target projector when projecting a target pattern onto a reference object at a preset distance from the target device according to the best projected brightness or the commonly used projected brightness selected from the historical brightness projected by the target projector, and so on. It should also be noted that the above examples of the preset adjustment method are only exemplary embodiments, and the preset adjustment method is not limited to the above examples.

[0046] In the above embodiment, the preset distance can be set in advance and can be set to 0.5 m, 1 m, 1.5 m, etc. For example, when the preset distance is 1 m and the preset adjustment method is the ascending adjustment method, under the ambient infrared brightness, adjust the brightness projected by the target projector when projecting a target pattern onto a reference object at a distance of 1 m from the target device in ascending order to determine the first brightness projected by the target projector. It should be noted that the above examples of the preset distance are only exemplary embodiments, and the preset distance is not limited to the above examples.

[0047] In an exemplary embodiment, adjusting the brightness projected by the target projector onto the reference object at the preset distance from the target device when projecting the target pattern according to a preset adjustment method under the ambient infrared brightness includes: determining the minimum brightness projected by the target projector onto the reference object when projecting the target pattern under the ambient infrared brightness, wherein when the target projector projects the minimum brightness, the clarity of the image including the reference object acquired by the first imaging device is greater than a predetermined clarity threshold; gradually increasing the brightness projected by the pattern projector onto the reference object when projecting the target pattern from the minimum brightness. In this embodiment, the predetermined clarity threshold can be preset and can be set to 70%, 80%, 90%, etc. For example, when the predetermined clarity threshold is 90%, when the target projector projects the minimum brightness, the clarity of the image including the reference object acquired by the first imaging device is greater than 90%. It should be noted that the above examples of the predetermined clarity threshold are only exemplary embodiments, and the examples of the predetermined clarity threshold are not limited to the above examples.

[0048] In an exemplary embodiment, parsing the first infrared image obtained by the first imaging device to determine the first moving speed of the target object includes: determining a second distance between the target object and the target device based on a previous infrared image adjacent to the first infrared image obtained by the first imaging device; determining a second delay time between the first imaging device obtaining the first infrared image and obtaining the previous infrared image; and determining the first moving speed based on the first distance, the second distance, and the second delay time. In this embodiment, the first infrared image including the target object obtained after the target pattern projector projects a target pattern onto the target object at a first distance from the target device and received by the infrared camera is parsed to determine the first distance between the target object and the target pattern projector. At the same time, the previous infrared image adjacent to the first infrared image and including the target object obtained after the target pattern projector projects a target pattern onto the target object at a second distance from the target device and received by the infrared camera is parsed to determine the second distance. Then, based on the second delay time, the first distance, and the second distance between the infrared camera receiving the previous infrared image and receiving the first infrared image, the first moving speed of the target object is determined. In addition, the infrared camera saves the received images. It should also be noted that in this embodiment, in addition to determining the distance between the target object and the target device based on the infrared image, a color camera (or visible light camera) provided in the target device can also be used to determine the distance between the target object and the target device. For example, when the color camera determines that the face of the target object is moving, the color camera triggers the environmental motion detection state. The number of times, the visible light image including the target object captured by the color camera can be parsed to determine the face coordinates of the target object, and then the distance between the target object and the target device can be determined based on the face position corresponding to the face coordinates. To further ensure the accuracy of the face coordinates, the face position corresponding to the face coordinates can be corrected on the infrared camera to obtain the corrected face coordinates. In addition, the target device includes but is not limited to a color camera.

[0049] In an exemplary embodiment, determining the target distance between the target object and the target device when calculating the distance between the target object and the target device for the current time based on the first moving speed and the first delay time includes: calculating the target distance d through the following formula real :

[0050] d real = d cur + k * T delay * v 人脸

[0051] where d curis the first distance, k is the first preset parameter, T delay is the first delay time, v 人脸 is the first moving speed. In this embodiment, the first preset parameter can be a preset reserved parameter, which can be set to 0.7, 0.8, 0.9, etc. For example, assuming the first distance is 1 m, the first preset parameter is 1, the first delay time is 0.2 s, and the first moving speed is 0.5 m / s, according to the above formula, the target distance is calculated to be 1.1 m. At this time, the target distance may be the distance of the next movement of the target object, or may be a distance larger than the distance of the next movement of the target object. Therefore, in order to prevent the calculated target distance from being too large compared to the actual distance of the next movement of the target object, a certain moving distance is reserved through the first preset parameter. For example, assuming the first distance is 1 m, the first preset parameter is 0.8, the first delay time is 0.2 s, and the first moving speed is 0.5 m / s, according to the above formula, the target distance is calculated to be 1.08 m. Comparing with the above 1.1 m, it can be seen that a moving distance of 0.02 m is reserved. It should be noted that the above examples of the first distance, the first preset parameter, the first delay time, and the first moving speed are only exemplary embodiments, and the first distance, the first preset parameter, the first delay time, and the first moving speed are not limited to the above examples.

[0052] In an exemplary embodiment, determining the target brightness when the target projector projects the target pattern onto the target object based on the target mapping table, the current ambient infrared brightness, and the target distance includes: determining the current projection brightness corresponding to the current ambient infrared brightness from the target mapping table; determining the second brightness projected by the target projector when projecting the target pattern onto the target object at the position corresponding to the target distance based on the current projection brightness, the preset distance, and the target distance; determining the target brightness based on the second brightness, the target mapping table, the current ambient infrared brightness, and the target distance. In this embodiment, there may be multiple current projection brightnesses corresponding to the current ambient infrared brightness in the target mapping table. The multiple current projection brightnesses include but are not limited to the current optimal projection brightness corresponding to the current ambient infrared brightness and other brightnesses close to the current optimal projection brightness corresponding to the current ambient infrared brightness. Furthermore, when using different projectors, when the projector does not support the current optimal projection brightness, other brightnesses close to the current optimal projection brightness corresponding to the current ambient infrared brightness can be determined from the mapping table. It should also be noted that the above examples of the current projection brightness are only exemplary embodiments, and the current projection brightness is not limited to the above examples.

[0053] In the above embodiment, the second brightness i2 can be calculated by the following formula:

[0054]

[0055] Among them, i1 can be the current projection brightness, d1 is the preset distance, and d2 is the target distance. It should be noted that the calculation method of the second brightness is only an exemplary embodiment, and the calculation method of the second brightness is not limited to the above examples.

[0056] In an exemplary embodiment, determining the target brightness based on the second brightness, the target mapping table, the current ambient infrared brightness, and the target distance includes: calculating the target brightness y through the following formula real :

[0057] y real =(y dest -y cur ) * e + y cur

[0058] Among them, the y dest is the second brightness, y cur is the brightness projected by the target projector on the target object at the position corresponding to the first distance when projecting the target pattern based on the preset distance and the first distance, and e is the second preset parameter. In this embodiment, the second preset parameter can be a preset adjustment parameter, which can be set to 0.7, 0.8, 0.9, etc. For example, when the target brightness calculated by the above formula is too high or too low, the second preset parameter can be appropriately adjusted to obtain the optimal target brightness. It should be noted that the above examples of the second preset parameter are only exemplary embodiments, and the second preset parameter is not limited to the above examples.

[0059] In an exemplary embodiment, after determining the target brightness when the target projector projects the target pattern on the target object based on the target mapping table, the current ambient infrared brightness, and the target distance, the method further includes: at the target brightness, adjusting the exposure speed of the automatic exposure (AE) of the first imaging device to the maximum exposure speed supported by the first imaging device; controlling the first imaging device to perform exposure processing on the target object based on the maximum exposure speed. In this embodiment, the first imaging device can also automatically capture only the target brightness when the target projector projects the target pattern on the target object, and at this target brightness, automatically adjust the exposure speed of the automatic exposure AE to the maximum exposure speed supported by itself, and then perform exposure processing on the target object through the maximum exposure speed.

[0060] Obviously, the embodiments described above are only a part of the embodiments of the present invention, rather than all embodiments.

[0061] The following is a specific description of the present invention in conjunction with specific embodiments (taking the above-mentioned first imaging device as an infrared camera as an example):

[0062] Operation 1:

[0063] Process and calibrate according to the ambient brightness INPUT, and output the pattern projector intensity, minimum pattern projector brightness, maximum pattern projector brightness, and maximum gain of device operation. And adaptively adjust the projector brightness according to the actual face situation.

[0064] In a specific embodiment of the present invention, a flow of a method for determining the brightness of a pattern projector is provided. Figure 3 It is a flowchart of a method for determining the brightness of a pattern projector according to a specific embodiment of the present invention. The process includes the following steps:

[0065] S302, determine the current ambient brightness;

[0066] 1. At multiple different infrared brightnesses (corresponding to the above-mentioned ambient infrared brightness), confirm the INPUT of the calibration list (corresponding to the ambient infrared brightness in the above-mentioned target mapping table) in an increasing manner.

[0067] 2. At each infrared brightness, place the face at the farthest recognizable distance position (corresponding to the above-mentioned preset distance).

[0068] 3. Adjust the intensity of the pattern projector (corresponding to the above-mentioned target projector) in an increasing order (corresponding to the above-mentioned preset adjustment method) until the pattern information can be clearly recognized at the receiving end of the infrared camera (corresponding to the first imaging device above).

[0069] 4. Gradually increase the intensity of the pattern projector until the pattern projection reaches the infrared safety limit value.

[0070] 5. Generate the projection brightness values of the pattern projector under different ambient infrared brightnesses from the above steps 1-4.

[0071] y 基准 = MAP(x INPUT ) Formula (1)

[0072] Among them, this formula (1) mainly represents the projection brightness values of the pattern projector under different ambient infrared brightnesses.

[0073] Figure 4 It is an example diagram of a mapping table of ambient infrared brightness - projector brightness according to a specific embodiment of the present invention, as Figure 4As shown, the specific generation method of this mapping table (corresponding to the above-mentioned target mapping table) is as follows (the values represented by the x-axis and y-axis can be changed, and the present application does not limit this):

[0074] The x-axis represents the index value (corresponding to the above-mentioned ambient infrared brightness), and for every doubling of the brightness, the index value increases by 1.

[0075] The y-axis represents the projector current, and an ideal projector brightness can be generated according to the input ambient infrared brightness.

[0076] Minimum projector brightness (corresponding to the above-mentioned minimum brightness):

[0077] It doesn't need to be too small, as the present application has covered the fully black scene in actual use.

[0078] Maximum projector brightness:

[0079] It is affected by factors such as hardware performance and infrared safety.

[0080] Calibration of the minimum pattern projector brightness:

[0081] In the case of no ambient infrared brightness scene, the minimum emission brightness of the pattern projector can be used as the minimum pattern projector brightness.

[0082] Maximum gain during device operation:

[0083] Based on the minimum pattern projector brightness, the maximum shutter and maximum gain during device operation can be further calibrated.

[0084] Maximum ambient brightness that the device can use:

[0085] As Figure 4 shown, after the index value is greater than about 16, when the ambient brightness increases further at this time, the brightness of the projector will not increase. Due to hardware reasons, the ambient brightness at this time is the maximum ambient brightness that the device can support.

[0086] S304. Map the projector brightness reference value according to the pre-calibrated ambient brightness;

[0087] The reference brightness corresponding to each ambient infrared brightness projected by the pattern projector in the ambient infrared brightness-projector brightness mapping table is the brightness projected onto the face at a distance of about 1m from the device position to the face position (corresponding to the above-mentioned preset distance). However, in actual applications, the face position may be at a distance of 1.5m from the device position, or may be at a distance of 0.5m from the device position, and so on. Therefore, it is very necessary to have an algorithm for adjusting the projector brightness at different face distances.

[0088] First, it is necessary to project the brightness of the projector for different face distances, and the brightness value needs to be changed based on the reference brightness. Here, the following formula (2) can be directly referred to:

[0089]

[0090] Among them, i represents the light intensity, and d represents the distance.

[0091] S306. Analyze the depth information according to the current face information;

[0092] Since the overall process time from the running state of the face in the real world to the camera capturing an image, then the captured image is input to the depth analysis algorithm, and finally the most recent face depth information is generated, there is a delay. If the method of directly using the distance of depth analysis to adjust the projector brightness is inappropriate. Therefore, the following model is proposed:

[0093] Model 01. Calculate the relative speed between the face and the camera according to the depth information and the situation of the algorithm platform. Optionally, a quadratic equation may also be used for operation, and an algorithm for predicting the running posture of the face through a neural network.

[0094] v 人脸 =(d last -d cur ) / T Formula (3)

[0095] d last represents the face depth information parsed by the previous depth analysis algorithm (corresponding to the above second distance). d cur represents the face depth information parsed by the most recent depth analysis algorithm (corresponding to the above first distance). T represents the algorithm running interval (corresponding to the above second delay time). Optionally, for platforms with better performance, the depth analysis speed is faster and can reach per-frame analysis. For platforms with poorer performance, the analysis speed is slower and it may take 2 - 3 frames to analyze once.

[0096] S308. Predict the distance between the face in the real world and the projector (corresponding to the above target distance);

[0097] d real =d cur +k*T delay *v 人脸 Formula (4)

[0098] T delayIt represents the delay time (corresponding to the above-mentioned first delay time) between the imported real face and the algorithm settlement result according to different platforms (corresponding to the above-mentioned algorithm platforms) and sensors (such as distance sensors, positioning sensors, etc.). Considering that the actual face movement distance is irregular and randomly variable. k (corresponding to the above-mentioned first preset parameter) represents a reserved parameter designed, which cannot fully predict the face distance, leaving a certain margin to prevent over-prediction.

[0099] According to the face distance calculation model including the prediction algorithm, the distance of the face in reality can be roughly calculated.

[0100] S310, calculate the final projector brightness value according to the predicted distance (corresponding to the above-mentioned target brightness);

[0101] Model 02, the algorithm for the brightness adjustment process. Since there is also a delay of about 2 frames between the brightness adjustment process and the actual effective process. Therefore, the process of adjusting the brightness can be carried out in the following formula way.

[0102] y real =(y dest -y cur ) * e + y cur Formula (5)

[0103] Among them, y dest represents the mapping according to the current ambient brightness (corresponding to the above-mentioned second brightness) and the predicted face distance d real (corresponding to the above-mentioned target distance) according to Formula 2. e represents the coefficient adjusted during the operation process. y cur represents the brightness of the current projector. y real (corresponding to the above-mentioned target brightness) represents the final brightness value of the projector.

[0104] S312, perform delayed effect, the actual projection intensity in the real world;

[0105] S314, finally take effect on the face in the actual real world.

[0106] Operation 2:

[0107] Obtain depth information by parsing the pattern on the pattern emitter.

[0108] The infrared camera (pattern receiver) only needs to ensure that it can receive the pattern normally.

[0109] The current operation mode of "Fast AE" mainly consists of two parts.

[0110] Camera 1 - Color camera:

[0111] The color camera determines the environmental motion detection;

[0112] The color camera performs face recognition in a color environment to obtain the maximum face coordinate position;

[0113] Camera 2 - Infrared camera:

[0114] The infrared camera is divided into two modes:

[0115] Mode 1:

[0116] 1. The color camera does not trigger environmental motion detection;

[0117] 2. The infrared camera is in a ready state and uses the automatic exposure adjustment method at normal speed;

[0118] 3. Adjust to global exposure and adjust the target brightness to about 10 - 20;

[0119] 4. Calculate the environmental brightness value INPUT.

[0120] Note: This is a preparation state to reduce noise, facilitating subsequent fast face exposure operations.

[0121] Mode 2:

[0122] 1. When it is determined that the current environment is in a motion detection state, quickly obtain the environmental brightness INPUT calculated by Camera 2;

[0123] 2. Generate the current pattern projector brightness according to INPUT and the adaptive projection brightness mapping table of the first module;

[0124] 3. When the face coordinates 1 on Camera 1 are obtained, perform position correction on Camera 2 to obtain face coordinates 2;

[0125] 4. According to face coordinates 2, turn on the face exposure mode of Camera 2.

[0126] Face exposure mode of Camera 2:

[0127] Since the main information collected by Camera 2 is pattern information and the background noise has been calibrated in advance, at this time, it is only necessary to ensure that the brightness of the face area is between 50 - 150, run the face exposure mode, that is, increase the AE operation speed to the maximum.

[0128] Figure 5 It is a schematic diagram of the pattern projected by the pattern projector in a specific embodiment of the present invention. As Figure 5 shown, the color camera 1 performs face recognition, and the infrared camera 2 receives the pattern of the pattern emitter at the same time. Finally, face recognition is performed on the image of the color camera 1, and depth information is calculated on the pattern of the infrared camera 2 to enhance the anti - forgery performance of the face and improve the security performance.

[0129] Figure 6 is a schematic diagram of the pattern projection effect in an indoor completely dark scene according to a specific embodiment of the present invention. As Figure 6 shown, in an indoor situation, the projector brightness can be adjusted to a lower level and the pattern is clear.

[0130] Figure 7 is a schematic diagram of the pattern projection effect when there is high background noise outdoors according to a specific embodiment of the present invention. As Figure 7 shown, when the ambient brightness is relatively high, the projector brightness can be adjusted to a higher level and the pattern is clear.

[0131] A specific embodiment of the present invention also provides an overall application process, which includes the following steps:

[0132] S1, calibrate the device;

[0133] Figure 8 is a flowchart for generating a mapping table of ambient infrared brightness - projector brightness according to a specific embodiment of the present invention. This process includes the following steps:

[0134] S802, place the device in a completely dark scene in a darkroom, and adjust the projection intensity of the speckle pattern projector so that its IR (IP - Camera, infrared camera) receiving end can stably and clearly receive the pattern;

[0135] S804, adjust the infrared ambient brightness to ensure that as the ambient brightness doubles, the projector brightness of the speckle pattern projector also increases accordingly, and finally ensure that when obtaining the pattern with clarity, the projector brightness is as small as possible;

[0136] S806, generate a mapping table of ambient infrared brightness - projector brightness.

[0137] It should be noted that the above process can calibrate multiple parameter values, including but not limited to the minimum projector brightness in complete darkness, the maximum projector brightness under full - bright sunlight, etc.

[0138] S2, the normal operation process of the device;

[0139] Color camera:

[0140] Figure 9 is a processing flowchart of a color camera according to a specific embodiment of the present invention. As Figure 9 shown, this process includes the following steps:

[0141] S902, a normal color camera;

[0142] S904, the color camera operates normally;

[0143] S906, perform a first judgment, and monitor in real time to determine whether the color camera triggers motion detection. If the result of the first judgment is negative, execute step S904;

[0144] S908, if the result of the first judgment is positive, generate the current projector brightness 1 according to the calibrated environmental infrared brightness - projector brightness mapping table;

[0145] S910, after motion detection is enabled, turn on the speckle projector, monitor in real time and perform face recognition to generate face data and face coordinates.

[0146] Infrared camera:

[0147] Figure 10 It is a processing flow chart of an infrared camera according to a specific embodiment of the present invention. As Figure 10 shown, the process includes the following steps:

[0148] S1002, the infrared camera operates normally;

[0149] S1004, the environmental infrared brightness 1 can be calibrated through a non - photosensitive algorithm;

[0150] S1006, perform a first judgment, and detect in real time to determine whether the speckle pattern projector is turned on;

[0151] S1008, if the result of the first judgment is negative, according to the current speckle pattern recognition principle, the principle that the background noise needs to be significantly different from the speckles;

[0152] S1010, set the image brightness to a lower value (for example, less than 20, 25, 30, etc.) or correct the global image brightness to a lower value.

[0153] S1012, if the result of the first judgment is positive, turn on the projector according to the generated projector brightness 1 and keep the brightness constant;

[0154] S1014, perform corresponding correction of the face coordinate position of the color camera on the infrared camera, and send the corrected face coordinate position to the infrared camera, and the infrared camera module executes the regional exposure strategy for the face area;

[0155] S1016, according to the characteristics of the pattern projector (it can clearly identify the pattern shape, deformation, texture, etc.), perform automatic exposure in the "Fast AE" mode.

[0156] "Fast AE" mode:

[0157] Figure 11 It is a running flow chart of a fast AE mode according to a specific embodiment of the present invention. As Figure 11As shown in the figure, the process includes the following steps:

[0158] S1102, after turning on the projector, initialize the exposure parameters. Since the ambient brightness has been calibrated before turning on the projector and the projection brightness of the corresponding pattern projector is known, the initial parameters of the exposure are closer to the ambient brightness;

[0159] S1104, ensure that the target brightness of the image is in the range of 50 - 150. According to the characteristics of the pattern projector and ensuring that the background noise is small (previously calibrated), the brightness of the image only needs to be in the range of 50 - 150 to accurately obtain the brightness value projected by the pattern projector and use it for depth data calculation;

[0160] S1106, adjust the AE speed to the maximum. Since the pattern brightness tolerance is large, there is no need to worry about over - adjustment during the overall AE operation. After obtaining the pattern brightness, the adjustment of the AE operation can be completed within 1 - 2 frames.

[0161] As can be seen from the foregoing embodiments, through the mapping method of the ambient brightness and the projection intensity of the pattern projector, the brightness determination method proposed in this application is applicable to both indoor environments with low background noise and low brightness and outdoor scenes including sunlight, solving the problem that most structured light devices in the related art are only applicable to indoor environments and the background noise is too large in outdoor environments, affecting the acquisition of structured light. Through the "fast AE" mode, the brightness determination method proposed in this application is further applicable to the scenario where the human face moves rapidly in front of the projector, that is, from the perspective of the pattern projector principle, the "fast AE" operation mode can make the operation speed of automatic exposure reach the maximum, solving the problem that in the case of human face movement, the image adjustment speed of most structured light devices in the related art is slow and over - exposure is likely to occur.

[0162] Through the description of the above - mentioned implementation manners, those skilled in the art can clearly understand that the method according to the above - mentioned embodiments can be implemented by means of software plus a necessary general - purpose hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such an understanding, the technical solution of the present invention, in essence, or the part that makes a contribution to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present invention.

[0163] In this embodiment, a device for determining brightness is further provided. This device is used to implement the above-mentioned embodiments and preferred implementation manners, and those that have been described will not be repeated. As used hereinafter, the term "module" can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0164] Figure 12 is a structural block diagram of a device for determining brightness according to an embodiment of the present invention. As Figure 12 shown, the device includes:

[0165] A first determination module 1202, configured to determine a first moving speed of a target object by analyzing a first infrared image obtained by a first imaging device, where the first infrared image is an image including the target object obtained after a target projector projects a target pattern onto the target object at a first distance from the target device, and the target device includes the first imaging device and the target projector;

[0166] A second determination module 1204, configured to determine a target distance between the target object and the target device when calculating the distance between the target object and the target device for the current time based on the first moving speed and a first delay time, where the first delay time is the time required to calculate the distance between the target object and the target device;

[0167] A third determination module 1206, configured to determine a target brightness when the target projector projects the target pattern onto the target object based on a target mapping table, the current ambient infrared brightness, and the target distance, where the target mapping table includes a correspondence relationship between each pre-determined ambient infrared brightness and the brightness respectively projected by the target projector when projecting the target pattern onto a reference object at a preset distance from the target device.

[0168] In an exemplary embodiment, the above device further includes:

[0169] An execution module, configured to sequentially perform the following steps based on each received ambient infrared brightness: at the ambient infrared brightness, adjust the brightness projected by the target projector when projecting the target pattern onto the reference object at the preset distance from the target device according to a preset adjustment method, so as to determine a first brightness projected by the target projector, wherein, when the target projector projects the first brightness, the clarity of the image including the reference object acquired by the first imaging device is greater than other clarities, and the other clarities are the clarities of the images including the reference object acquired by the first imaging device when the target projector projects other brightnesses except the first brightness;

[0170] A fourth determination module, configured to generate the target mapping table based on each ambient infrared brightness and the corresponding determined first brightness.

[0171] In an exemplary embodiment, the above-mentioned execution module includes:

[0172] A first determination sub-module, configured to determine the minimum brightness projected by the target projector when projecting the target pattern onto the reference object at the ambient infrared brightness, wherein, when the target projector projects the minimum brightness, the clarity of the image including the reference object acquired by the first imaging device is greater than a predetermined clarity threshold;

[0173] An increase sub-module, configured to gradually increase the brightness projected by the target projector when projecting the target pattern onto the reference object from the minimum brightness.

[0174] In an exemplary embodiment, the above-mentioned first determination module 1202 includes:

[0175] A second determination sub-module, configured to determine a second distance between the target object and the target device based on a previous infrared image adjacent to the first infrared image acquired by the first imaging device;

[0176] A third determination sub-module, configured to determine a second delay time between the first imaging device acquiring the first infrared image and acquiring the previous infrared image;

[0177] A fourth determination sub-module, configured to determine the first moving speed based on the first distance, the second distance, and the second delay time.

[0178] In an exemplary embodiment, the above-mentioned second determination module 1204 includes:

[0179] A calculation sub-module, configured to calculate the target distance d through the following formula real :

[0180] d real = d cur + k * T delay * v 人脸

[0181] Wherein, d cur is the first distance, k is the first preset parameter, T delay is the first delay time, and v 人脸 is the first moving speed.

[0182] In an exemplary embodiment, the above third determination module 1206 includes:

[0183] A fifth determination sub-module, configured to determine the current projection brightness corresponding to the current ambient infrared brightness from the target mapping table;

[0184] A sixth determination sub-module, configured to determine the second brightness projected by the target projector when projecting the target pattern onto the target object at the position corresponding to the target distance based on the current projection brightness, the preset distance, and the target distance;

[0185] A seventh determination sub-module, configured to determine the target brightness based on the second brightness, the target mapping table, the current ambient infrared brightness, and the target distance.

[0186] In an exemplary embodiment, the above seventh determination sub-module includes:

[0187] A calculation unit, configured to calculate the target brightness y through the following formula real :

[0188] y real = (y dest - y cur ) * e + y cur

[0189] Wherein, the y dest is the second brightness, y cur is the brightness projected by the target pattern projector when projecting the target pattern onto the target object at the position corresponding to the first distance determined based on the preset distance and the first distance, and e is the second preset parameter.

[0190] In an exemplary embodiment, the above device further includes:

[0191] A processing module, configured to, after determining a target brightness when the target projector projects a target pattern onto a target object based on a target mapping table, a current ambient infrared brightness, and the target distance, adjust an exposure speed of an automatic exposure (AE) of the first imaging device to a maximum exposure speed supported by the first imaging device at the target brightness;

[0192] A control module, configured to control the first imaging device to perform an exposure process on the target object based on the maximum exposure speed.

[0193] It should be noted that the above-mentioned various modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited thereto: the above-mentioned modules are all located in the same processor; or, the above-mentioned various modules are separately located in different processors in any combination form.

[0194] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored. Wherein, the computer program is configured to execute the steps in any one of the above method embodiments when running.

[0195] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: various media such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disc that can store a computer program.

[0196] An embodiment of the present invention further provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0197] In an exemplary embodiment, the above electronic device may further include a transmission device and an input / output device. Wherein, the transmission device is connected to the above processor, and the input / output device is connected to the above processor.

[0198] Specific examples in this embodiment may refer to the examples described in the above embodiments and exemplary embodiments, and will not be repeated here.

[0199] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the present invention can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. They can be implemented by program codes executable by the computing device. Thus, they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described can be executed in a different order from here, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. In this way, the present invention is not limited to any specific combination of hardware and software.

[0200] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for determining brightness, characterized in that, Including: Analyzing a first infrared image obtained by a first imaging device to determine a first moving speed of a target object, where the first infrared image is an image including the target object obtained after a target projector projects a target pattern onto the target object at a first distance from the target device, and the target device includes the first imaging device and the target projector; Determining a target distance between the target object and the target device when calculating the distance between the target object and the target device for the current time based on the first moving speed and a first delay time, where the first delay time is the time required to calculate the distance between the target object and the target device; Determining a target brightness when the target projector projects the target pattern onto the target object based on a target mapping table, a current ambient infrared brightness, and the target distance, where the target mapping table includes a correspondence relationship between each pre-determined ambient infrared brightness and the brightness projected by the target projector when projecting the target pattern onto a reference object at a preset distance from the target device.

2. The method according to claim 1, wherein The method further includes: Successively performing the following steps based on each received ambient infrared brightness: adjusting, in the ambient infrared brightness, the brightness projected by the target projector when projecting the target pattern onto the reference object at the preset distance from the target device according to a preset adjustment method to determine a first brightness projected by the target projector, where, when the target projector projects the first brightness, the clarity of the image including the reference object obtained by the first imaging device is greater than other clarities, and the other clarities are the clarities of the images including the reference object obtained by the first imaging device when the target projector projects other brightnesses other than the first brightness; Generating the target mapping table based on each ambient infrared brightness and the corresponding determined first brightness.

3. The method according to claim 2, characterized in that, Adjusting, in the ambient infrared brightness, the brightness projected by the target projector when projecting the target pattern onto the reference object at the preset distance from the target device according to a preset adjustment method includes: Determining a minimum brightness projected by the target projector when projecting the target pattern onto the reference object in the ambient infrared brightness, where, when the target projector projects the minimum brightness, the clarity of the image including the reference object obtained by the first imaging device is greater than a predetermined clarity threshold; Gradually increasing the brightness projected by the target projector when projecting the target pattern onto the reference object from the minimum brightness.

4. The method according to claim 1, wherein Analyzing a first infrared image obtained by a first imaging device to determine a first moving speed of a target object includes: Determining a second distance between the target object and the target device based on a previous frame infrared image adjacent to the first infrared image obtained by the first imaging device; Determine a second delay time between the first imaging device acquiring the first infrared image and acquiring the previous infrared image; Determine the first moving speed based on the first distance, the second distance, and the second delay time.

5. The method according to claim 1, wherein Based on the first moving speed and the first delay time, when calculating the distance between the target object and the target device for the current time, the target distance between the target object and the target device includes: Calculate the target distance d through the following formula real : d real = d cur + k * T delay * v 人脸 where d cur is the first distance, k is a first preset parameter, T delay is the first delay time, v 人脸 is the first moving speed.

6. The method according to claim 1, wherein Based on the target mapping table, the current ambient infrared brightness, and the target distance, the target brightness when the target projector projects the target pattern onto the target object includes: Determine the current projection brightness corresponding to the current ambient infrared brightness from the target mapping table; Based on the current projection brightness, the preset distance, and the target distance, determine the second brightness projected by the target projector when projecting the target pattern onto the target object at the position corresponding to the target distance; Determine the target brightness based on the second brightness, the target mapping table, the current ambient infrared brightness, and the target distance.

7. The method according to claim 6, wherein Determining the target brightness based on the second brightness, the target mapping table, the current ambient infrared brightness, and the target distance includes: Calculate the target brightness y using the following formula real :[[]]END]] y real = (y dest - y cur ) * e + y cur Among them, the y dest is the second brightness, and y cur is the brightness projected by the target projector when projecting the target pattern onto the target object at the position corresponding to the first distance, which is determined based on the preset distance and the first distance. e is a second preset parameter.

8. The method according to claim 1, wherein After determining the target brightness when the target projector projects the target pattern onto the target object based on the target mapping table, the current ambient infrared brightness, and the target distance, the method further includes: At the target brightness, adjust the exposure speed of the automatic exposure (AE) of the first imaging device to the maximum exposure speed supported by the first imaging device; Control the first imaging device to perform exposure processing on the target object based on the maximum exposure speed.

9. A device for determining brightness, characterized in that, Includes: A first determination module, configured to determine the first moving speed of the target object by analyzing the first infrared image acquired by the first imaging device, where the first infrared image is an image including the target object obtained after the target projector projects the target pattern onto the target object at a first distance from the target device, and the target device includes the first imaging device and the target projector; A second determination module, configured to determine the target distance between the target object and the target device when calculating the distance between the target object and the target device for the current time based on the first moving speed and the first delay time, where the first delay time is the time required to calculate the distance between the target object and the target device; A third determination module, configured to determine the target brightness when the target projector projects the target pattern onto the target object based on the target mapping table, the current ambient infrared brightness, and the target distance, where the target mapping table includes the corresponding relationships between each pre-determined ambient infrared brightness and the brightness projected by the target projector when projecting the target pattern onto a reference object at a preset distance from the target device.

10. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, wherein when the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 8 are implemented.

11. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, the steps of the method described in any one of claims 1 to 8 are implemented.

Images