Active exposure method, device and electronic equipment based on TOF imaging system

Abstract

The present application provides an automatic exposure method based on a TOF imaging system. The method includes: obtaining grayscale information of a target scene image of a calculation frame in a current frame; calculating an automatic exposure time according to the obtained grayscale information; and adjusting the exposure time of multiple subframes after the calculation frame in the current frame according to the automatic exposure time. During the imaging process, a TOF imaging system using the automatic exposure method can apply the automatic exposure time calculated according to the target scene of the current frame to the current frame, which is beneficial to improving measurement accuracy and imaging accuracy.

Description

Technical Field

[0001] The present application relates to the field of machine vision, and more specifically, to an automatic exposure method based on a TOF imaging system, an automatic exposure device, an electronic device, a computer-readable medium, and a TOF imaging system. Background Art

[0002] At present, more and more mobile robots are equipped with visual sensors such as TOF (Time of Flight) cameras to perceive the external environment in order to achieve automatic navigation and obstacle avoidance functions.

[0003] The TOF camera emits light to the target scene, and then uses an image sensor to receive the light reflected from the target scene to calculate the flight (round-trip) time of the emitted light and the reflected light, thereby obtaining the distance between the TOF camera and the target scene. In this way, the image taken by the TOF camera has not only conventional two-dimensional image information, but also depth information of the distance between the TOF camera and the target scene, and thus generates a three-dimensional image of the target scene.

[0004] As the robot's moving speed increases, when the scene switches quickly, the image generated by the TOF camera will be overexposed or underexposed, causing the robot to collide with objects in the target scene. Summary of the invention

[0005] The present application provides an automatic exposure method based on a TOF imaging system, an automatic exposure device, an electronic device, a computer-readable medium, and a TOF imaging system that can at least partially solve the above-mentioned technical problems.

[0006] One aspect of the present application provides an automatic exposure method, which includes: obtaining grayscale information of a target scene image of a calculation frame in a current frame; calculating an automatic exposure time according to the obtained grayscale information; and adjusting the exposure time of multiple subframes after the calculation frame in the current frame according to the automatic exposure time.

[0007] In one embodiment, the calculation frame may include a valid frame and an invalid frame after the valid frame, wherein the step of acquiring grayscale information may include: acquiring grayscale information of a target scene image of the valid frame in the calculation frame.

[0008] In one embodiment, the exposure time within the invalid frame may be zero.

[0009] In one embodiment, the exposure time within the invalid frame may be a non-zero constant.

[0010] In one embodiment, the step of acquiring grayscale information may include: acquiring grayscale information of the calculation frame in the current frame according to amplitude information of the target scene image of the calculation frame in the current frame.

[0011] In one embodiment, the grayscale information of the target scene image may include grayscale values ​​of multiple pixels of the target scene image, wherein the step of calculating the automatic exposure time may include: calculating the automatic exposure time according to an average value of the grayscale values ​​of the multiple pixels.

[0012] In one embodiment, the method further includes: adjusting the exposure time of the calculation frame in the subsequent frame according to the automatic exposure time.

[0013] In one embodiment, the plurality of subframes after the calculation frame in the current frame is four subframes.

[0014] In one embodiment, the step of adjusting the exposure time of the plurality of subframes may include: adjusting each of the plurality of subframes according to the same automatic exposure time.

[0015] In one embodiment, there may be a time interval between the current frame and the next frame.

[0016] Another aspect of the present application provides an automatic exposure device, which includes: an acquisition module for acquiring grayscale information of a target scene image of a calculation frame in a current frame; a calculation module for calculating an automatic exposure time based on the acquired grayscale information; and an adjustment module for adjusting the exposure time of multiple subframes after the calculation frame in the current frame according to the automatic exposure time.

[0017] In one embodiment, the calculation frame may include a valid frame and an invalid frame after the valid frame, and the grayscale information may come from the target scene image of the valid frame in the calculation frame.

[0018] In one embodiment, the exposure time within the invalid frame may be zero.

[0019] In one embodiment, the exposure time within the invalid frame may be a non-zero constant.

[0020] In one embodiment, the adjustment module may be further configured to adjust the exposure time of the calculation frame in the subsequent frame according to the automatic exposure time.

[0021] Another aspect of the present application provides an electronic device, which includes: a memory for storing a program; and a processor for executing any implementation of the automatic exposure method described above when executing the program.

[0022] Another aspect of the present application provides a computer-readable medium, on which a computer program is stored, and when the computer program is executed by a processor, any implementation of the automatic exposure method described above is executed.

[0023] Another aspect of the present application provides a TOF imaging system. The TOF imaging system includes: a light source module for emitting light to a target scene; a photosensitive module for receiving light reflected from the target scene; and a working module for controlling the light source module and the photosensitive module, and processing the reflected light received by the photosensitive module to perform any embodiment of the automatic exposure method described above.

[0024] During the imaging process, the TOF imaging system using this automatic exposure method can calculate the automatic exposure time according to the target scene of the current frame and apply it to the current frame, which is beneficial to improving the measurement accuracy and imaging quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0026] Figure 1 is a working principle diagram of a TOF imaging system according to an embodiment of the present application;

[0027] Figure 2 is a schematic block diagram of a TOF imaging system according to an embodiment of the present application;

[0028] Figure 3 is a flow chart of an automatic exposure method based on a TOF imaging system according to an embodiment of the present application;

[0029] Figure 4 is a schematic diagram of a current frame format and a subsequent frame format according to an implementation manner of the present application;

[0030] Figure 5 is a schematic diagram of an existing current frame format and a subsequent frame format; and

[0031] Figure 6 It is a schematic diagram of the structure of an electronic device suitable for implementing the automatic exposure method of the embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to better understand the present application, various aspects of the present application will be described in more detail with reference to the accompanying drawings. It should be understood that these detailed descriptions are only descriptions of exemplary embodiments of the present application, and do not limit the scope of the present application in any way.

[0033] The terms used herein are for the purpose of describing specific exemplary embodiments and are not intended to be limiting. When used in this specification, the terms "comprise", "comprising", "including" and / or "comprising" indicate the presence of the features, integral bodies, elements, parts and / or combinations thereof, but do not exclude the presence of one or more other features, integral bodies, elements, parts and / or combinations thereof.

[0034] This document is described with reference to schematic diagrams of exemplary embodiments. The exemplary embodiments disclosed herein should not be construed as limited to the specific shapes and sizes shown, but include various equivalent structures that can achieve the same functions and shape and size deviations caused by, for example, manufacturing. The positions shown in the drawings are schematic in nature and are not intended to limit the positions of the components.

[0035] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art to which the present disclosure belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and will not be interpreted in an idealized or overly formal sense unless explicitly defined as such herein.

[0036] The TOF imaging system can obtain the distance information from the observation point (TOF imaging system) to the target scene, and then obtain the depth information of the target scene image to generate a three-dimensional image. In addition, the TOF imaging system has the advantages of high real-time performance, stable accuracy, and small footprint. It can be implemented as a device such as a camera and is widely used in machine vision processing, such as object recognition, three-dimensional reconstruction, and mobile robot navigation.

[0037] Figure 1 Schematic diagram of the working principle of the TOF imaging system 100 according to the embodiment of the present application. Figure 1 As shown, the TOF imaging system 100 is suitable for acquiring an image of a target scene 200. Different from a conventional imaging system, the TOF imaging system 100 can acquire three-dimensional image information of the target scene 200 to be measured according to the time of flight (TOF) method. In simple terms, the process of the TOF imaging system 100 acquiring the three-dimensional image information of the target scene 200 is that the reflected light of the target scene 200 enters the TOF imaging system 100 and is received by the image sensor of the TOF imaging system 100. After the image sensor performs a preliminary analysis and processing on the incoming light, the reflected light carrying the image information is converted into the three-dimensional image information of the target scene 200 to complete the shooting of the target scene 200. During the imaging process of the TOF imaging system 100, the exposure performance of the TOF imaging system 100 directly or indirectly affects the imaging effect of the target scene 200, especially when the switching speed between the TOF imaging system 100 and the target scene 200 is too fast.

[0038] Figure 2 FIG. 1 is a schematic block diagram of a TOF imaging system 100 according to an embodiment of the present application. Figure 2 As shown, the TOF imaging system 100 may include a light source module 110 , a photosensitive module 120 and a working module 130 .

[0039] The light source module 110 can actively emit light to the target scene, and the emitted light can be used as an optical signal for detecting the target scene. The light source module 110 may include at least one light source, such as a laser light source or an LED light source. In some embodiments, the light source may be implemented as a vertical cavity surface emitting laser (VCSEL). The core parameter of the light source is the central wavelength, and infrared light of 850nm or 940nm can be used, for example. The illumination of sunlight is stronger at a wavelength of 850nm and weaker at a wavelength of 940nm. Therefore, when infrared light of 940nm wavelength is used as a light source, it is more able to avoid interference from outdoor sunlight as ambient light.

[0040] The optical signal emitted by the light source module 110 may be a modulated optical pulse or a continuous wave, and the frequency of the modulated optical pulse may reach 100 MHz.

[0041] The photosensitive module 120 can receive reflected light formed by reflection from the target scene. The reflected light signal carried by the reflected light may include distance information between the target scene 200 and the TOF imaging system 100. The depth information of the target scene 200 image can be obtained by correspondingly processing the distance information.

[0042] The photosensitive module 120 may include at least one image sensor, which may be, for example, a CCD image sensor or a CMOS image sensor with a high-speed shutter and exposure accumulation function. The image sensor may include a plurality of pixels arranged in a two-dimensional array. The difference from a conventional image sensor is that the image sensor applied to the TOF imaging system 100 also includes a modulation control unit and an A / D conversion unit, etc., for demodulating the reflected light signal and performing preliminary data processing. Therefore, the size of a single pixel of the image sensor applied to the TOF imaging system 100 is larger than that of a single pixel of a conventional image sensor. For example, the size of the image sensor of the TOF imaging system 100 may be 20 μm.

[0043] In some embodiments, the photosensitive module 120 may further include a lens (not shown) disposed on the photosensitive path of the image sensor to collect light reflected by the target scene. The photosensitive module 120 may also include a filter element (not shown), such as an infrared bandpass filter. The filter element is disposed between the image sensor and the lens to ensure that only reflected light with the same wavelength as the emitted light emitted by the light source module 110 enters the image sensor. The filter element improves the imaging quality of the TOF imaging system 100 by filtering out stray light.

[0044] The working module 130 can be communicatively connected to the light source module 110 and the photosensitive module 120 to control the light source module 110 and the photosensitive module 120 and perform data interaction. It should be understood that the communication connection method is not limited to being implemented as a wired or wireless connection method, such as an electrical connection method, a signal connection method, etc.

[0045] In some embodiments, the working module 130 may include a control unit 131. The control unit 131 may control the working states of the light source module 110 and the photosensitive module 120 by transmitting control signals to the light source module 110 and the photosensitive module 120. For example, the control unit 131 may control the light source of the light source module 110 to emit light having a preset wavelength and frequency or may control the exposure time of the received reflected light of the photosensitive module 120. In other words, the light source module 110 and the photosensitive module 120 may change their working states through the working module 130. Optionally, the control unit 131 may synchronously transmit control signals to the light source module 110 and the photosensitive module 120 so that the light source module 110 and the photosensitive module 120 work synchronously, so as to improve the accuracy of the depth information of the target scene image acquired by the TOF imaging system 100.

[0046] In some embodiments, the working module 130 may further include a processing unit 132. The processing unit 132 may be communicatively connected to the photosensitive module 120, and is used to receive the target scene image information collected by the photosensitive unit 120 for further processing to generate grayscale information or depth information of the image of the target scene, and then generate a three-dimensional image of the target scene.

[0047] The processing unit 132 can also be communicatively connected to the control unit 131, and is used to obtain a control signal for controlling the light source module 110 and the photosensitive module 120 after corresponding calculation and processing based on the collected partial information about the target scene image, and transmit it to the control unit 131. For example, the processing unit 132 can calculate and generate a control signal for the control unit 132 to perform an automatic exposure operation according to a certain automatic exposure method based on the control requirements of automatic exposure, and transmit the control signal to the control unit 131, so that the control unit 131 controls the photosensitive module 120 to perform an automatic exposure operation.

[0048] The working process of the TOF imaging system 100 can be described as follows. The light source module 110 emits an emission light with a preset wavelength, and the emission light is reflected to form reflected light after encountering the surface of the object to be measured in the target scene. The photosensitive module 120 can be controlled to quickly receive the reflected light in different time periods to obtain the grayscale information and depth information of the target scene. Specifically, after the light source module 110 emits the emission light, the photosensitive module 120 quickly exposes to obtain several pictures, and calculates the time difference or phase difference between the emission light and the reflected light based on these pictures to calculate the distance information between the target scene object and the TOF imaging system 100, that is, the depth information. Moreover, the grayscale information of the target scene can be obtained while obtaining the depth information of the target scene. By integrating the depth information and the grayscale information, a three-dimensional image of the target scene is obtained.

[0049] It should be understood that the distance measurement method can be divided into direct TOF measurement method and indirect TOF measurement method according to the different modulation methods of the light source signal. In the direct TOF measurement method, the light signal appears in the form of pulses. By obtaining the time from the emission to the reception of the light pulse, the distance from the light signal reflected from the target scene to the image sensor is obtained. In the indirect TOF measurement method, the emitted light signal can be a periodic light signal after modulation, such as a sinusoidal light signal. After the emitted light signal is delayed by the round-trip distance, the phase of the light signal arriving at the image sensor changes. Through a certain phase detection technology, the phase change is calculated to obtain the distance from the light signal reflected from the target scene to the image sensor.

[0050] The present application provides an automatic exposure method 1000, which is described below with reference to Figure 2 The TOF imaging system 100 is shown, and each step is described in detail. The working module 130 can control the exposure time of the photosensitive module 120 through the method 1000, so that the photosensitive module 120 acquires the target scene image in an automatic exposure manner. Through the description of the method 1000, it can be further explained Figure 2 The specific processing and mutual cooperation of each module of the TOF imaging system 100 are shown.

[0051] Figure 3 FIG. 1 is a flow chart of an automatic exposure method 1000 based on a TOF imaging system 100 according to an embodiment of the present application. Figure 3 As shown, method 1000 may include the following steps:

[0052] Step S110, obtaining grayscale information of the target scene image of the calculation frame in the current frame.

[0053] Step S120, calculating the automatic exposure time according to the acquired grayscale information.

[0054] Step S130 , adjusting the exposure time of a plurality of subframes after the calculation frame in the current frame according to the automatic exposure time.

[0055] In step S110, the light source module 110 actively emits light to the target scene, and the emitted light forms reflected light after detecting the target scene, and the photosensitive module 120 receives the reflected light to obtain the grayscale information of the target scene image. The above describes in detail the working process of the TOF imaging system 100 to obtain the target scene image and the functions of each hardware in the working process, which will not be repeated here.

[0056] The TOF imaging system 100 can acquire the target scene image at a certain frame rate, and as the frame rate increases, the TOF imaging system 100 can generate a three-dimensional image of the target scene in real time. For example, a device embedded in the TOF imaging system 100, such as a camera, can be mounted on a mobile robot, and based on SLAM (Simultaneous localization and mapping) technology, the navigation and obstacle avoidance functions of the mobile robot are realized.

[0057] Each frame (e.g., the current frame and the next frame) can be divided into a calculation frame and multiple subframes after the calculation frame. The calculation frame may have grayscale information of the target scene image of the current frame. Multiple subframes may have a set of similar measurement values, and a set of similar measurement values ​​can be used to obtain depth information of the target scene image of the current frame. It should be understood that the current frame described in the present application can be any frame of multiple frames of target scene images continuously generated by the TOF imaging system 100 during operation.

[0058] In some embodiments, the emitted light of the TOF imaging system 100 may be a periodic emitted light signal with a preset wavelength, such as a sinusoidal light signal. At this time, as described above in the distance measurement principle, the phase difference between the emitted light signal and the reflected light signal can be used to indirectly calculate the round trip time of the light signal, thereby calculating the distance between the TOF imaging system 100 and the target scene. In other words, the target scene image information obtained at different phases by the reflected light signal can be used to solve the depth information of the target scene image. Therefore, the target scene image information at multiple phase frames (subframes) can be collected by exposing at different phases of the periodic reflected light signal with a certain exposure time, and the depth information of the target scene image can be solved. For a sinusoidal light signal, four measured values ​​of the reflected light signal with a phase interval of 90 degrees can be collected, so that the current frame can solve the target scene depth information through the measured values ​​of the four subframes. In other words, the multiple subframes after the calculation frame of the current frame can be four subframes.

[0059] In some embodiments, the calculation frame may include a valid frame for obtaining the grayscale of the target scene image and an invalid frame after the calculation frame. The grayscale information obtained in the calculation frame needs to be further processed to calculate the automatic exposure time for the current frame. Since the emission light signal usually needs to undergo high-frequency modulation, the time interval between the valid frame and multiple subframes in the calculation frame is very short. Adding an invalid frame after the valid frame and before the multiple subframes can reserve the time required for the calculation for step S120.

[0060] In some embodiments, the exposure time in the invalid frame may be zero. In other words, during the invalid frame, the photosensitive module 120 does not perform an exposure operation, and the invalid frame may serve as a time interval between a valid frame and subsequent multiple subframes.

[0061] In some embodiments, the exposure time in the invalid frame may be a non-zero constant. The invalid frame may not perform the exposure operation using the automatic exposure time of the current frame calculated by using the grayscale information obtained by the valid frame, but may perform the exposure operation using a preset fixed exposure time. With this automatic exposure method, the target scene image information collected by the invalid frame can be used for the detection of ambient light. At the same time, sufficient time required for the calculation can be reserved for step S120.

[0062] In addition, the method of obtaining the grayscale information of the target scene can be to set the TOF imaging system 100 to the grayscale mode, so that the two-dimensional image information of the target scene image is collected in the calculation frame, and the grayscale information of the target scene image of the calculation frame of the current frame is obtained by using the two-dimensional image information. Optionally, when the TOF imaging system emits a periodic light signal, such as a sinusoidal light signal, the reflected light signal reflected by the target scene can also be periodic, and because the reflectivity of each object in the target scene is different, the amplitude of the reflected light signal is different. Therefore, the grayscale information of the target scene image of the calculation frame of the current frame can be obtained based on the amplitude information of the target scene image of the calculation frame of the current frame.

[0063] In step S120, the grayscale information of the image may be a set of grayscale values ​​of each pixel in the image. The grayscale value may be a number of levels divided into logarithmic levels between white and black, generally ranging from 0 to 255, where white is 255 and black is 0.

[0064] In order to ensure sufficient stable measurement accuracy when a dynamic target scene has a large distance span, the exposure time can be controlled. The exposure condition can be visually manifested as a change in grayscale. High exposure results in a higher grayscale and a brighter image; low exposure results in a lower grayscale and a darker image. Therefore, the grayscale information of the target scene image of the calculation frame of the current frame can be applied to the processing unit 132 in the TOF imaging system 100 to determine an automatic exposure time that matches it.

[0065] In some embodiments, the average value of the grayscale value of each pixel of the image can be used as a reference for calculating the automatic exposure time, thereby calculating the automatic exposure time of the current frame. It should be understood that the automatic exposure time calculation method of the embodiment of the present application should not be limited to this, and other automatic exposure time calculation methods such as local sampling method can also be used to calculate the automatic exposure time of the current frame.

[0066] In step S130 , according to the automatic exposure time of the current frame calculated in step S120 , the control unit 131 of the working module 130 adjusts the exposure time of each of the multiple subframes of the current frame.

[0067] In some embodiments, the automatic exposure time calculated in step S120 can also be used to perform an exposure operation on a valid frame in a subsequent frame, so that the exposure time of the valid frame in each frame can be variable. Specifically, the exposure time of the valid frame in the subsequent frame can be adjusted according to the grayscale information of the target scene image obtained by the valid frame in the current frame, so that the grayscale information of the target scene image of the subsequent frame can be obtained for the valid frame in the subsequent frame, providing a more accurate exposure time.

[0068] In some implementations, there may be a time interval between a current frame and a subsequent frame, so that the TOF imaging system 100 can maintain a preset frame rate (eg, 10 fps).

[0069] In some embodiments, when the exposure procedure is performed on multiple subframes, each subframe may be adjusted according to the same automatic exposure time obtained in step S120. This is beneficial to the control of the exposure time of multiple subframes in the current frame, and the use of the same automatic exposure time is also beneficial to the accuracy of the measurement values ​​collected by each subframe.

[0070] Figure 4 is the current frame F according to the implementation mode of the present application k Format next frame F k+1 Format diagram. Figure 4 As shown, the current frame F k It may include calculating frame F c and four subframes (a first subframe F1, a second subframe F2, a third subframe F3, and a fourth subframe F4). The TOF imaging system is in the complete current frame F k The target scene image information for generating a three-dimensional image can be obtained in the frame F. The target scene image information may include the depth information D and the grayscale information G of the image. c In the image processing module 120, the grayscale information G of the target scene image can be obtained; then, the grayscale information G can be transmitted to the processing unit 132 in the working module 130, and an algorithm such as the global grayscale value average is used to calculate the grayscale information G according to the frame F.c The grayscale information G generated in the image is used to calculate the automatic exposure time E suitable for the current frame. k .

[0071] For example, the calculation frame F c Further divided into valid frames F a and the following invalid frame F na In the valid frame F a In the image processing, the photosensitive module 120 can obtain the grayscale information G of the target scene image, and the invalid frame F na It can be used to calculate the F suitable for the current frame k Automatic exposure time E k Provide a time interval.

[0072] For example, the current frame F k And the next F k+1 There may be a time interval T between them, so that the TOF imaging system 100 can maintain a certain frame rate when it is in a working state.

[0073] In the first subframe F1 to the fourth subframe F4, the image sensor in the photosensitive module 120 can be automatically exposed during the automatic exposure time E. k The four measured values ​​used to calculate the depth information D of the target scene image are obtained by exposing the four measured values. The depth information D of the target scene can be obtained by calculating the four measured values. c The grayscale information G of the target scene image obtained in the complete current frame F k Within, a three-dimensional image of the target scene is generated through the processing unit 132 of the working module 130.

[0074] It should be understood that the TOF imaging system 100 can continuously generate a three-dimensional image of the target scene in a frame period when it is working, and the format of each frame can be the same. In this way, the above command is repeatedly executed in each frame to control the photosensitive module 120 to obtain the target scene image information in an automatic exposure manner, so as to realize the automatic exposure function of the TOF imaging system 100.

[0075] Figure 5 is the current frame F' k Format and next frame F' k+1 Format diagram. Figure 5 As shown, in the existing automatic exposure method, in the current frame F' k In the case of four subframes (a first subframe F'1, a second subframe F'2, a third subframe F'3 and a fourth subframe F'4), that is, when the target scene image information (grayscale information G and depth information D) of the current frame is obtained through four subframes, the current frame F' k Automatic exposure time E'k It needs to be determined based on the grayscale information of the target scene image information of the previous frame (not shown). In other words, the current frame F' k The grayscale information G' of the target scene image information is used to determine the next frame F' k+1 Automatic exposure time E' k+1 In addition, in the current frame F' k and the next frame F' k+1 Set the time interval T' to calculate the next frame F' k+1 Automatic exposure time E' k Sufficient computing time is provided, and at the same time, the TOF imaging system 100 can maintain a certain frame rate during normal operation.

[0076] According to the existing automatic exposure method, the current frame F' k Automatic exposure time E' k According to the previous frame F' k-1 This will result in the automatic exposure time E' k The grayscale information G' of the acquired target scene image is delayed in time. For example, when the TOF imaging system continuously generates a three-dimensional image of the target scene at 10fps, the current frame F' k Automatic exposure time E' k It lags behind the grayscale information G' of the acquired target scene image by 100ms.

[0077] When the device embedded with the TOF imaging system 100 is mounted on a mobile robot for obstacle avoidance and automatic navigation, it is necessary to continuously generate a three-dimensional image of the target scene. When the mobile robot (TOF imaging system) switches quickly with the target scene, such as when an obstacle actively approaches, the robot rotates quickly, or when the frame rate of the TOF imaging system is low, and the time interval between the current frame and the next frame is long, the TOF imaging system will obtain a large difference in the target scene image in adjacent frames. In this way, the automatic exposure time of the current frame determined based on the grayscale information of the target scene image obtained in the previous frame is not suitable for exposure in the current frame. In other words, if the grayscale information of the target scene image of the previous frame is used to determine the automatic exposure time of the current frame, the target scene image generated by the current frame may be overexposed or underexposed, and the accuracy of the depth information of the target scene image may be affected.

[0078] The automatic exposure method provided by the present application can apply the automatic exposure time determined according to the grayscale information of the target scene image of the current frame to the current frame for exposure. This can avoid the time delay when executing the automatic exposure time of the current frame, so that the target scene image of the current frame can obtain correct exposure, and is conducive to improving the accuracy of the depth information of the target scene image of the current frame.

[0079] Figure 6 3 is a schematic diagram of the structure of an electronic device 300 suitable for implementing the automatic exposure method of the embodiment of the present application. It should be understood that, Figure 6 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0080] like Figure 6 As shown, the electronic device 300 includes one or more processors 301 (e.g., CPU), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 302 or the program loaded from the storage part 306 into the random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processor 301, the ROM 302, and the RAM 303 are connected to each other through a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0081] The following components are connected to the I / O interface 305: a storage section 306 such as a hard disk, etc.; and a communication section 307 including a network interface card such as a LAN card, a modem, etc. The communication section 307 performs communication processing via a network such as the Internet. A drive 308 is also connected to the I / O interface 305 as needed. A removable medium 309, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 308 as needed, so that a computer program read therefrom is installed into the storage section 306 as needed.

[0082] A person skilled in the art can understand that all or part of the steps in the method for implementing the above-mentioned embodiment can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes the following steps: S110, obtaining the grayscale information of the target scene image of the calculation frame in the current frame; S120, calculating the automatic exposure time according to the obtained grayscale information; and S130, adjusting the exposure time of multiple subframes after the calculation frame in the current frame according to the automatic exposure time. The storage medium includes, for example, ROM / RAM, a disk, an optical disk, etc.

[0083] The computer program may be divided into a plurality of modules, which are stored in the memory and executed by the processor. The plurality of modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the automatic exposure device. For example, the computer program may be divided into an acquisition module, a calculation module, and an adjustment module, and the specific functions of each module are as follows:

[0084] The acquisition module is used to obtain the grayscale information of the target scene image of the calculation frame in the current frame.

[0085] A calculation module, used to calculate the automatic exposure time according to the acquired grayscale information; and

[0086] The adjustment module is used to adjust the exposure time of multiple subframes after the calculation frame in the current frame according to the automatic exposure time.

[0087] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other.

Claims

1. An automatic exposure method based on a TOF imaging system, characterized in that: include: Obtain the grayscale information of the target scene image of the calculation frame in the current frame; Calculating automatic exposure time according to the acquired grayscale information; as well as According to the automatic exposure time, adjusting the exposure time of a plurality of subframes after the calculation frame in the current frame, wherein the calculation frame and the plurality of subframes all belong to the current frame; The calculation frame is used to collect two-dimensional image information of the target scene image, and the multiple sub-frames are used to obtain depth information of the target scene image of the current frame.

2. The automatic exposure method according to claim 1, characterized in that: The calculation frame includes a valid frame and an invalid frame after the valid frame, wherein the step of obtaining the grayscale information includes: Obtain grayscale information of the target scene image of the valid frame in the calculation frame.

3. The automatic exposure method according to claim 2, characterized in that: The exposure time in the invalid frame is zero.

4. The automatic exposure method according to claim 2, characterized in that: The exposure time in the invalid frame is a non-zero constant.

5. The automatic exposure method according to any one of claims 1 to 4, characterized in that: The step of obtaining the grayscale information includes: According to the amplitude information of the target scene image of the calculation frame in the current frame, the grayscale information of the calculation frame in the current frame is obtained.

6. The automatic exposure method according to any one of claims 1 to 4, characterized in that: The grayscale information of the target scene image includes grayscale values ​​of a plurality of pixels of the target scene image, wherein the step of calculating the automatic exposure time includes: The automatic exposure time is calculated according to an average value of the grayscale values ​​of the plurality of pixels.

7. The automatic exposure method according to any one of claims 1 to 4, characterized in that: Also includes: The exposure time of the calculation frame in the subsequent frame is adjusted according to the automatic exposure time.

8. The automatic exposure method according to any one of claims 1 to 4, characterized in that: The multiple subframes after the calculation frame in the current frame are four subframes.

9. The automatic exposure method according to any one of claims 1 to 4, characterized in that: The step of adjusting the exposure time of the plurality of subframes comprises: Each subframe in the plurality of subframes is adjusted according to the same automatic exposure time.

10. The automatic exposure method according to claim 7, characterized in that: There is a time interval between the current frame and the next frame.

11. An automatic exposure device, characterized in that An acquisition module, used for acquiring grayscale information of a target scene image of a calculation frame in a current frame; A calculation module, used for calculating the automatic exposure time according to the acquired grayscale information; as well as an adjustment module, configured to adjust the exposure time of a plurality of subframes following the calculation frame in the current frame according to the automatic exposure time, wherein the calculation frame and the plurality of subframes both belong to the current frame; The calculation frame is used to collect two-dimensional image information of the target scene image, and the multiple sub-frames are used to obtain depth information of the target scene image of the current frame.

12. The automatic exposure device according to claim 11, characterized in that: The calculation frame includes a valid frame and an invalid frame following the valid frame, and the grayscale information comes from a target scene image of the valid frame in the calculation frame.

13. The automatic exposure device according to claim 12, characterized in that: The exposure time in the invalid frame is zero.

14. The automatic exposure device according to claim 12, characterized in that: The exposure time in the invalid frame is a non-zero constant.

15. The automatic exposure device according to claim 11, characterized in that: The adjustment module is further used to adjust the exposure time of the calculation frame in the subsequent frame according to the automatic exposure time.

16. An electronic device, characterized in that include: Memory, used to store programs; A processor, configured to execute the method according to any one of claims 1 to 10 when executing the program.

17. A computer readable medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor, the method according to any one of claims 1 to 10 is performed.

18. A TOF imaging system, characterized in that: include: A light source module, used for emitting light to a target scene; A photosensitive module, used for receiving the light reflected by the target scene; as well as A working module is used to control the light source module and the photosensitive module, and process the reflected light received by the photosensitive module to execute the method according to any one of claims 1 to 10.

Images