A method for setting exposure time based on TOF sensor and related device

Abstract

The present application provides an exposure time setting method based on a TOF sensor and a related device, the exposure time setting method includes: obtaining the first exposure time of the TOF sensor to collect the current frame image; if the first exposure time is within the preset standard exposure time interval, obtaining the original phase image obtained based on the first exposure time; counting the over-exposed pixel ratio of the line light area corresponding to the original phase image; according to the comparison relationship between the over-exposed pixel ratio and the standard over-exposed pixel ratio threshold, setting the second exposure time of the TOF sensor to collect the next frame image. Through the implementation of the scheme of the present application, the current frame exposure time is automatically adjusted in real time based on the over-exposed pixel ratio of the line light area in the original phase image, and the dynamic setting of the next frame exposure time is realized, which ensures the real-time setting of the exposure time and is fully applicable to dynamic scenes.

Description

Technical Field

[0001] The present application relates to the technical field of TOF sensors, and in particular to an exposure time setting method based on a TOF sensor and a related device. Background Art

[0002] Time of Flight (TOF) is a technology that calculates the distance of a measured object by emitting a modulated light beam to the measured object and receiving the light beam reflected by the measured object, thereby detecting the flight (round trip) time of the emitted light beam. Among them, the line light beam generated by TOF based on a line light source can make the detection distance longer, and there is less crosstalk and multipath interference between line light beams.

[0003] However, since the sensor used in TOF technology is essentially a photosensitive element, its exposure performance has a crucial impact on image quality. In practical applications, if the exposure time of the sensor pixel is too long, overexposure will occur, resulting in an overly bright image. Conversely, if the exposure time of the sensor pixel is too short, underexposure will occur, resulting in an overly dark image. In related technologies, a technical solution is usually adopted to adjust the exposure time based on the deviation between the average amplitude of the current frame and the optimal amplitude. However, in dynamic scenes, it is difficult to obtain the optimal amplitude in real time, which leads to significant limitations on the application scenarios of this technology. In addition, the real-time performance of the exposure time adjustment of this technology cannot be guaranteed. Summary of the invention

[0004] The embodiment of the present application provides an exposure time setting method and related device based on a TOF sensor, which can at least solve the problem that the exposure time setting solution provided in the related art is not fully applicable to dynamic scenes and cannot meet the real-time requirements of exposure time adjustment.

[0005] In a first aspect, an embodiment of the present application provides an exposure time setting method based on a TOF sensor, which is applied to a TOF depth camera, including: obtaining a first exposure time for the TOF sensor to collect a current frame of an original phase image; determining whether the first exposure time is within a preset standard exposure time interval, and if the first exposure time is within the preset standard exposure time interval, obtaining an original phase image based on the first exposure time; counting the proportion of over-exposed pixels in a line light area corresponding to the original phase image; wherein the line light area is a pixel area that responds to a target reflected light signal; comparing the relationship between the over-exposed pixel ratio and a standard over-exposed pixel ratio threshold, and adaptively setting a second exposure time for the TOF sensor to collect a next frame of image according to the comparison relationship.

[0006] According to a second aspect of an embodiment of the present application, there is provided an exposure time setting device based on a TOF sensor, which is applied to a TOF depth camera, and includes: an acquisition module, used to acquire a first exposure time for the TOF sensor to collect a current frame of the original phase image; a judgment module, used to judge whether the first exposure time is within a preset standard exposure time interval, and if the first exposure time is within the preset standard exposure time interval, then obtaining the original phase image obtained based on the first exposure time; a statistical module, used to count the proportion of over-exposed pixels in the line light area corresponding to the original phase image; wherein the line light area is a pixel area that responds to the target reflected light signal; a setting module, used to compare the relationship between the over-exposed pixel ratio and the standard over-exposed pixel ratio threshold, and adaptively set the second exposure time for the TOF sensor to collect the next frame of the image according to the comparison relationship.

[0007] A third aspect of an embodiment of the present application provides a TOF depth camera, comprising: a transmitting end, a receiving end and a processor, wherein: the transmitting end is used to transmit an infrared light beam with phase information to the object to be measured; the receiving end includes a TOF sensor, which is used to collect the light beam reflected back by the object to be measured and generate an original phase image to be transmitted to the processor; the processor is used to obtain a first exposure time corresponding to the original phase image, and use the exposure time setting method provided in the first aspect of the embodiment of the present application to process the first exposure time to achieve adaptive adjustment of the second exposure time of the TOF sensor to collect the next frame of image.

[0008] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, each step in the exposure time setting method provided in the first aspect of the embodiment of the present application is implemented.

[0009] As can be seen from the above, according to the exposure time setting method and related devices provided by the present application, the first exposure time of the TOF sensor for collecting the current frame image is obtained; if the first exposure time is within the preset standard exposure time interval, the original phase image obtained based on the first exposure time is obtained; the over-exposed pixel ratio of the line light area corresponding to the original phase image is counted; according to the comparison relationship between the over-exposed pixel ratio and the standard over-exposed pixel ratio threshold, the second exposure time of the TOF sensor for collecting the next frame image is set. Through the implementation of the present application, the current frame exposure time is automatically adjusted in real time based on the over-exposed pixel ratio of the line light area in the original phase image, and the adaptive dynamic setting of the next frame exposure time is realized, which ensures the real-time setting of the exposure time and is fully suitable for dynamic scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A schematic diagram of the structure of a TOF depth camera provided in the fourth embodiment of the present application;

[0011] Figure 2 A schematic diagram of the basic flow of the exposure time setting method provided in the first embodiment of the present application;

[0012] Figure 3 A schematic diagram of a binarized infrared intensity image provided in the first embodiment of the present application;

[0013] Figure 4 A schematic diagram of a detailed process of the exposure time setting method provided in the second embodiment of the present application;

[0014] Figure 5 A schematic diagram of a program module of an exposure time setting device provided in the third embodiment of the present application. DETAILED DESCRIPTION

[0015] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0016] Figure 1 The present invention is a schematic diagram of the structure of a TOF depth camera provided according to the present application. In one embodiment, the TOF depth camera includes a transmitter, a receiver, and a processor; wherein the transmitter is used to transmit an infrared light beam with phase information to the object to be measured; the receiver collects the light beam reflected by the object to be measured to generate a rawphase image and transmits it to the processor; the processor is used to calculate the flight time of the transmitted light beam according to the rawphase image and obtain depth information based on the flight time.

[0017] In one embodiment, the receiving end includes a TOF sensor, which collects light signals reflected by the target and outputs a raw phase image (ie, a rawphase image) containing depth information of the measured object, wherein the rawphase image is the raw data obtained by the TOF sensor converting the collected light signals into digital signals.

[0018] Preferably, the TOF sensor includes multiple pixels, each pixel includes at least one tap (i.e., tap, for storing charge), each tap accumulates charge within a preset exposure time, in one embodiment, each pixel includes 3 taps, and the charge amounts collected by the 3 taps within the preset exposure time are A1, A2 and A3, respectively. Then, the charge amount collected by each pixel within the exposure time is A1+A2+A3, and the charge amount collected by each pixel represents the light intensity of the reflected light signal collected by the pixel.

[0019] Based on this, when the exposure time of a pixel is too long, overexposure will occur, resulting in a rawphase image that is too bright; and when the exposure time of a pixel is too short, underexposure will occur, resulting in a rawphase image that is too dark. In order to solve the problem that the exposure time setting scheme provided in the related art cannot be fully applied to dynamic scenes and cannot meet the real-time requirements of exposure time adjustment, in the TOF depth camera, the processor processes the current frame raw phase image acquired by the TOF sensor according to the exposure time setting method provided in one or more embodiments of the present application to adaptively adjust the exposure time of the TOF sensor to collect the next frame of image.

[0020] In one embodiment, the TOF depth camera further includes a memory for storing an exposure time setting program corresponding to the exposure time setting method provided in one or more embodiments of the present application, so that the processor can read the program when processing the image acquired by the TOF sensor, thereby adaptively adjusting the exposure time of the next frame of image collected by the TOF sensor. It should be noted that the memory can be independently set or integrated with the processor, and it can be a high-speed random access memory (RAM) memory or a non-volatile memory, such as a disk memory, which is not limited here.

[0021] Figure 2 The following is a basic flow chart of a method for setting exposure time based on a TOF sensor provided in the present application. The method for setting exposure time includes the following steps:

[0022] Step 201: Obtain a first exposure time of a TOF sensor for collecting an original phase image of a current frame.

[0023] Specifically, the TOF sensor of this embodiment includes rolling exposure or global exposure; wherein, when the TOF sensor is in rolling exposure mode, during the acquisition process of each frame of image, the time difference between the exposure start time of the first row of pixels of the TOF sensor and the exposure end time of the last row of pixels corresponds to the exposure time of the single frame image.

[0024] Step 202: determine whether the first exposure time is within a preset standard exposure time interval. If the first exposure time is within the preset standard exposure time interval, obtain an original phase image based on the first exposure time.

[0025] Specifically, the TOF depth camera of this embodiment is preset with an exposure time range, which is also the standard exposure time interval [t min , t max], if the first exposure time is within the preset standard exposure time interval, it means that the exposure time of the TOF sensor when collecting the current frame of the original phase image is adjustable. On the contrary, if the first exposure time is outside the preset standard exposure time interval, the second exposure time for the TOF sensor to collect the next frame of image is set with reference to the endpoint value of the standard exposure time interval; preferably, when the first exposure time is less than the lower limit value of the standard exposure time interval, the lower limit value is set as the second exposure time for the TOF sensor to collect the next frame of image; when the first exposure time is greater than the upper limit value of the standard exposure time interval, the upper limit value of the duration is set as the second exposure time for the TOF sensor to collect the next frame of image.

[0026] Step 203: Count the proportion of overexposed pixels in the line light area corresponding to the original phase image.

[0027] Specifically, the line light area of ​​this embodiment is a pixel area that responds to the target reflected light signal. In this embodiment, a standard overexposure threshold of a pixel brightness can be pre-set according to the environment in which the TOF depth camera is located. If the actual brightness value of the pixel exceeds the standard overexposure threshold, the pixel is an overexposed pixel. It should be noted that the overexposed pixel ratio of this embodiment can be understood as the ratio of the number of overexposed pixels in the line light area to the total number of pixels in the line light area.

[0028] In an optional implementation of the present embodiment, before the above-mentioned step of counting the proportion of overexposed pixels in the line light area corresponding to the original phase image, it also includes: acquiring an infrared intensity image corresponding to the original phase image; binarizing the infrared intensity image, and obtaining a connected area with pixel values ​​of the target value based on the binarization result, and the connected area is the line light area corresponding to the infrared intensity image; mapping the line light area corresponding to the infrared intensity image to the original phase image to obtain the line light area corresponding to the original phase image.

[0029] Specifically, there is a linear relationship between the size of the rawphase image and the infrared intensity image, and the pixel value of a pixel in the infrared intensity image is determined by the amount of charge stored in the taps included in the pixel. More specifically, the pixel value of the corresponding pixel in the infrared intensity image can be obtained by summing the amount of charge stored in each tap in each pixel. All pixels of the TOF sensor are traversed, and the amount of charge stored in all taps in each pixel is summed to obtain the infrared intensity image.

[0030] In this embodiment, binarization of the infrared intensity image specifically includes: assigning values ​​to different types of pixels in the infrared intensity image using a first value (e.g., 0) and a second value (e.g., 1). It should be noted that the binary values ​​include but are not limited to 0 and 1, and in actual applications, the binary values ​​can be flexibly set according to application requirements.

[0031] Figure 3 FIG. 1 is a schematic diagram of a binarized infrared intensity image provided by the present embodiment. In one example, the present embodiment may assign a pixel value corresponding to a pixel that responds to a light signal reflected by a target to 1, and assign a pixel value corresponding to a pixel that does not respond to a reflected light signal to 0, thereby obtaining a binarized infrared intensity image. The connected area (i.e., the connected area) whose pixel value corresponds to the target value of the present embodiment, i.e., 1, is calculated. Figure 2 The line light region of the infrared intensity image can be obtained by mapping the line light region on the infrared intensity image to the rawphase image. The connected region is the image region composed of pixels with the same pixel value and adjacent positions in the infrared intensity image. Furthermore, since there is a linear relationship between the sizes of the rawphase image and the infrared intensity image, after obtaining the line light region on the infrared intensity image, the line light region is mapped to the rawphase image to obtain the line light region of the rawphase image.

[0032] Step 204 : compare the relationship between the over-exposed pixel ratio and the standard over-exposed pixel ratio threshold, and adaptively set a second exposure time for the TOF sensor to capture the next frame of image according to the comparison relationship.

[0033] Specifically, this embodiment compares the over-exposed pixel ratio of the line light area of ​​the rawphase image with the standard over-exposed pixel ratio threshold, and adaptively adjusts the exposure time of the TOF sensor to collect the next frame of image according to the comparison relationship.

[0034] In some embodiments, if the overexposed pixel ratio is greater than the standard overexposed pixel ratio threshold, the adjustment step size is calculated according to the overexposed pixel ratio; and the second exposure time for the TOF sensor to capture the next frame of image is set in combination with the adjustment step size and the first exposure time. It is worth noting that the adjustment step size can be understood as the adjustment ratio.

[0035] In one embodiment, before the step of comparing the relationship between the over-exposed pixel ratio and the standard over-exposed pixel ratio threshold, the step further includes: comparing the first exposure time with the lower limit of the standard exposure time interval, that is, determining whether the exposure time of the current frame image is the minimum exposure time t in the standard exposure time interval. min If so, a warning is issued, warning that the current exposure time is the minimum value, but overexposure still exists, and a corresponding adjustment prompt is issued, prompting to increase the standard overexposure threshold of the pixel or reduce the minimum exposure time of the standard exposure time interval and then re-execute the above steps.

[0036] Further optionally, if the exposure time of the current frame image is not the minimum exposure time in the standard exposure time interval, the over-exposed pixel ratio can also be compared with a preset over-exposed pixel ratio threshold. The value of the over-exposed pixel ratio is greater than the standard over-exposed pixel ratio threshold, and preferably can be 2 times the standard over-exposed pixel ratio threshold or above. If the over-exposed pixel ratio is greater than the over-exposed pixel ratio threshold, it means that a large area in the current image is overexposed and the exposure time needs to be quickly reduced. The first exposure time can be reduced (such as reduced to half or less than half of the current exposure time), and the exposure time of the next frame image collected by the TOF sensor can be adaptively adjusted to quickly reduce the exposure of a large area in the image.

[0037] Furthermore, if the first exposure time is greater than the lower limit of the standard exposure time interval, a step of comparing the relationship between the over-exposed pixel ratio and a standard over-exposed pixel ratio threshold is performed.

[0038] In one embodiment, if the overexposed pixel ratio is greater than the standard overexposed pixel ratio threshold, the step of setting the second exposure time for the TOF sensor to collect the next frame of image in combination with the adjustment step length and the first exposure time includes: setting the second exposure time for the TOF sensor to collect the next frame of image by a preset first time length calculation formula in combination with the adjustment step length and the first exposure time; the first time length calculation formula is expressed as:

[0039]

[0040] t out =tt*t step

[0041] Among them, t step represents the adjustment step size, represents the preset standard adjustment step range, R represents the overexposed pixel ratio, and R t Indicates the standard overexposed pixel ratio threshold, Indicates that the over-exposure pixel ratio threshold is exceeded. The over-exposure pixel ratio threshold is greater than the standard over-exposure pixel ratio threshold. t indicates the first exposure time. t out Indicates the second exposure time.

[0042] From the above formula, we can see that the adjustment step size t step It is related to the overexposed pixel ratio R and may change with the overexposed pixel ratio R. The larger the overexposed pixel ratio is, the larger the adjustment step size is.

[0043] In other embodiments, the step of setting the second exposure time for the TOF sensor to capture the next frame of image based on the comparison relationship between the overexposed pixel ratio and the standard overexposed pixel ratio threshold value further includes: if the overexposed pixel ratio is less than the standard overexposed pixel ratio threshold value, then counting the proportion of overdark pixels in the corresponding line light area of ​​the original phase image; and setting the second exposure time for the TOF sensor to capture the next frame of image based on the comparison relationship between the overdark pixel ratio and the standard overdark pixel ratio threshold value.

[0044] Specifically, in this embodiment, when the overexposed pixel ratio in the line light area of ​​the rawphase image is less than the standard overexposed pixel ratio threshold, this embodiment pre-sets a standard over-dark threshold of the pixel brightness according to the environment in which the TOF depth camera is located. If the pixel brightness is lower than the standard over-dark threshold, it means that the pixel is an over-dark pixel, and then compares the brightness of each pixel in the line light area of ​​the rawphase image with the standard over-dark threshold, and counts the number of over-dark pixels in the line light area whose brightness is lower than the standard over-dark threshold, and finally calculates the ratio of the over-dark pixels to the total number of pixels in the line light area to obtain the over-dark pixel ratio. Further, the over-dark pixel ratio is compared with the standard over-dark pixel ratio threshold, and the exposure time corresponding to the next frame of the image is set according to the comparison relationship.

[0045] In one implementation, if the ratio of overly dark pixels is less than the standard overly dark pixel ratio threshold, the first exposure time is set as the second exposure time for the TOF sensor to collect the next frame of image. That is, the ratio of overly dark pixels in the line light area is small, indicating that the image brightness is just right, and the current exposure time can be kept unchanged, and the first exposure time is used as the second exposure time for the TOF sensor to collect the next frame of image.

[0046] In an optional implementation, if the overexposed pixel ratio is less than the standard overexposed pixel ratio threshold, before the step of calculating the adjustment step size according to the overdark pixel ratio, the step further includes: comparing the first exposure time with the upper limit of the standard exposure time interval, that is, determining whether the exposure time of the current frame image is the maximum exposure time t in the standard exposure time interval. max If so, a warning is issued, warning that the current exposure time is the maximum value, but there is still a dark situation, and a corresponding adjustment prompt is issued, prompting to lower the standard overexposure threshold of the pixel or increase the maximum exposure time of the standard exposure time interval and then re-execute the above steps.

[0047] Furthermore, if the first exposure time is less than the upper limit of the time length, a step of calculating the adjustment step length according to the proportion of overly dark pixels is executed.

[0048] In an optional implementation of this embodiment, if the overexposed pixel ratio is less than the standard overexposed pixel ratio threshold, the step of setting the second exposure time for the TOF sensor to collect the next frame of image in combination with the adjustment step size and the first exposure time includes: setting the second exposure time for the TOF sensor to collect the next frame of image by a preset second time length calculation formula in combination with the adjustment step size and the first exposure time; the second time length calculation formula is expressed as:

[0049]

[0050]

[0051] t out =t+t*t step

[0052] Among them, t step represents the adjustment step size, represents the preset standard adjustment step interval, t represents the first exposure time, [t min , t max ] represents the standard exposure time interval, ir t Represents the pixel brightness threshold of the infrared intensity image corresponding to the original phase image, ir m represents the mean pixel brightness of the line light area corresponding to the infrared intensity image, t out Indicates the second exposure time.

[0053] It can be seen from the above formula that the adjustment step size takes into account both the exposure time and the brightness of the infrared intensity image, and can adjust the exposure time adaptively in real time. Based on the above formula, when the brightness of the infrared intensity image is constant, the smaller the exposure time, the larger the adjustment step size; when the exposure time is constant, the smaller the average brightness of the infrared intensity image, the larger the adjustment step size.

[0054] Based on the technical solution of the embodiment of the present application, the first exposure time of the TOF sensor for collecting the current frame image is obtained; if the first exposure time is within the preset standard exposure time interval, the original phase image obtained based on the first exposure time is obtained; the over-exposed pixel ratio of the line light area corresponding to the original phase image is counted; according to the comparison relationship between the over-exposed pixel ratio and the standard over-exposed pixel ratio threshold, the second exposure time of the TOF sensor for collecting the next frame image is set. Compared with the prior art, through the implementation of the solution of the present application, the current frame exposure time is automatically adjusted in real time based on the over-exposed pixel ratio of the line light area in the original phase image, and the dynamic setting of the next frame exposure time is realized, which ensures the real-time setting of the exposure time and is fully applicable to dynamic scenes.

[0055] Figure 4 The method in is a refined exposure time setting method provided in the second embodiment of the present application, and the exposure time setting method includes:

[0056] Step 401: Obtain a first exposure time of a TOF sensor for capturing a current frame image.

[0057] For the rolling shutter exposure mode, the time difference between the exposure start time of the first row of pixels of the TOF sensor and the exposure end time of the last row of pixels is the exposure time of a single frame image.

[0058] Step 402 , determining whether the first exposure time is within a preset standard exposure time interval; if so, executing step 403 , if not, executing step 411 .

[0059] In this embodiment, the standard exposure time interval is also a standard exposure time interval [t min , t max ].

[0060] Step 403: Acquire an original phase image obtained based on the first exposure time, and acquire an infrared intensity image corresponding to the original phase image.

[0061] The original phase image of this embodiment, that is, the rawphase image, is the original data obtained by converting the collected light signal into a digital signal by the TOF sensor.

[0062] Step 404: binarize the infrared intensity image, and determine the connected area whose pixel values ​​are the target value according to the binarization result as the line light area corresponding to the infrared intensity image.

[0063] In this embodiment, a first value (for example, 0) and a second value (for example, 1) are used to assign values ​​to different types of pixels in the infrared intensity image to obtain a binary infrared intensity image, and then the connected area whose pixel values ​​correspond to the target value (for example, 1) of this embodiment is calculated to obtain the line light area of ​​the infrared intensity image.

[0064] Step 405: Map the line-light region corresponding to the infrared intensity image to the original phase image to obtain the line-light region corresponding to the original phase image.

[0065] Specifically, since there is a linear relationship between the sizes of the rawphase image and the infrared intensity image, after obtaining the line light area on the infrared intensity image, the line light area can be further mapped to the rawphase image to obtain the line light area of ​​the rawphase image.

[0066] Step 406 , counting the over-exposed pixel ratio of the line light area corresponding to the original phase image, and determining whether the over-exposed pixel ratio is greater than a standard over-exposed pixel ratio threshold; if so, executing step 407 , if not, executing step 408 .

[0067] Step 407: Calculate the adjustment step length according to the overexposed pixel ratio, and set the second exposure time for the TOF sensor to capture the next frame of image by combining the adjustment step length and the first exposure time through a preset first time length calculation formula.

[0068] Specifically, the first duration calculation formula of this embodiment is expressed as:

[0069]

[0070] t out =tt*t step

[0071] Step 408: Count the percentage of overly dark pixels in the line-light region corresponding to the original phase image, and compare the percentage of overly dark pixels with a standard threshold of the percentage of overly dark pixels.

[0072] Step 409: If the over-dark pixel ratio is greater than the standard over-dark pixel ratio threshold, the adjustment step length is calculated according to the over-dark pixel ratio.

[0073] In this embodiment, for the line light area of ​​the rawphase image, if the number of pixels whose pixel brightness is lower than the standard over-dark threshold is relatively large, it means that the image brightness of the current frame is low and the exposure needs to be increased to improve the quality of the subsequently collected image.

[0074] Step 410: Set the second exposure time for the TOF sensor to capture the next frame of image by presetting the second time calculation formula in combination with the adjustment step size and the first exposure time.

[0075] In this embodiment, the second duration calculation formula is expressed as:

[0076]

[0077]

[0078] t out =t+t*t step

[0079] Step 411: Set a second exposure time for the TOF sensor to capture the next frame of image with reference to the standard exposure time interval.

[0080] Among them, when the first exposure time is less than the lower limit value of the standard exposure time interval, the lower limit value is set to the second exposure time for the TOF sensor to collect the next frame image; when the first exposure time is greater than the upper limit value of the standard exposure time interval, the upper limit value is set to the second exposure time for the TOF sensor to collect the next frame image.

[0081] It should be understood that the size of the serial number of each step in this embodiment does not mean the order of execution of the steps. The execution order of each step should be determined by its function and internal logic, and should not constitute a sole limitation on the implementation process of the embodiment of this application.

[0082] Figure 5 The third embodiment of the present application provides an exposure time setting device. The exposure time setting device can be used to implement the exposure time setting method in the above-mentioned embodiment, and the exposure time setting device mainly includes:

[0083] An acquisition module 501 is used to acquire a first exposure time of a TOF sensor for collecting an original phase image of a current frame;

[0084] A judgment module 502 is used to judge whether the first exposure time is within a preset standard exposure time interval, and if the first exposure time is within the preset standard exposure time interval, obtain an original phase image based on the first exposure time;

[0085] A statistical module 503 is used to count the ratio of overexposed pixels in the line light area corresponding to the original phase image; wherein the line light area is a pixel area that responds to the target reflected light signal;

[0086] The setting module 504 is used to compare the relationship between the over-exposed pixel ratio and the standard over-exposed pixel ratio threshold, and adaptively set the second exposure time for the TOF sensor to capture the next frame of image according to the comparison relationship.

[0087] In some implementations of this embodiment, the setting module is further used to: if the first exposure time is outside the preset standard exposure time interval, set the second exposure time for the TOF sensor to capture the next frame of image with reference to the standard exposure time interval.

[0088] Furthermore, in some implementations of the present embodiment, the setting module is specifically used to: when the first exposure time is less than the lower limit value of the standard exposure time interval, set the lower limit value to the second exposure time for the TOF sensor to collect the next frame of image; and / or, when the first exposure time is greater than the upper limit value of the standard exposure time interval, set the upper limit value to the second exposure time for the TOF sensor to collect the next frame of image.

[0089] In some implementations of the present embodiment, the exposure time setting device of the present embodiment also includes: an infrared acquisition module, used to acquire an infrared intensity image corresponding to the original phase image; binarization of the infrared intensity image, and determining, based on the binarization result, a connected area whose pixel values ​​are target values ​​as the line-light area corresponding to the infrared intensity image; mapping the line-light area corresponding to the infrared intensity image to the original phase image to obtain the line-light area corresponding to the original phase image.

[0090] In some implementations of this embodiment, the setting module is specifically used to: if the overexposed pixel ratio is greater than the standard overexposed pixel ratio threshold, calculate the adjustment step size according to the overexposed pixel ratio; and set the second exposure time for the TOF sensor to capture the next frame of image in combination with the adjustment step size and the first exposure time.

[0091] Furthermore, in some implementations of the present embodiment, when the setting module executes the above-mentioned function of calculating the adjustment step size according to the overexposed pixel ratio if the overexposed pixel ratio is greater than the standard overexposed pixel ratio threshold, it is specifically used to: if the overexposed pixel ratio is greater than the standard overexposed pixel ratio threshold, compare the first exposure time with the lower limit value of the standard exposure time interval; if the first exposure time is greater than the lower limit value, calculate the adjustment step size according to the overexposed pixel ratio.

[0092] In some implementations of this embodiment, the setting module is also used to: if the over-exposed pixel ratio is less than the standard over-exposed pixel ratio threshold, then count the over-dark pixel ratio of the corresponding line light area of ​​the original phase image; and set the second exposure time for the TOF sensor to capture the next frame of image based on the comparison relationship between the over-dark pixel ratio and the standard over-dark pixel ratio threshold.

[0093] Furthermore, in some implementations of the present embodiment, when the setting module executes the above-mentioned step of setting the second exposure time for the TOF sensor to capture the next frame of image based on the comparison relationship between the over-dark pixel ratio and the standard over-dark pixel ratio threshold, it is specifically used to: if the over-dark pixel ratio is greater than the standard over-dark pixel ratio threshold, calculate the adjustment step size according to the over-dark pixel ratio; set the second exposure time for the TOF sensor to capture the next frame of image in combination with the adjustment step size and the first exposure time; if the over-dark pixel ratio is less than the standard over-dark pixel ratio threshold, set the first exposure time as the second exposure time for the TOF sensor to capture the next frame of image.

[0094] Furthermore, in some implementations of the present embodiment, when the setting module executes the above-mentioned function of calculating the adjustment step size according to the over-dark pixel ratio if the over-dark pixel ratio is greater than the standard over-dark pixel ratio threshold, it is specifically used to: if the over-dark pixel ratio is greater than the standard over-dark pixel ratio threshold, compare the first exposure time with the upper limit value of the standard exposure time interval; if the first exposure time is less than the upper limit value, calculate the adjustment step size according to the over-dark pixel ratio.

[0095] It should be noted that the exposure time setting methods in the first and second embodiments can be implemented based on the exposure time setting device provided in this embodiment. Ordinary technicians in the relevant field can clearly understand that for the convenience and conciseness of description, the specific working process of the exposure time setting device described in this embodiment can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0096] According to the exposure time setting device provided in this embodiment, the first exposure time of the TOF sensor for collecting the current frame image is obtained; if the first exposure time is within the preset standard exposure time interval, the original phase image obtained based on the first exposure time is obtained; the over-exposed pixel ratio of the line light area corresponding to the original phase image is counted; according to the comparison relationship between the over-exposed pixel ratio and the standard over-exposed pixel ratio threshold, the second exposure time of the TOF sensor for collecting the next frame image is set. Through the implementation of the scheme of this application, the current frame exposure time is automatically adjusted in real time based on the over-exposed pixel ratio of the line light area in the original phase image, and the dynamic setting of the next frame exposure time is realized, which ensures the real-time setting of the exposure time and is fully applicable to dynamic scenes.

[0097] An embodiment of the present application further provides a computer-readable storage medium, which may be disposed in the TOF depth camera in the above embodiments, and the computer-readable storage medium may be the memory in the above embodiments.

[0098] The computer readable storage medium stores a computer program, such as the aforementioned exposure time setting program, which implements the exposure time setting method in the aforementioned embodiment when executed by the processor. Furthermore, the computer storable medium may also be a U disk, a mobile hard disk, a read-only memory (ROM), a RAM, a magnetic disk, or an optical disk, etc., which may store program codes.

[0099] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0100] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0101] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0102] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a readable storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned readable storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0103] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0104] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0105] The above is a description of the exposure time setting method based on the TOF sensor and the related devices provided in the present application. For technicians in this field, according to the ideas of the embodiments of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for setting exposure time based on a TOF sensor, applied to a TOF depth camera, characterized in that: include: Obtaining a first exposure time for the TOF sensor to collect the original phase image of the current frame; Determining whether the first exposure time is within a preset standard exposure time interval, and if the first exposure time is within the preset standard exposure time interval, acquiring an original phase image based on the first exposure time; Acquire an infrared intensity image corresponding to the original phase image; Binarization is performed on the infrared intensity image, and a connected area whose pixel values ​​are target values ​​is determined according to the binarization result as a line light area corresponding to the infrared intensity image; Mapping the line light region corresponding to the infrared intensity image to the original phase image to obtain the line light region corresponding to the original phase image; Counting the ratio of overexposed pixels in the line light area corresponding to the original phase image; wherein the line light area is a pixel area that responds to the target reflected light signal; The relationship between the over-exposed pixel ratio and the standard over-exposed pixel ratio threshold is compared, and the second exposure time for the TOF sensor to capture the next frame of image is adaptively set according to the comparison relationship.

2. The exposure time setting method according to claim 1, characterized in that: After acquiring the first exposure time of the TOF sensor to collect the original phase image of the current frame, the method further includes: If the first exposure time is outside a preset standard exposure time interval, a second exposure time for the TOF sensor to capture a next frame of image is set with reference to the standard exposure time interval.

3. The exposure time setting method according to claim 2, characterized in that: The step of setting a second exposure time for the TOF sensor to collect a next frame of image with reference to the standard exposure time interval comprises: When the first exposure time is less than the lower limit value of the standard exposure time interval, the lower limit value is set as the second exposure time for the TOF sensor to collect a next frame of image; And / or, when the first exposure time is greater than an upper limit value of the standard exposure time interval, the upper limit value is set as a second exposure time for the TOF sensor to capture a next frame of image.

4. The exposure time setting method according to any one of claims 1 to 3, characterized in that: The step of adaptively setting the second exposure time for the TOF sensor to collect the next frame of image according to the comparison relationship comprises: If the overexposed pixel ratio is greater than a standard overexposed pixel ratio threshold, calculating an adjustment step size according to the overexposed pixel ratio; In combination with the adjustment step size and the first exposure time, a second exposure time for the TOF sensor to capture a next frame of image is set.

5. The exposure time setting method according to claim 4, characterized in that: Before the step of comparing the relationship between the overexposed pixel ratio and the standard overexposed pixel ratio threshold, the method further includes: Comparing the first exposure time with a lower limit value of the standard exposure time interval; If the first exposure time is greater than the lower limit, a step of comparing the relationship between the over-exposed pixel ratio and a standard over-exposed pixel ratio threshold is performed.

6. The exposure time setting method according to claim 4, characterized in that: The step of setting a second exposure time for the TOF sensor to collect a next frame of image in combination with the adjustment step size and the first exposure time includes: By presetting the first time length calculation formula, combined with the adjustment step and the first exposure time, the second exposure time for the TOF sensor to collect the next frame of image is set; the first time length calculation formula is expressed as: t out =t-t*t step Among them, t step represents the adjustment step size, represents the preset standard adjustment step interval, R represents the overexposed pixel ratio, R t represents the standard over-exposed pixel ratio threshold, represents an over-exposure pixel ratio threshold, the over-exposure pixel ratio threshold is greater than the standard over-exposure pixel ratio threshold, t represents the first exposure time, t out represents the second exposure time.

7. The exposure time setting method according to claim 4, characterized in that: The step of adaptively setting the second exposure time for the TOF sensor to collect the next frame of image according to the comparison relationship also includes: If the overexposed pixel ratio is less than a standard overexposed pixel ratio threshold, then counting the overdark pixel ratio of the line light area corresponding to the original phase image; According to the comparison relationship between the overly dark pixel ratio and the standard overly dark pixel ratio threshold, a second exposure time for the TOF sensor to capture a next frame of image is set.

8. The exposure time setting method according to claim 7, characterized in that: The step of setting a second exposure time for the TOF sensor to collect a next frame of image according to a comparison relationship between the overly dark pixel ratio and a standard overly dark pixel ratio threshold comprises: If the overly dark pixel ratio is greater than a standard overly dark pixel ratio threshold, calculating an adjustment step size according to the overly dark pixel ratio; In combination with the adjustment step size and the first exposure time, a second exposure time for the TOF sensor to capture a next frame of image is set.

9. The exposure time setting method according to claim 8, characterized in that: The step of setting a second exposure time for the TOF sensor to collect a next frame of image according to a comparison relationship between the overly dark pixel ratio and a standard overly dark pixel ratio threshold value further includes: If the overly dark pixel ratio is less than a standard overly dark pixel ratio threshold, the first exposure time is set as a second exposure time for the TOF sensor to capture a next frame of image.

10. The exposure time setting method according to claim 8, characterized in that: Before the step of calculating the adjustment step size according to the proportion of overly dark pixels, the method further includes: Comparing the first exposure time with an upper limit value of the standard exposure time interval; If the first exposure time is less than the upper limit, the step of calculating the adjustment step size according to the proportion of too dark pixels is performed.

11. The exposure time setting method according to claim 8, characterized in that: The step of setting a second exposure time for the TOF sensor to collect a next frame of image in combination with the adjustment step size and the first exposure time includes: By presetting the second time length calculation formula, combined with the adjustment step and the first exposure time, the second exposure time for the TOF sensor to collect the next frame of image is set; the second time length calculation formula is expressed as: t out =t+t*t step Among them, t step represents the adjustment step size, represents the preset standard adjustment step interval, t represents the first exposure time, [t min , t max ] represents the standard exposure time interval, ir t represents the pixel brightness threshold of the infrared intensity image corresponding to the original phase image, ir m represents the average pixel brightness of the line light area corresponding to the infrared intensity image, t out represents the second exposure time.

12. An exposure time setting device based on a TOF sensor, applied to a TOF depth camera, characterized in that: include: An acquisition module, used to acquire a first exposure time of the TOF sensor for collecting the original phase image of the current frame; a judging module, configured to judge whether the first exposure time is within a preset standard exposure time interval, and if the first exposure time is within the preset standard exposure time interval, obtain an original phase image based on the first exposure time; An infrared acquisition module, used for acquiring an infrared intensity image corresponding to the original phase image; Binarization is performed on the infrared intensity image, and a connected area whose pixel values ​​are target values ​​is determined according to the binarization result as a line light area corresponding to the infrared intensity image; Mapping the line light region corresponding to the infrared intensity image to the original phase image to obtain the line light region corresponding to the original phase image; A statistical module, used to count the ratio of overexposed pixels in the line light area corresponding to the original phase image; wherein the line light area is a pixel area that responds to the target reflected light signal; The setting module is used to compare the relationship between the over-exposed pixel ratio and the standard over-exposed pixel ratio threshold, and adaptively set the second exposure time of the TOF sensor to collect the next frame of image according to the comparison relationship.

13. A TOF depth camera, characterized in that: It includes a transmitter, a receiver and a processor, wherein: The transmitting end is used to transmit an infrared light beam with phase information to the object to be measured; The receiving end includes a TOF sensor for collecting the light beam reflected by the measured object and generating an original phase image to transmit to the processor; The processor is used to obtain a first exposure time corresponding to the original phase image, and use the method described in any one of claims 1 to 11 to process the first exposure time to adaptively adjust the second exposure time of the TOF sensor to capture the next frame of image.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 11 are implemented.

Images