A Calibration Method, Detection Method and System Based on DToF

Abstract

This application relates to the field of optical detection technologies, and in particular, to a calibration method, a detection method, and a system based on DToF. The calibration method includes: controlling at least one light source in a transmitter to emit a speckle beam towards a target object according to a preset projection pattern, where the transmitter includes a light source array composed of multiple light sources; synchronously activating a pixel array in a collector to receive at least part of the speckle beam reflected back by the target object; at least part of the pixels in the pixel array receive the speckle beam and output photon detection signals; generating a grayscale image according to the number of the photon detection signals, and determining the calibration relationship between each light source in the light source array and the pixels in the pixel array according to the grayscale image and an expected standard speckle pattern. Embodiments of this application can eliminate the process of pre-calibration.

Description

Technical Field

[0001] This application relates to the field of optical detection technology, and in particular, to a calibration method, a detection method, and a system based on DToF. Background Art

[0002] Time of Flight (ToF) technology belongs to two-way ranging technology, which measures the distance to a target object by using the flight time of an optical signal traveling back and forth between a transmitter and a collector. Currently, detection systems based on ToF technology have been widely used in fields such as consumer electronics, unmanned driving, Augmented Reality (AR), and Virtual Reality (VR).

[0003] ToF technology can be divided into Direct ToF (DToF) technology and Indirect ToF (iToF) technology. Among them, DToF technology measures the flight time of photons in a light beam from emission to reception based on Time-Correlated Single Photon Counting (TCSPC) technology; iToF technology measures the phase delay of a reflected light beam relative to an emitted light beam, and then calculates the flight time from the phase delay. Among them, DToF technology has advantages such as high signal-to-noise ratio, good sensitivity, and high accuracy, and has received increasing attention.

[0004] Before using a detection system based on DToF technology, it is usually necessary to calibrate the correspondence between a light source and pixels in advance. However, calibration methods such as using a calibration board are usually complex.

[0005] The disclosure of the above background art content is only used to assist in understanding the concept and technical solution of this application, and it does not necessarily belong to the prior art of this application. Without clear evidence indicating that the above content was publicly available before the filing date of this application, the above background art should not be used to evaluate the novelty and inventiveness of this application. Summary of the Invention

[0006] The purpose of the embodiments of this application is to provide a detection system and method based on DToF, aiming to solve one or more technical problems in the related art.

[0007] To achieve the above object, in a first aspect, an embodiment of the present application provides a detection method based on DToF, including: S1, controlling a transmitter to emit a speckle beam towards a target according to a preset projection pattern, where the transmitter includes a light source array composed of multiple light sources; S2, synchronously activating a pixel array in a collector to receive at least part of the speckle beam reflected by the target; at least part of the pixels in the pixel array receive the speckle beam and output photon detection signals; S3, generating a grayscale image according to the number of the photon detection signals, and determining a calibration relationship between each light source in the light source array and a pixel in the pixel array according to the grayscale image and an expected standard speckle pattern.

[0008] In some embodiments, determining the calibration relationship between each light source in the light source array and a pixel in the pixel array according to the grayscale image and the expected standard speckle pattern includes:

[0009] Inputting the grayscale image into a first neural network model to obtain a speckle imaging map, and determining the calibration relationship between each light source in the light source array and a pixel in the pixel array according to a comparison result between the speckle imaging map and the expected standard speckle pattern.

[0010] In some embodiments, determining the calibration relationship between each light source in the light source array and a pixel in the pixel array according to the comparison result between the speckle imaging map and the expected standard speckle pattern includes;

[0011] If it is determined that the difference between the speckle imaging map and the expected standard speckle pattern is less than a threshold, determining the calibration relationship between each light source in the light source array and a pixel in the pixel array according to the comparison result between the speckle imaging map and the expected standard speckle pattern;

[0012] If it is determined that the difference between the speckle imaging map and the expected standard speckle pattern is greater than or equal to the threshold, obtaining a new preset projection pattern, and returning to execute steps S1, S2, and S3 until the number of times of obtaining the preset projection pattern is greater than a preset number, or until it is determined that the difference between the new speckle imaging map and the expected standard speckle pattern is less than the threshold, and determining the calibration relationship between each light source in the light source array and a pixel in the pixel array according to the comparison result between the new speckle imaging map and the expected standard speckle pattern.

[0013] In some embodiments, determining the calibration relationship between each light source in the light source array and a pixel in the pixel array according to the comparison result between the speckle imaging map and the expected standard speckle pattern includes:

[0014] Comparing the speckle imaging map with the expected standard speckle pattern to determine difference pixels corresponding to each light source in the light source array, and taking the expected imaging pixels corresponding to each light source and the corresponding difference pixels as effective pixels corresponding to the light source.

[0015] In some embodiments, determining the calibration relationship between each light source in the light source array and the pixels in the pixel array according to the comparison result of the spot imaging map and the expected standard spot map includes:

[0016] Comparing the spot imaging map with the expected standard spot map to determine the offset direction and offset amount of the actual imaging pixels of each light source in the light source array relative to the expected imaging pixels, and determining the effective pixels of each light source in the light source array according to the offset direction and the offset amount.

[0017] In a second aspect, an embodiment of the present application provides a DToF-based detection method, including: controlling at least one working light source in the transmitter to emit a speckle beam towards a target object; and controlling the activation of the working pixels corresponding to the working light sources in the collector according to the calibration relationship as described above; the working pixels receive at least part of the speckle beam reflected by the target object and output a photon detection signal to a readout circuit, and the readout circuit generates a time-of-flight value according to the photon detection signal.

[0018] In some embodiments, it further includes: calculating the distance information of the target object according to the time-of-flight value.

[0019] In a third aspect, an embodiment of the present application provides a DToF-based detection system, including: a transmitter, a collector, and a control and processing circuit,

[0020] The transmitter includes a light source array composed of a plurality of light sources, and the transmitter is configured to emit a speckle beam towards a target object by turning on at least one light source according to a preset projection pattern;

[0021] The collector includes a pixel array composed of a plurality of pixels, and the collector is configured to activate the pixel array under the control of the control and processing circuit to receive at least part of the speckle beam reflected back by the target object, and at least part of the pixels in the pixel array receive the speckle beam and output a photon detection signal;

[0022] The control and processing circuit is configured to generate a grayscale map according to the number of the photon detection signals, and determine the calibration relationship between each light source in the light source array and the pixels in the pixel array according to the grayscale map and the expected standard spot map.

[0023] In some embodiments, determining the calibration relationship between each light source in the light source array and the pixels in the pixel array according to the grayscale map and the expected standard spot map includes:

[0024] Input the grayscale image into the first neural network model to obtain a spot imaging map, and determine the calibration relationship between each light source in the light source array and the pixels in the pixel array according to the comparison result between the spot imaging map and the expected standard spot map.

[0025] In some embodiments, the control and processing circuit is further configured to control at least one working light source in the emitter to emit a speckle beam towards the target object, and synchronously control the activation of the working pixels corresponding to the working light source in the collector according to the calibration relationship; the working pixels receive at least part of the speckle beam reflected by the target object and output a photon detection signal to the readout circuit, and the readout circuit generates a time-of-flight value according to the photon detection signal.

[0026] In some embodiments, the control and processing circuit is further configured to calculate the distance information of the target object according to the time-of-flight value.

[0027] In some embodiments, the readout circuit or the control and processing circuit includes a counting circuit, and the counting circuit is used to count the number of photon detection signals output by at least part of the pixels in the pixel array.

[0028] In some embodiments, the readout circuit further includes a TDC circuit, a histogram circuit and a readout unit. The TDC circuit is used to generate a time signal according to the photon detection signal output by the corresponding connected pixel, the histogram circuit is used to generate a histogram according to the time signal, and the readout unit is used to generate a time-of-flight value according to the histogram.

[0029] In some embodiments, the pixels are connected to the readout circuit one by one.

[0030] In a fourth aspect, an embodiment of the present application provides an electronic device, including the DToF-based detection system in any embodiment of the second aspect above, and the emitter and the collector of the DToF-based detection system are arranged on the same side of the electronic device body.

[0031] The beneficial effects of the embodiments of the present application are as follows: When detecting, the effective pixels corresponding to the light source can be determined, the pre-calibration process can be omitted, the calibration and precise detection can be completed simultaneously, and power can be saved. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0033] Figure 1Schematic diagram of a detection system based on DToF provided by an embodiment of the present application.

[0034] Figure 2 Schematic diagram of a collector provided by an embodiment of the present application.

[0035] Figure 3 Schematic diagram of the implementation process of a detection method based on DToF provided by an embodiment of the present application.

[0036] Figure 4 Schematic diagrams of six preset projection patterns provided by an embodiment of the present application.

[0037] Figure 5 For an embodiment of the present application, Figure 4 Schematic diagrams of the expected standard spot patterns corresponding to two of the preset projection patterns.

[0038] Figure 6 Schematic diagram of the process of step S33 in a detection method based on DToF provided by an embodiment of the present application.

[0039] Figure 7 Schematic diagram of the comparison between a spot imaging diagram and an expected standard spot diagram provided by an embodiment of the present application.

[0040] Figure 8 Schematic diagram of the comparison between a spot imaging diagram and an expected standard spot diagram provided by another embodiment of the present application.

[0041] Figure 9 Schematic diagram of the implementation process of another detection method based on DToF provided by an embodiment of the present application.

[0042] Figure 10 Schematic diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0043] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0044] As used herein, the phrase "in one embodiment" or "in some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "in one embodiment," "in some embodiments," "in other some embodiments," "in still other embodiments," etc., which appear in various places in this specification, are not necessarily all referring to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and their variants mean "including but not limited to," unless otherwise specifically emphasized.

[0045] It should be noted that when an element is referred to as being "connected" to another element, it can be directly connected to the other element or indirectly connected to the other element. In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present application, "a plurality" means two or more, unless otherwise specifically defined.

[0046] To illustrate the technical solutions described in the present application, the following specific embodiments are used for illustration.

[0047] Please refer to Figure 1 , an embodiment of the present application provides a DToF-based detection system 10. The DToF-based detection system 10 includes a transmitter 11, a collector 12, and a control and processing circuit 13. Among them, the transmitter 11 is configured to emit an emission beam 30 towards a target area 20. The emission beam 30 is emitted into the space of the target area 20 to illuminate the target object in the space. At least part of the beam 30 forms a reflected beam 40 after being reflected by the target object. At least part of the reflected beam 40 is received by the collector 12. The collector 12 receives the reflected beam 40 and generates detection information. The control and processing circuit 13 is respectively connected to the transmitter 11 and the collector 12, synchronizes the trigger signals of the transmitter 11 and the collector 12, and the control and processing circuit 13 can obtain the detection information fed back by the collector 12 and perform subsequent processing and control based on the detection information.

[0048] In some embodiments, the emitter 11 includes a light source 111, an emission optical element 112, a driver 113, etc. Among them, the light source 111 can be a light-emitting diode (LED), a laser diode (LD), an edge emitting laser (EEL), a vertical cavity surface emitting laser (VCSEL), etc., or can be a one-dimensional or two-dimensional light source array composed of multiple light sources. Preferably, the light source array is a VCSEL array light source chip formed by generating multiple VCSEL light sources on a single semiconductor substrate, and the arrangement of the light sources in the light source array can be regular or irregular. The light beam emitted by the light source 111 can be visible light, infrared light, ultraviolet light, etc.

[0049] The emission optical element 112 receives the emission light beam 30 emitted from the light source 111, shapes it, and projects it onto the target area 20. The emission optical element 112 receives the pulsed light beam from the light source 111, and optically modulates the pulsed light beam, such as modulation by diffraction, refraction, reflection, etc., and then emits the modulated light beam, such as a focused light beam, a floodlight beam, a structured light beam, etc., towards the target area 20. The emission optical element 112 can be one or a combination of forms such as a lens, a liquid crystal element, a diffractive optical element, a microlens array, a metasurface optical element, a mask plate, a mirror, a MEMS galvanometer mirror, etc.

[0050] In one embodiment, the light source 111 emits a pulsed light beam outward at a certain frequency (or pulse period) under the drive of the driver 113 and under the control of the control and processing circuit 13. The pulsed light beam is projected onto the target object through the emission optical element 112 to form an illumination spot, where the frequency can be set according to the measured distance.

[0051] In some embodiments, the collector 12 includes a pixel array 121, a filtering unit 122, a receiving optical element 123, and a readout circuit array ( Figure 1 not shown in the figure). The receiving optical element 123, the filtering unit 122, and the pixel array 121 are arranged in sequence along the propagation path of the optical signal.

[0052] Among them, the receiving optical element 123 is used to receive at least part of the reflected light beam 40 reflected back by the target object and guide at least part of the reflected light beam 40 onto the pixel array 121 to image the target object onto the pixel array 121.

[0053] The filtering unit 122 is used to filter out background light or stray light. For example, it is a band-pass filter, etc.

[0054] The pixel array 121 consists of multiple pixels, and the pixel array 121 is configured to activate (i.e., actuate) all pixels under the control of the control and processing circuit 13 to collect at least part of the reflected light beam 40 reflected by the target object and generate corresponding photon detection signals. The readout circuit array consists of multiple readout circuits. In one embodiment, the readout circuit is used on the one hand to receive the photon detection signals of each pixel for processing to generate a histogram, and further the time-of-flight value can be obtained according to the histogram; on the other hand, it is used to count the number of photon detection signals output by each pixel in the pixel array. In one embodiment, the pixel array 121 is a SPAD array composed of multiple single-photon avalanche diodes (Single Photon Avalanche Diode, SPAD). Optionally, the SPAD array is an addressable matrix. Generally, the control and processing circuit 13 can interact with the logic components of the SPAD array to select one or more pixels to be actuated, i.e., working pixels, at any given time, and the pixel array 121 is configured to actuate the working pixels under the control of the control and processing circuit 13 to receive the reflected light beam 40.

[0055] The pixel array 121 is connected to the readout circuit array. Specifically, each pixel in the pixel array 121 can be correspondingly connected to a readout circuit, and each pixel can include one or more SPADs. The readout circuit receives and accumulates the photon detection signals from the correspondingly connected pixels to generate a histogram of the pixel and outputs the time-of-flight value of the pixel according to the histogram. In some embodiments, the readout circuit can also count the number of photon detection signals generated by the correspondingly connected pixels. In some embodiments, the readout circuit includes one or more of devices such as a signal amplifier, a time-to-digital converter (Time-to-Digital Converter, TDC), an analog-to-digital converter (Analog-to-digital converter, ADC), a readout (R / O) unit, a counter, etc.

[0056] In one embodiment, the readout circuit includes a counting circuit, a TDC circuit, a histogram circuit, and an R / O unit, as Figure 2As shown, each pixel in the pixel array is correspondingly connected to a counting circuit, a TDC circuit, a histogram circuit, and an R / O unit. Specifically, the pixel receives photons in the pulsed light signal reflected by the target object and generates corresponding photon detection signals. The counting circuit receives and counts the number of these photon detection signals; the TDC circuit receives and calculates the time interval of the photon detection signals and converts the time interval into a time code. The histogram circuit accumulates the time codes output by the TDC circuit to generate a histogram of the pixel, and the R / O unit outputs the time-of-flight value of the pixel according to the histogram generated by the histogram circuit. Further, the readout circuit can be connected to the control and processing circuit. The control and processing circuit can receive the number of photon detection signals of each pixel output by the counting circuit to generate a grayscale image, and then determine the calibration relationship between each light source and the pixels in the pixel array according to the grayscale image and the expected standard spot pattern, where the expected standard spot pattern is determined according to the preset projection pattern; the control and processing circuit can also receive the time-of-flight value of the pixel output by the R / O unit and calculate the distance information of the target object according to the time-of-flight value of the pixel. In this embodiment, the counting circuit counts the number of photon detection signals generated by each pixel, and the control and processing circuit uses the number of each pixel as the grayscale value of the pixel to obtain a grayscale image.

[0057] In another embodiment, the readout circuit includes a TDC circuit, a histogram circuit, and an R / O unit. Each pixel in the pixel array is correspondingly connected to a TDC circuit, a histogram circuit, and an R / O unit. The control and processing circuit includes a counting circuit, and the counting circuit is used to count the number of photon detection signals generated by each pixel. Specifically, the pixel receives photons in the pulsed light signal reflected by the target object and generates corresponding photon detection signals. The TDC circuit receives and calculates the time interval of the photon detection signals and converts the time interval into a time code. The histogram circuit accumulates the time codes output by the TDC circuit to generate a histogram of the pixel, and the R / O unit outputs the time-of-flight value of the pixel according to the histogram generated by the histogram circuit. Further, the readout circuit can be connected to the control and processing circuit. The control and processing circuit can receive the histograms of each pixel output by the readout circuit, count the number of photon detection signals generated by each pixel according to the histograms of each pixel to generate a grayscale image, and then determine the calibration relationship between each light source and the pixels in the pixel array according to the grayscale image and the expected standard spot pattern, where the expected standard spot pattern is determined according to the preset projection pattern; the control and processing circuit can also receive the time-of-flight value of the pixel output by the R / O unit and calculate the distance information of the target object according to the time-of-flight value of the pixel. In this embodiment, the control and processing circuit sums up the number of photons in each time bin of the histogram of each pixel as the grayscale value of the pixel to obtain a grayscale image.

[0058] It should be noted that theoretical calculations can be performed in advance based on multiple parameters of the detection system to predict the imaging positions of the speckle beams emitted by each light source on the pixel array, so that the expected standard speckle pattern corresponding to one or more turned-on light sources can be determined. That is to say, the expected standard speckle pattern reflects the expected positions of the speckle beams emitted by the turned-on light sources when imaged on the pixel array.

[0059] The calibration was completed through one measurement before, that is, the calibration relationship between each light source and the pixels in the pixel array was determined. The subsequent measurement process can rely on this calibration result. The following describes this measurement process through specific embodiments.

[0060] In some embodiments, the control and processing circuit can also control at least one working light source in the collector to emit a speckle beam towards the target object, and synchronously control the activation of the working pixels in the pixel array and control other pixels in the pixel array except the working pixels, that is, the non-working pixels are turned off according to the calibration relationship. The working pixels are the effective pixels in the pixel array corresponding to the light source that emits the speckle beam, that is, the working light source, and the non-working pixels are the remaining pixels in the pixel array except the working pixels; the working pixels receive at least part of the speckle beam and output a photon detection signal to the readout circuit, and the readout circuit generates a histogram according to the photon detection signal and generates a time-of-flight value according to the histogram. In the embodiments of the present application, the calibration relationship between each light source and the pixels in the pixel array is determined before detection, and then during the detection process, the working pixels corresponding to the turned-on light sources are determined and rely on the working pixels to output photon detection signals in response to the incidence of photons, so that the pre-calibration process before detection can be omitted, enabling calibration and precise detection to be completed simultaneously, and power can be saved.

[0061] Furthermore, in some embodiments, the detection information generated when all pixels are turned on, such as the number of photon detection signals, time intervals, time codes, histograms, time-of-flight values, etc., of these invalid pixels can be not stored in the pixel array, which can save storage space.

[0062] The control and processing circuit 13 synchronizes the trigger signals of the transmitter 11 and the collector 12. Optionally, the control and processing circuit 13 can control the light sources that need to be turned on and / or off at each given time, as well as the pixels that need to be activated (i.e., actuated) and / or deactivated (i.e., turned off). All kinds of detection information generated by the readout circuit can be output to the control and processing circuit 13 for its subsequent use. In one embodiment, the pixels respond to incident single photons and output photon detection signals. The readout circuit receives the photon detection signals from the corresponding connected pixels, processes them to generate a histogram, and outputs the time-of-flight value of the pixel. The readout circuit or the control and processing circuit 13 counts the number of photon detection signals of the pixels. The control and processing circuit 13 generates a grayscale map based on the number of photon detection signals of each pixel, and then determines the calibration relationship between each light source and the pixels in the pixel array according to the grayscale map and the expected standard spot map. It can be understood that the control and processing circuit 13 can be an independent dedicated circuit, such as a dedicated system-on-chip (SOC), a field-programmable gate array (FPGA) chip, an application-specific integrated circuit (ASIC) chip, etc., or can include a general-purpose processing circuit.

[0063] In some embodiments, the DToF-based detection system 10 further includes a memory. The memory can be used to store pulse coding programs to control the excitation time, emission frequency, etc. of the light source 111 to emit light beams. The memory can be used to store time intervals, time codes, histograms, time-of-flight values, the number of photon detection signals, distance information, etc.

[0064] In some embodiments, the DToF-based detection system 10 may further include devices such as a color camera, an infrared camera, an IMU, etc. The combination with these devices can achieve more abundant functions, such as 3D texture modeling, infrared face recognition, SLAM and other functions.

[0065] The embodiments of the present application further provide a DToF-based detection method. The DToF-based detection method can be applied to the DToF-based detection system in the foregoing embodiments. In some embodiments, the DToF-based detection method can be executed by the control and processing circuit of the DToF-based detection system. In some embodiments, the DToF-based detection method can be executed by an electronic device. It should be noted that for the parts not described in detail in the embodiments of the DToF-based detection method for the DToF-based detection system, reference can be made to the foregoing, and details will not be repeated here.

[0066] As Figure 3As shown in the figure, an embodiment of the present application provides a detection method based on DToF, which may include the following steps S31 to S33.

[0067] S31. Control the emitter to emit a speckle beam towards the target object according to a preset projection pattern.

[0068] Among them, the emitter includes a light source array composed of multiple light sources. The preset projection pattern reflects the light sources that need to be turned on, that is, the preset projection pattern identifies the working light sources, and the working light sources can be one or more. The light sources that need to be turned on can be determined according to the preset projection pattern. The present application does not limit the preset projection pattern. For example, it can be any one of the patterns shown in FIGS. A to F as shown Figure 4 below. Each sub-figure represents a light source array, one grid represents one light source, and the circled grid represents that the light source at this position is turned on. It should be understood that Figure 4 the description shown is only an example of the light source array and cannot be construed as a limitation to the present application. Figure 4

[0069] In some embodiments, in order to further improve the anti-interference ability of the DToF-based detection system, different detection systems in the same area can use different preset projection patterns for system calibration.

[0070] S32. Synchronously activate the pixel array in the collector to receive at least part of the speckle beam reflected back by the target object.

[0071] Among them, at least some pixels in the pixel array receive at least part of the speckle beam reflected back by the target object and output photon detection signals to the readout circuit.

[0072] The collector includes a pixel array composed of multiple pixels and a readout circuit array composed of multiple readout circuits. The pixels are connected to the readout circuits in a one-to-one correspondence.

[0073] In one embodiment, the readout circuit is used to count the number of photon detection signals output by the pixels. As a possible implementation, the readout circuit includes a counting circuit, each pixel is connected to a counting circuit correspondingly, and the counting circuit is used to count the number of photon detection signals output by the correspondingly connected pixels. The readout circuit transmits the counted number of photon detection signals to the control and processing circuit. Furthermore, the control and processing circuit generates a grayscale image according to the number of photon detection signals corresponding to the pixels.

[0074] In another embodiment, the readout circuit is used to generate a histogram according to the photon detection signal output by the pixel. The control and processing circuit includes a counting circuit, which is used to count the histogram corresponding to the pixel to obtain the number of photon detection signals. As a possible implementation, the readout circuit includes a TDC circuit and a histogram circuit. The TDC circuit is used to generate a time signal according to the photon detection signal output by the correspondingly connected pixel, and the histogram circuit is used to generate a histogram according to the time signal. The readout circuit transmits the histogram corresponding to the pixel to the control and processing circuit. Furthermore, the control and processing circuit sums up the number of photons in all time bins in the histogram corresponding to the pixel as the gray value, thus generating a gray-scale image. It should be noted that the sum of the number of photons in all time bins in the histogram corresponding to the pixel is equal to the number of photon detection signals output by the pixel.

[0075] S33. Generate a gray-scale image according to the number of photon detection signals, and determine the calibration relationship between each light source in the light source array and the pixels in the pixel array according to the gray-scale image and the expected standard spot map, where the expected standard spot map is determined according to the preset projection pattern.

[0076] Among them, the expected standard spot map identifies the predicted imaging positions of the turned-on light sources in the pixel array. It should be noted that the predicted imaging positions of the spot beams emitted by each light source on the pixel array can be theoretically calculated according to multiple parameters of the detection system, so that the expected standard spot map can be determined according to the turned-on light sources in the preset projection pattern. For example, when the preset projection pattern adopts Figure 4 the style shown in Figure A in Figure 5 the corresponding expected standard spot map is as shown in Figure A in Figure 4 ; another example is that when the preset projection pattern adopts Figure 5 the style shown in Figure B in Figure 5 each subfigure in represents a pixel array, a grid represents a pixel, and the circled grid represents the imaging position of the reflected light spot. It should be understood that Figure 5 the illustration shown is only an exemplary description of the pixel array and should not be construed as a limitation to this application.

[0077] Optionally, based on the embodiment shown in Figure 3 , in some embodiments, step S33, generating a gray-scale image according to the number of each pixel in the pixel array and determining the effective pixels corresponding to the light source according to the gray-scale image and the expected standard spot map, may include: inputting the gray-scale image into a first neural network model to obtain a spot imaging map, and determining the calibration relationship between each light source in the light source array and the pixels in the pixel array according to the comparison result between the spot imaging map and the expected standard spot map. The process schematic diagram of step S33 is as shown in Figure 6 .

[0078] Among them, the first neural network model is used to generate a spot imaging map corresponding to the grayscale map. By generating the spot imaging map using the grayscale map, the spots in the image can be clearly identified, and the spot imaging map can be accurately compared with the expected standard spot map to obtain a more accurate calibration result.

[0079] The embodiments of the present application do not specifically limit the first neural network model. The first neural network model can be a trained neural network model. In one embodiment, the grayscale map sample and its corresponding spot imaging map sample are used as a group of image samples, and multiple groups of image samples are obtained to train the second neural network model to obtain the first neural network model. The second neural network model is the initial neural network model. The second neural network model and the first neural network model have the same network structure, and the first neural network model is a network obtained by optimizing the parameters of the second neural network model.

[0080] In one embodiment, determining the calibration relationship between each light source in the light source array and the pixels in the pixel array according to the comparison result between the spot imaging map and the expected standard spot map includes: comparing the spot imaging map with the expected standard spot map to determine the differential pixels corresponding to the light source, and using the expected imaging pixels corresponding to the light source and the corresponding differential pixels as the calibration relationship between each light source in the light source array and the pixels in the pixel array.

[0081] As a non-limiting example, as Figure 7 shown, for the spot beam emitted by a certain light source, the spot beam reflected by the target object is imaged on the pixel array. The imaging spot of this light source in the spot imaging map is Figure 7 shown by the solid circle 71 in Figure 7 , while the spot in the expected standard spot map is

[0082] shown by the dashed circle 72 in

[0083] Determining the differential pixels between the dashed circle 72 and the solid circle 71 includes the four pixels in the solid square 73. The four pixels in the solid square 73 and the four pixels covered by the dashed circle 72, a total of eight pixels, are used as the effective pixels of this light source. Figure 8 shown, for the spot beam emitted by a certain light source, the spot beam reflected by the target object is imaged on the pixel array. The imaging spot of this light source in the spot imaging map is Figure 8 shown by the solid circle 81 in Figure 8As shown by the dashed circle 82, it is determined that the solid circle 81 is shifted one pixel to the left relative to the dashed circle 82, as shown by the black arrow 83 in the figure. Therefore, it can be determined that a total of six pixels, namely two pixels in the solid square 84 and four pixels covered by the dashed circle 82, are the effective pixels of the light source.

[0084] It should be noted that the imaging position change trends of the light spots of different light sources in the light source array are basically the same. Therefore, during the calibration process, the effective pixels of other unactivated light sources can be analogously calibrated based on the effective pixels corresponding to the activated light source, so that the effective pixels corresponding to all light sources can be calibrated at one time. During the subsequent detection process, if the emission light source is switched, it is no longer necessary to activate all pixels, but only the activated light source, that is, the effective pixels corresponding to the working light source, can be activated according to the previous calibration results.

[0085] By executing steps S31 to S33, the calibration process of the light source is completed, and the calibration relationship between each light source and the pixels in the pixel array is determined. On this basis, an embodiment of the present application provides a detection method, including: controlling at least one working light source in the collector to emit a spot beam towards the target object, synchronously controlling the activation of the working pixels in the pixel array and controlling other pixels in the pixel array except the working pixels, that is, the non-working pixels are turned off. The working pixels are the effective pixels corresponding to the working light source in the pixel array, and the non-working pixels are the remaining pixels in the pixel array except the working pixels; the working pixels receive at least part of the spot beam reflected by the target object and output a photon detection signal to the readout circuit, and the readout circuit generates a histogram according to the photon detection signal and generates a time-of-flight value according to the histogram. Further, the distance information of the target object can also be calculated according to the time-of-flight value.

[0086] Specifically, the readout circuit is used to generate a histogram according to the photon detection signals of the effective pixels corresponding to the activated light source and obtain the time-of-flight value according to the histogram.

[0087] In one embodiment, the readout circuit further includes a TDC circuit, a histogram circuit, and an R / O unit. Each pixel is correspondingly connected to a TDC circuit, a histogram circuit, and an R / O unit. The TDC circuit is used to generate a time signal according to the photon detection signal output by the corresponding working pixel, the histogram circuit is used to generate a histogram according to the time signal, and the R / O unit is used to generate a time-of-flight value according to the histogram.

[0088] Figure 9 Shown is a DToF-based detection method provided by another embodiment of the present application. As Figure 9 shown, the DToF-based detection method may include the following steps S91 to S95. It should be noted that the same parts as those in the foregoing embodiments are not described herein again.

[0089] S91. Control at least one light source in the emitter to emit a speckle beam towards the target object according to a preset projection pattern.

[0090] S92. Synchronously activate the pixel array in the collector to receive at least part of the speckle beam reflected by the target object.

[0091] S93. Generate a grayscale image according to the number of photon detection signals of the pixels in the pixel array.

[0092] In one embodiment, the readout circuit is further configured to count the number of photon detection signals generated by each pixel in the pixel array and transmit the number to the control and processing circuit. The control and processing circuit uses the number as the grayscale value of the pixel to obtain a grayscale image.

[0093] In another embodiment, the control and processing circuit is further configured to count the number of photon detection signals generated by each pixel in the pixel array. Specifically, the control and processing circuit sums up the number of photons in each time bin of the histogram corresponding to each pixel in the pixel array, and then uses the sum result as the grayscale value of the pixel to obtain a grayscale image.

[0094] S94. Input the grayscale image into the first neural network model to obtain a speckle imaging map.

[0095] In some cases, due to the limitation of the number of samples, the robustness of the first neural network model is not strong enough, so errors may occur. To avoid the adverse effects of such errors on the calibration results, the embodiments of the present application set a difference threshold. When it is determined that the difference between the speckle imaging map and the expected standard speckle map is less than the threshold, then continue to execute step S95; otherwise, continue to execute steps S96 to S98.

[0096] It should be noted that the difference may include an offset, etc. The threshold may be an empirical value, and the embodiments of the present application do not limit this. In addition, it should be understood that in other equivalent solutions, when it is determined that the difference is equal to the threshold, step S95 may also be continued to be executed.

[0097] S95. If it is determined that the difference between the speckle imaging map and the expected standard speckle map is less than the threshold, then determine the calibration relationship between each light source in the light source array and the pixels in the pixel array according to the comparison result between the speckle imaging map and the expected standard speckle map.

[0098] S96. If it is determined that the difference between the speckle imaging map and the expected standard speckle map is greater than or equal to the threshold, then obtain a new preset projection pattern. If it is determined that the number of times of obtaining the preset projection pattern is less than or equal to the preset number of times, then continue to execute step S97; if it is determined that the number of times of obtaining the preset projection pattern is greater than the preset number of times, then continue to execute S98.

[0099] S97, if it is determined that the number of times of obtaining the preset projection pattern is less than or equal to the preset number of times, then return to execute step S91 and subsequent steps.

[0100] S98, if it is determined that the number of times of obtaining the preset projection pattern is greater than the preset number of times, then end.

[0101] That is to say, when it is determined that the number of times of obtaining the preset projection pattern is greater than the preset number of times, the process can be ended, or when it is determined that the difference between the new spot imaging diagram and the expected standard spot diagram is less than the threshold value, the calibration relationship between each light source in the light source array and the pixel in the pixel array can be determined according to the comparison result between the new spot imaging diagram and the expected standard spot diagram, and the process can be ended.

[0102] It should be noted that the preset number of times can be set to any integer number of times such as 3 times or greater than 3 times.

[0103] An embodiment of the present application further provides an electronic device. Referring to Figure 10 As shown, the electronic device includes: a processor 1000, a memory 1001, and a computer program 1002 stored in the memory 1001 and executable on the processor 1000, such as a DToF-based detection program. When the processor 1000 executes the computer program 1002, the steps in the DToF-based detection method embodiment of any of the above embodiments are implemented, such as Figure 3 the steps S31 to S33 shown.

[0104] Exemplarily, the computer program 1002 can be divided into one or more modules / units. The one or more modules / units are stored in the memory 1001 and executed by the processor 1000 to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 1002 in the electronic device.

[0105] Those skilled in the art can understand that Figure 10 it is only an example of the electronic device and does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine some components, or different components. For example, the electronic device may further include input / output devices, network access devices, buses, etc.

[0106] The so-called processor 1000 may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), ASICs, FPGAs, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0107] The memory 1001 may be an internal storage unit of the electronic device, such as the hard disk or memory of the electronic device. The memory 1001 may also be an external storage device of the electronic device, such as a plug-in hard disk equipped on the electronic device, a Smart Media Card (SMC), a Secure Digital (SD) card, a FlashCard, etc. Further, the memory 1001 may also include both an internal storage unit and an external storage device of the electronic device. The memory 1001 is used to store the computer program and other programs and data required by the electronic device. The memory 1001 may also be used to temporarily store the data that has been output or will be output.

[0108] An embodiment of the present application further provides an electronic device. The electronic device includes the DToF-based detection system in any of the foregoing embodiments, wherein the transmitter and the collector of the detection system are arranged on the same side of the electronic device body.

[0109] As a non-limiting example, the electronic device may be a lidar, etc.

[0110] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the steps in the foregoing embodiments of various DToF-based detection methods can be implemented.

[0111] An embodiment of the present application provides a computer program product, and when the computer program product runs on an electronic device, the electronic device can implement the steps in the foregoing embodiments of various DToF-based detection methods.

[0112] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0113] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0114] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of devices or units can be in electrical, mechanical or other forms.

[0115] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0116] In addition, the functional units in each embodiment of this application can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0117] When the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-described embodiment methods of the present application, it can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, ROM, RAM, electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0118] The above are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A calibration method based on DToF, characterized in that, Including: S1. According to a preset projection pattern, control a transmitter to emit a speckle beam towards a target object, where the transmitter includes a light source array composed of a plurality of light sources; The preset projection pattern identifies the working light source; S2. Synchronously activate a pixel array in a collector to receive at least part of the speckle beam reflected back by the target object; at least part of the pixels in the pixel array receive the speckle beam and output photon detection signals; S3. Generate a histogram based on the photon detection signals and count the corresponding histogram to obtain the number of photon detection signals to generate a grayscale image, and determine the calibration relationship between each light source in the light source array and the pixels in the pixel array according to the grayscale image and an expected standard speckle pattern; the expected standard speckle pattern is determined according to the preset projection pattern; the expected standard speckle pattern identifies the predicted imaging positions of the working light sources in the pixel array, and the expected standard speckle pattern is theoretically calculated based on multiple parameters of the detection system for the speckle beams emitted by each light source in the pixel array and determined according to the light sources turned on in the preset projection pattern; The determining the calibration relationship between each light source in the light source array and the pixels in the pixel array according to the grayscale image and the expected standard speckle pattern includes: Input the grayscale image into a first neural network model to obtain a speckle imaging map, If it is determined that the difference between the speckle imaging map and the expected standard speckle pattern is less than a threshold, determine the calibration relationship between each light source in the light source array and the pixels in the pixel array according to the comparison result between the speckle imaging map and the expected standard speckle pattern; If it is determined that the difference between the speckle imaging map and the expected standard speckle pattern is greater than or equal to the threshold, obtain a new preset projection pattern, and return to execute step S1 and subsequent steps until the number of times of obtaining the preset projection pattern is greater than a preset number, or until it is determined that the difference between the new speckle imaging map and the expected standard speckle pattern is less than the threshold, and determine the calibration relationship between each light source in the light source array and the pixels in the pixel array according to the comparison result between the new speckle imaging map and the expected standard speckle pattern.

2. The calibration method according to claim 1, characterized in that, The determining the calibration relationship between each light source in the light source array and the pixels in the pixel array according to the comparison result between the speckle imaging map and the expected standard speckle pattern includes: Compare the speckle imaging map with the expected standard speckle pattern to determine the difference pixels corresponding to each light source in the light source array, and use the expected imaging pixels and the corresponding difference pixels corresponding to each light source as the effective pixels corresponding to the light source.

3. The calibration method according to claim 1, wherein The determining the calibration relationship between each light source in the light source array and the pixels in the pixel array according to the comparison result between the speckle imaging map and the expected standard speckle pattern includes: Compare the speckle imaging map with the expected standard speckle pattern to determine the offset direction and offset amount of the actual imaging pixels of each light source in the light source array relative to the expected imaging pixels, and determine the effective pixels of each light source in the light source array according to the offset direction and the offset amount.

4. A detection method based on DToF, characterized in that, Including: Control at least one working light source in the transmitter to emit a speckle beam towards a target object; And control the activation of the working pixels corresponding to the working light source in the collector according to the calibration relationship described in any one of claims 1-3; the working pixels receive at least part of the speckle beam reflected by the target object and output a photon detection signal to the readout circuit, and the readout circuit generates a time-of-flight value according to the photon detection signal.

5. A detection system based on DToF, characterized in that, Comprising: A transmitter, a collector, and a control and processing circuit, The transmitter includes a light source array composed of multiple light sources, and the transmitter is configured to emit a speckle beam towards the target object according to a preset projection pattern; The preset projection pattern identifies the working light source; The collector includes a pixel array composed of multiple pixels, and the collector is configured to activate the pixel array under the control of the control and processing circuit to receive at least part of the speckle beam reflected back by the target object, and at least part of the pixels in the pixel array receive the speckle beam and output a photon detection signal; The control and processing circuit is configured to generate a histogram according to the photon detection signal and count the corresponding histogram to obtain the number of photon detection signals to generate a grayscale map, and determine the calibration relationship between each light source in the light source array and the pixels in the pixel array according to the grayscale map and the expected standard speckle pattern. The expected standard speckle pattern identifies the predicted imaging position of the working light source in the pixel array, and the expected standard speckle pattern is theoretically calculated based on multiple parameters of the detection system for each light source to emit a speckle beam in the pixel array and determined according to the turned-on light sources in the preset projection pattern; The determining the calibration relationship between each light source in the light source array and the pixels in the pixel array according to the grayscale map and the expected standard speckle pattern includes: Input the grayscale map into the first neural network model to obtain a speckle imaging map, If it is determined that the difference between the speckle imaging map and the expected standard speckle pattern is less than the threshold, then determine the calibration relationship between each light source in the light source array and the pixels in the pixel array according to the comparison result between the speckle imaging map and the expected standard speckle pattern; If it is determined that the difference between the speckle imaging map and the expected standard speckle pattern is greater than or equal to the threshold, then obtain a new preset projection pattern, and return to execute step S1 and subsequent steps until the number of times of obtaining the preset projection pattern is greater than the preset number of times, or until it is determined that the difference between the new speckle imaging map and the expected standard speckle pattern is less than the threshold, and determine the calibration relationship between each light source in the light source array and the pixels in the pixel array according to the comparison result between the new speckle imaging map and the expected standard speckle pattern.

6. The DToF-based detection system according to claim 5, characterized in that, The control and processing circuit is further configured to control at least one working light source in the transmitter to emit a speckle beam towards the target object, and the transmitter includes a light source array composed of multiple light sources; and synchronously control the activation of the working pixels corresponding to the working light source in the collector according to the calibration relationship; the working pixels receive at least part of the speckle beam reflected by the target object and output a photon detection signal to the readout circuit, and the readout circuit generates a time-of-flight value according to the photon detection signal; The control and processing circuit is further configured to calculate the distance information of the target object according to the time-of-flight value.

7. The DToF-based detection system according to claim 6, characterized in that, The readout circuit or the control and processing circuit includes a counting circuit for counting the number of photon detection signals output by each of at least some of the pixels in the pixel array; The readout circuit further includes a TDC circuit, a histogram circuit, and a readout unit. The TDC circuit is configured to generate a time signal based on the photon detection signals output by the correspondingly connected pixels. The histogram circuit is configured to generate a histogram based on the time signal. The readout unit is configured to generate a time-of-flight value based on the histogram.

Images