A Detection System Based on DToF and Its Calibration Method

Abstract

This application relates to the field of optical detection technology, and particularly to a detection system based on DToF and its calibration method. The calibration method includes: controlling a transmitter to emit a speckle beam towards a target according to a preset projection pattern; the transmitter includes a light source array composed of multiple light sources; synchronously activating a pixel array in a collector to receive at least part of the speckle beam reflected by the target; at least part of the pixels in the pixel array receive the speckle beam and output photon detection signals; generating a grayscale image according to the number of the photon detection signals, and determining the calibration emission angle of the light source according to the comparison result between the grayscale image and an expected standard speckle pattern. The embodiments of this application can avoid the waste caused by designing a large imaging array due to the need to accommodate imaging deviation.

Description

Technical Field

[0001] This application relates to the field of optical detection technology, and in particular, to a detection system based on DToF and its calibration method. Background Art

[0002] Time of Flight (ToF) technology belongs to two-way ranging technology, which measures the distance of an object by using the flight time of an optical signal traveling back and forth between a transmitter and a collector. Currently, detection systems based on ToF technology have been widely used in fields such as consumer electronics, unmanned driving, Augmented Reality (AR), and Virtual Reality (VR).

[0003] ToF technology can be divided into Direct ToF (DToF) technology and Indirect ToF (IToF) technology. Among them, DToF technology measures the flight time of photons in a light beam from emission to reception based on Time-Correlated Single Photon Counting (TCSPC) technology; IToF technology measures the phase delay of a reflected light beam relative to an emitted light beam, and then calculates the flight time from the phase delay. Among them, DToF technology has advantages such as high signal-to-noise ratio, good sensitivity, and high accuracy, and has received more and more extensive attention.

[0004] When using a detection system based on DToF technology, during system assembly and use, it is inevitable to introduce certain system errors, which will cause the position of the spot image to deviate on the receiving pixel array. In order to accommodate the deviation of the spot image, the pixel array is usually designed to be larger than the actual spot area, resulting in waste.

[0005] The disclosure of the above background art content is only used to assist in understanding the concept and technical solution of this application, and it does not necessarily belong to the prior art of this application. Without clear evidence that the above content was publicly available before the filing date of this application, the above background art should not be used to evaluate the novelty and inventiveness of this application. Summary of the Invention

[0006] The purpose of the embodiments of this application is to provide a detection system based on DToF and its calibration method, aiming to solve one or more technical problems in the related art.

[0007] To achieve the above object, in a first aspect, an embodiment of the present application provides a calibration method for a detection system based on DToF, including: S1, controlling a transmitter to emit a speckle beam towards a target according to a preset projection pattern; the transmitter includes a light source array composed of a plurality of light sources; S2, synchronously activating a pixel array in a collector to receive at least part of the speckle beam reflected back by the target; at least part of the pixels in the pixel array receive the speckle beam and output photon detection signals; S3, determining a calibrated emission angle of the light source according to a comparison result between the grayscale image and an expected standard speckle pattern.

[0008] In some embodiments, determining the calibrated emission angle of the light source according to the comparison result between the grayscale image and the expected standard speckle pattern includes: inputting the grayscale image into a first neural network model to obtain a speckle imaging map, comparing the speckle imaging map with the expected standard speckle pattern, and determining the calibrated emission angle of the light source according to the comparison result.

[0009] In some embodiments, comparing the speckle imaging map with the expected standard speckle pattern and determining the calibrated emission angle of the light source according to the comparison result includes: comparing the speckle imaging map with the expected standard speckle pattern. If it is determined that there is no difference between the speckle imaging map and the expected standard speckle pattern, record the current emission angle of the light source as the calibrated emission angle; if it is determined that there is a difference between the speckle imaging map and the expected standard speckle pattern, adjust the emission angle of the light source until there is no difference between the generated new speckle imaging map and the expected standard speckle pattern, and record the current emission angle of the light source as the calibrated emission angle.

[0010] In some embodiments, if it is determined that there is a difference between the speckle imaging map and the expected standard speckle pattern, adjusting the emission angle of the light source until there is no difference between the generated new speckle imaging map and the expected standard speckle pattern, and recording the current emission angle of the light source as the calibrated emission angle includes: if it is determined that the difference between the speckle imaging map and the expected standard speckle pattern is less than a threshold, adjust the emission angle of the light source until there is no difference between the generated new speckle imaging map and the expected standard speckle pattern, and record the current emission angle of the light source as the calibrated emission angle; if it is determined that the difference between the speckle imaging map and the expected standard speckle pattern is greater than or equal to the threshold, obtain a new preset projection pattern, and return to execute step S1 and subsequent steps until the number of times of obtaining the preset projection pattern is greater than a preset number, or until it is determined that the difference between the new speckle imaging map and the expected standard speckle pattern is less than the threshold, adjust the emission angle of the light source until there is no difference between the generated new speckle imaging map and the expected standard speckle pattern, and record the current emission angle of the light source as the calibrated emission angle.

[0011] In some embodiments, adjusting the emission angle of the light source includes: determining the offset direction of the actual imaging pixels of the light source in the spot imaging diagram relative to the expected imaging pixels, and adjusting the emission angle of the light source according to the offset direction; or, determining the offset direction and offset amount of the actual imaging pixels of the light source in the spot imaging diagram relative to the expected imaging pixels, and adjusting the emission angle of the light source according to the offset direction and the offset amount.

[0012] In some embodiments, it further includes: controlling the working light sources in the light source array to emit detection beams towards the target object according to the calibrated emission angle; synchronously controlling the working pixels corresponding to the working light sources in the pixel array to be turned on and receive the detection beams to output photon detection signals to the readout circuit; the readout circuit is configured to generate a time-of-flight value according to the photon detection signals output by the working pixels.

[0013] In some embodiments, it further includes: calculating the distance information of the target object according to the time-of-flight value.

[0014] In a second aspect, an embodiment of the present application provides a detection system based on DToF, including: a transmitter, a collector, and a control and processing circuit. The transmitter includes a light source array composed of multiple light sources, and the transmitter is configured to emit spot beams towards the target object according to a preset projection pattern; the collector includes a pixel array composed of multiple pixels, and the collector is configured to activate the pixel array under the control of the control and processing circuit to receive at least part of the spot beams reflected back by the target object, and at least part of the pixels in the pixel array receive the spot beams and output photon detection signals; the control and processing circuit is configured to generate a grayscale image according to the number of the photon detection signals, and determine the calibrated emission angle of the light source according to the comparison result between the grayscale image and an expected standard spot image.

[0015] In some embodiments, determining the calibrated emission angle of the light source according to the comparison result between the grayscale image and the expected standard spot image includes: inputting the grayscale image into a first neural network model to obtain a spot imaging diagram, comparing the spot imaging diagram with the expected standard spot image, and determining the calibrated emission angle of the light source according to the comparison result.

[0016] In some embodiments, comparing the spot imaging map with the expected standard spot map and determining the calibrated emission angle of the light source according to the comparison result includes: comparing the spot imaging map with the expected standard spot map. If it is determined that there is no difference between the spot imaging map and the expected standard spot map, record the current emission angle of the light source as the calibrated emission angle; if it is determined that there is a difference between the spot imaging map and the expected standard spot map, adjust the emission angle of the light source until there is no difference between the newly generated spot imaging map and the expected standard spot map, and record the current emission angle of the light source as the calibrated emission angle.

[0017] In some embodiments, the readout circuit or the control and processing circuit includes a counting circuit, and the counting circuit is used to count the number of photon detection signals output by each pixel.

[0018] In some embodiments, the readout circuit further includes a TDC circuit, a histogram circuit, and a readout unit. The TDC circuit is used to generate a time signal according to the photon detection signals output by the corresponding connected pixels, the histogram circuit is used to generate a histogram according to the time signal, and the readout unit is used to generate a time-of-flight value according to the histogram.

[0019] In some embodiments, the pixels are connected to the readout circuit in a one-to-one correspondence.

[0020] In some embodiments, the control and processing circuit is further configured to control the working light sources in the light source array to emit detection beams toward the target object at the calibrated emission angle; synchronously control the working pixels corresponding to the working light sources in the pixel array to turn on and receive the detection beams to output photon detection signals to the readout circuit, and the readout circuit is used to generate a time-of-flight value according to the photon detection signals output by the working pixels.

[0021] In some embodiments, the control and processing circuit is further configured to calculate the distance information of the target object according to the time-of-flight value.

[0022] In a third aspect, an embodiment of the present application provides an electronic device, including the DToF-based detection system described in any one of the embodiments of the second aspect, and the transmitter and the collector of the DToF-based detection system are arranged on the same side of the electronic device body.

[0023] The beneficial effects of the embodiments of the present application are as follows: calibrate the detection system, adjust the emission angle of the emission beam to calibrate the spot imaging position, so that the spot falls within the size of the pixel array, thereby saving the area of the receiving array and reducing costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the accompanying drawings required for use in the embodiments or the description of the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0025] Figure 1 It is a schematic structural diagram of a detection system based on DToF provided by an embodiment of the present application.

[0026] Figure 2 It is a schematic structural diagram of a collector provided by an embodiment of the present application.

[0027] Figure 3 It is a schematic diagram of the implementation process of a calibration method for a detection system based on DToF provided by an embodiment of the present application.

[0028] Figure 4 It is a schematic diagram of six preset projection patterns provided by an embodiment of the present application.

[0029] Figure 5 It is provided by an embodiment of the present application and Figure 4 Schematic diagram of the expected standard spot pattern corresponding to two of the preset projection patterns.

[0030] Figure 6 It is a schematic diagram of the process of step S33 in a calibration method for a detection system based on DToF provided by an embodiment of the present application.

[0031] Figure 7 It is a schematic diagram for comparing a spot imaging diagram with an expected standard spot diagram provided by an embodiment of the present application.

[0032] Figure 8 It is a schematic diagram for comparing a spot imaging diagram with an expected standard spot diagram provided by another embodiment of the present application.

[0033] Figure 9 It is a schematic diagram of the implementation process of another calibration method for a detection system based on DToF provided by an embodiment of the present application.

[0034] Figure 10 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0035] In order to make the technical problems to be solved, technical solutions and beneficial effects of the present application more clearly understood, the following further details the present application in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0036] As used in the description of the present application, "one embodiment" or "some embodiments" etc. mean that in one or more embodiments of the present application, specific features, structures or characteristics described in connection with that embodiment are included. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments" etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in another way.

[0037] It should be noted that when an element is referred to as being "connected" to another element, it can be directly connected to the other element or indirectly connected to the other element. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present application, "a plurality of" means two or more, unless otherwise specifically and clearly defined.

[0038] In order to illustrate the technical solution described in the present application, the following will be described through specific embodiments.

[0039] Please refer to Figure 1 , an embodiment of the present application provides a DToF-based detection system 10. The DToF-based detection system 10 includes a transmitter 11, a collector 12, and a control and processing circuit 13. Among them, the transmitter 11 is used to emit an emission beam 30 towards a target area 20. The emission beam 30 is emitted into the space of the target area 20 to illuminate the target object in the space. At least part of the beam 30 forms a reflected beam 40 after being reflected by the target object. At least part of the reflected beam 40 is received by the collector 12, and the collector 12 generates detection information by receiving the reflected beam 40; the control and processing circuit 13 is respectively connected to the transmitter 11 and the collector 12, synchronizes the trigger signals of the transmitter 11 and the collector 12, and the control and processing circuit 13 can obtain the detection information fed back by the collector 12 and perform subsequent processing and control etc. according to the detection information.

[0040] In some embodiments, the transmitter 11 includes a light source 111, a transmitting optical element 112, a driver 113, etc. Among them, the light source 111 can be a light-emitting diode (LED), a laser diode (LD), an edge emitting laser (EEL), a vertical cavity surface emitting laser (VCSEL), etc., or can be a one-dimensional or two-dimensional light source array composed of multiple light sources. Preferably, the light source array is a VCSEL array light source chip formed by generating multiple VCSEL light sources on a single semiconductor substrate, and the arrangement of the light sources in the light source array can be regular or irregular. The light beam emitted by the light source 111 can be visible light, infrared light, ultraviolet light, etc.

[0041] The transmitting optical element 112 receives the emitted light beam 30 emitted by the light source 111, shapes it, and projects it onto the target area 20. The transmitting optical element 112 receives the pulsed light beam from the light source 111, and optically modulates the pulsed light beam, such as modulation by diffraction, refraction, reflection, etc., and then emits the modulated light beam, such as a focused light beam, a floodlight beam, a structured light beam, etc., towards the target area 20. The transmitting optical element 112 can be one or a combination of forms such as a lens, a liquid crystal element, a diffractive optical element, a microlens array, a metasurface optical element, a mask plate, a mirror, a MEMS galvanometer mirror, etc.

[0042] In one embodiment, the light source 111 emits a pulsed light beam outward at a certain frequency (or pulse period) under the drive of the driver 113 and under the control of the control and processing circuit 13. The pulsed light beam is projected onto the target object through the transmitting optical element 112 to form an illumination spot, where the frequency can be set according to the measured distance.

[0043] In some embodiments, the collector 12 includes a pixel array 121, a filtering unit 122, a receiving optical element 123, and a readout circuit array ( Figure 1 not shown in the figure). The receiving optical element 123, the filtering unit 122, and the pixel array 121 are arranged in sequence along the propagation path of the optical signal.

[0044] Among them, the receiving optical element 123 is used to receive at least part of the reflected light beam 40 reflected back from the target object and guide at least part of the reflected light beam 40 onto the pixel array 121 to image the target object onto the pixel array 121.

[0045] The filtering unit 122 is used to filter out background light or stray light. For example, it is a band-pass filter, etc.

[0046] The pixel array 121 consists of multiple pixels. The pixel array 121 is configured to activate (i.e., actuated) all pixels under the control of the control and processing circuit 13 to collect at least part of the reflected light beam 40 reflected back by the target object and generate corresponding photon detection signals. The readout circuit array consists of multiple readout circuits. In one embodiment, the readout circuit is used on the one hand to receive the photon detection signals of each pixel for processing to generate a histogram, and further the time-of-flight value can be obtained according to the histogram; on the other hand, it is used to count the number of photon detection signals output by each pixel in the pixel array. In one embodiment, the pixel array 121 is a SPAD array composed of multiple single-photon avalanche diodes (Single Photon Avalanche Diode, SPAD). Optionally, the SPAD array is an addressable matrix. Generally, the control and processing circuit 13 can interact with the logic components of the SPAD array to select one or more pixels to be actuated at any given time, that is, the working pixels. The pixel array 121 is configured to actuate the working pixels under the control of the control and processing circuit 13 to receive the reflected light beam 40.

[0047] The pixel array 121 is connected to the readout circuit array. Specifically, each pixel in the pixel array 121 can be set to be correspondingly connected to a readout circuit, and each pixel can include one or more SPADs. The readout circuit receives and accumulates the photon detection signals from the correspondingly connected pixels to generate a histogram of the pixel and outputs the time-of-flight value of the pixel according to the histogram. In some embodiments, the readout circuit can also count the number of photon detection signals generated by the correspondingly connected pixels. In some embodiments, the readout circuit includes one or more of devices such as a signal amplifier, a time-to-digital converter (Time-to-Digital Converter, TDC), an analog-to-digital converter (Analog-to-digital converter, ADC), a readout (R / O) unit, a counter, etc.

[0048] In one embodiment, the readout circuit includes a counting circuit, a TDC circuit, a histogram circuit, and an R / O unit, as Figure 2As shown, each pixel in the pixel array is correspondingly connected to a counting circuit, a TDC circuit, a histogram circuit, and an R / O unit. Specifically, at least some pixels in the pixel array receive photons in the pulsed light signal reflected by the target object and generate corresponding photon detection signals. The counting circuit receives and counts the number of these photon detection signals. The TDC circuit receives and calculates the time interval of the photon detection signals and converts the time interval into a time code. The histogram circuit accumulates the time codes output by the TDC circuit to generate a histogram of the pixel. The R / O unit outputs the time-of-flight value of the pixel according to the histogram generated by the histogram circuit. Further, the readout circuit can be connected to the control and processing circuit. The control and processing circuit can receive the number of photon detection signals of each pixel output by the counting circuit to generate a grayscale image, and then determine the calibration emission angle of each light source according to the comparison result between the grayscale image and the expected standard spot pattern. The control and processing circuit can also receive the time-of-flight value of the pixel output by the R / O unit and calculate the distance information of the target object according to the time-of-flight value of the pixel. In this embodiment, the readout circuit includes a counting circuit. The counting circuit counts the number of photon detection signals generated by each pixel. The control and processing circuit uses the number of photon detection signals generated by the pixel as the grayscale value of the pixel to obtain a grayscale image.

[0049] In another embodiment, the readout circuit includes a TDC circuit, a histogram circuit, and an R / O unit. Each pixel in the pixel array is correspondingly connected to a TDC circuit, a histogram circuit, and an R / O unit. The control and processing circuit includes a counting circuit, and the counting circuit is used to count the number of photon detection signals generated by each pixel. Specifically, at least some pixels in the pixel array receive photons in the pulsed light signal reflected by the target object and generate corresponding photon detection signals. The TDC circuit receives and calculates the time interval of the photon detection signals and converts the time interval into a time code. The histogram circuit accumulates the time codes output by the TDC circuit to generate a histogram of the pixel. The R / O unit outputs the time-of-flight value of the pixel according to the histogram generated by the histogram circuit. Further, the readout circuit can be connected to the control and processing circuit. The control and processing circuit can receive the histograms of each pixel output by the readout circuit, count the number of photon detection signals generated by each pixel according to the histograms of each pixel to generate a grayscale image, and then determine the calibration emission angle of each light source according to the comparison result between the grayscale image and the expected standard spot pattern, and the expected standard spot pattern is determined according to the preset projection pattern. The control and processing circuit can also receive the time-of-flight value of the pixel output by the R / O unit and calculate the distance information of the target object according to the time-of-flight value of the pixel. In this embodiment, the control and processing circuit sums up the number of photons in each time bin in the histogram of each pixel as the grayscale value of the pixel to obtain a grayscale image.

[0050] It should be noted that theoretical calculations can be performed in advance based on multiple parameters of the detection system to predict the imaging positions of the speckle beams emitted by each light source on the pixel array, so that the expected standard speckle pattern corresponding to one or more activated light sources can be determined. That is to say, the expected standard speckle pattern reflects the expected positions of the speckle beams emitted by the activated light sources on the pixel array.

[0051] The calibration was completed through one measurement above, that is, the calibration emission angle corresponding to each light source was determined. The subsequent measurement process can rely on this calibration result. The following describes this measurement process through specific embodiments.

[0052] In some embodiments, the control and processing circuit can also control the working light sources in the light source array to emit detection beams towards the target object according to the calibration emission angle; synchronously control the working pixels corresponding to the working light sources in the pixel array to turn on and receive at least part of the detection beams reflected by the target object to output photon detection signals to the readout circuit, and the readout circuit is used to generate time-of-flight values according to the photon detection signals output by the working pixels. In this way, the detection system can be calibrated during the idle time before detection, and then accurate detection can be completed, which can avoid the waste caused by designing a large imaging array to accommodate imaging deviations. In some embodiments, further, the control and processing circuit can also calculate the distance information of the target object according to the time-of-flight value.

[0053] The control and processing circuit 13 synchronizes the trigger signals of the transmitter 11 and the collector 12. Optionally, the control and processing circuit 13 can control the light sources that need to be turned on and / or off at each given time, and the pixels that need to be activated (i.e., actuated) and / or deactivated (i.e., turned off). All kinds of detection information generated by the readout circuit can be transmitted to the control and processing circuit 13 for its subsequent use. In one embodiment, the pixel responds to a single incident photon and outputs a photon detection signal, and the readout circuit receives the photon detection signal from the corresponding connected pixel for processing to generate a histogram and output the time-of-flight value of the pixel. The readout circuit or the control and processing circuit 13 counts the number of photon detection signals of the pixels, and the control and processing circuit 13 generates a grayscale image according to the number of photon detection signals of each pixel, and then determines the calibration reflection angle of the light source according to the comparison result between the grayscale image and the expected standard speckle pattern. It can be understood that the control and processing circuit 13 can be an independent dedicated circuit, such as a dedicated system-on-chip (SOC), a field-programmable gate array (FPGA) chip, an application-specific integrated circuit (ASIC) chip, etc., or can also include a general processing circuit.

[0054] In some embodiments, the DToF-based detection system 10 further includes a memory. The memory can be used to store a pulse coding program, and the coding program is used to control the excitation time, emission frequency, etc. of the light source 111 to emit a light beam. The memory can be used to store time intervals, time codes, histograms, time-of-flight values, the number of photon detection signals, distance information, etc.

[0055] In some embodiments, the DToF-based detection system 10 may further include devices such as a color camera, an infrared camera, an IMU, etc. The combination with these devices can achieve more diverse functions, such as 3D texture modeling, infrared face recognition, SLAM and other functions.

[0056] The embodiments of the present application also provide a calibration method for a DToF-based detection system. The calibration method for a DToF-based detection system can be applied to the DToF-based detection system in the foregoing embodiments. In some embodiments, the calibration method for a DToF-based detection system can be executed by the control and processing circuit of the DToF-based detection system. In some embodiments, the calibration method for a DToF-based detection system can be executed by an electronic device. It should be noted that for the parts not described in detail about the DToF-based detection system in the embodiments of the calibration method for a DToF-based detection system, reference can be made to the foregoing, and details will not be repeated here.

[0057] As Figure 3 shown, a calibration method for a DToF-based detection system provided by an embodiment of the present application may include the following steps S31 to S33.

[0058] S31, according to a preset projection pattern, control at least one light source in the emitter to emit a speckle light beam towards the target object.

[0059] Among them, the emitter includes a light source array composed of multiple light sources. The preset projection pattern reflects the light sources that need to be turned on, that is, the preset projection pattern identifies the working light sources, and the working light sources can be one or more. The light sources that need to be turned on can be determined according to the preset projection pattern. The present application does not limit the preset projection pattern. For example, it can be any one of the patterns shown in FIGS. A to F as Figure 4 shown, Figure 4 each sub-diagram in which represents a light source array, one grid represents one light source, and the grid with a circle drawn represents that the light source at this position is turned on. It should be understood that Figure 4 the illustration shown is only an exemplary description of the light source array and should not be construed as a limitation to the present application.

[0060] In some embodiments, in order to further improve the anti-interference ability of the detection system, different detection systems in the same area can use different preset projection patterns for system calibration.

[0061] S32. Synchronously activate the pixel array in the collector to receive at least part of the speckle beam reflected back by the target object.

[0062] Among them, at least some pixels in the pixel array receive at least part of the speckle beam reflected back by the target object and output photon detection signals to the readout circuit. The collector includes a pixel array composed of multiple pixels and a readout circuit array composed of multiple readout circuits. The pixels are connected to the readout circuits in a one-to-one correspondence.

[0063] In one embodiment, the readout circuit is used to count the number of photon detection signals output by each pixel. As a possible implementation, the readout circuit includes a counting circuit, each pixel is connected to a counting circuit correspondingly, and the counting circuit is used to count the number of photon detection signals output by the correspondingly connected pixel. The readout circuit transmits the counted number of photon detection signals to the control and processing circuit. Furthermore, the control and processing circuit generates a grayscale image according to the number of photon detection signals corresponding to the pixels.

[0064] In another embodiment, the readout circuit is used to generate a histogram according to the photon detection signal output by the pixel. The control and processing circuit includes a counting circuit, and the counting circuit is used to count the histogram corresponding to the pixel to obtain the number of photon detection signals. As a possible implementation, the readout circuit includes a TDC circuit and a histogram circuit. The TDC circuit is used to generate a time signal according to the photon detection signal output by the correspondingly connected pixel, and the histogram circuit is used to generate a histogram according to the time signal. The readout circuit transmits the histogram corresponding to the pixel to the control and processing circuit. Furthermore, the control and processing circuit sums up the number of photons in all time bins in the histogram corresponding to the pixel as the grayscale value, and thus a grayscale image is generated. It should be noted that the sum of the number of photons in all time bins in the histogram corresponding to the pixel is equal to the number of photon detection signals output by the pixel.

[0065] S33. Generate a grayscale image according to the number of photon detection signals, and determine the calibration emission angle of the light source according to the comparison result between the grayscale image and the expected standard speckle pattern, where the expected standard speckle pattern is determined according to the preset projection pattern.

[0066] Among them, the predicted imaging positions of the turned-on light sources in the pixel array are marked in the expected standard speckle pattern. It should be noted that the predicted imaging positions of the speckle beams emitted by each light source on the pixel array can be theoretically calculated according to multiple parameters of the detection system, so that the expected standard speckle pattern can be determined according to the turned-on light sources in the preset projection pattern. For example, when the preset projection pattern adopts the style shown in FIG. A in Figure 4 , the corresponding expected standard speckle pattern is as shown in FIG. A in Figure 5 ; another example is that when the preset projection pattern adopts the style shown in FIG. B in Figure 4 , the corresponding expected standard speckle pattern is as shown in FIG. B in Figure 5As shown in Figure B in Figure 5 Each sub - figure therein represents a pixel array, a grid represents a pixel, and the grid with a circle drawn represents the imaging position of the reflected light spot. It should be understood that Figure 5 The description shown is only an exemplary description of the pixel array and cannot be construed as a limitation to this application.

[0067] Optionally, in some embodiments, based on step S33, it further includes: controlling the working light sources in the light source array to emit detection beams towards the target object according to the calibrated emission angles; synchronously controlling the working pixels corresponding to the working light sources in the pixel array to turn on and receive at least part of the detection beams reflected by the target object to output photon detection signals to the read - out circuit; the read - out circuit is used to generate a time - of - flight value according to the photon detection signals output by the working pixels. Further, the distance information of the target object can also be calculated according to the time - of - flight value. This process is a target detection process using the calibrated detection system after the detection system is calibrated.

[0068] Specifically, in these embodiments, the read - out circuit is further used to generate a histogram according to the photon detection signals output by the working pixels and obtain the time - of - flight value according to the histogram.

[0069] As a possible implementation manner, the read - out circuit includes a TDC circuit, a histogram circuit, and an R / O unit. Each pixel is correspondingly connected to a TDC circuit, a histogram circuit, and an R / O unit. The TDC circuit is used to generate a time signal according to the photon detection signals output by the correspondingly connected working pixels, the histogram circuit is used to generate a histogram according to the time signal, and the R / O unit is used to generate a time - of - flight value according to the histogram.

[0070] Optionally, in some embodiments, in step S33, determining the calibrated emission angle of the light source according to the comparison result between the grayscale image and the expected standard light - spot image may include: inputting the grayscale image into a first neural network model to obtain a light - spot imaging map, comparing the light - spot imaging map with the expected standard light - spot image, and determining the calibrated emission angle of the light source according to the comparison result between the light - spot imaging map and the expected standard light - spot image. The schematic diagram of this process is as Figure 6 shown.

[0071] Among them, the first neural network model is used to generate a light - spot imaging map corresponding to the grayscale image. By using the grayscale image to generate a light - spot imaging map, the light spots in the image can be clearly identified, and the light - spot imaging map can be accurately compared with the expected standard light - spot image to obtain a more accurate calibration result.

[0072] The embodiments of the present application do not specifically limit the first neural network model. The first neural network model can be a trained neural network model. In one embodiment, grayscale image samples and their corresponding spot imaging map samples are used as a set of image samples, and multiple sets of image samples are obtained to train the second neural network model to obtain the first neural network model. The second neural network model is the initial neural network model. The second neural network model and the first neural network model have the same network structure, and the first neural network model is a network obtained by optimizing the parameters of the second neural network model.

[0073] In one embodiment, determining the calibration emission angle of the light source according to the comparison result between the spot imaging map and the expected standard spot map includes: comparing the spot imaging map with the expected standard spot map. If it is determined that there is no difference between the spot imaging map and the expected standard spot map, record the current emission angle of the light source as the calibration emission angle; if it is determined that there is a difference between the spot imaging map and the expected standard spot map, adjust the emission angle of the light source until there is no difference between the generated new spot imaging map and the expected standard spot map, and record the current emission angle of the light source as the calibration emission angle.

[0074] Specifically, when it is determined that there is no difference between the spot imaging map and the expected standard spot map, it means that there is no systematic error in the detection system and no calibration is required. When it is determined that there is a difference between the spot imaging map and the expected standard spot map, it means that there is a systematic error in the detection system and calibration is required. In this embodiment, by adjusting the emission angle of the light source at the transmitting end, the landing point of the imaging spot on the receiving array is adjusted so that the landing point after calibration reaches the predicted imaging position, that is, there is no difference between the new spot imaging map generated after adjusting the emission angle and the expected standard spot map, avoiding waste caused by designing a large imaging array to accommodate imaging deviation.

[0075] As a possible implementation, adjusting the emission angle of the light source includes: determining the offset direction of the actual imaging pixel of the light source in the spot imaging map relative to the expected imaging pixel, and adjusting the emission angle of the light source according to the offset direction.

[0076] As a non-limiting example, as Figure 7 shown, for the spot beam emitted by a certain light source, the spot beam reflected by the target object is imaged on the pixel array. The imaging spot of the light source in the spot imaging map is Figure 7 shown by the solid circle 71 in the figure, while the spot in the expected standard spot map is Figure 7As shown by the dashed circle 72, it is determined that the solid circle 71 is shifted to the left relative to the dashed circle 72. Therefore, the emission angle of the light source can be adjusted to shift the imaging spot to the right, so that the solid circle 71 and the dashed circle 72 basically coincide, that is, there is basically no difference between the two. Record the current emission angle of the light source at this time as the calibrated emission angle. It should be noted that the emission angle can be adjusted by a preset angle each time, and the preset angle is usually a small angle.

[0077] As another possible implementation, adjusting the emission angle of the light source includes: determining the offset direction and offset amount of the actual imaging pixels of the light source in the spot imaging diagram relative to the expected imaging pixels, and adjusting the emission angle of the light source according to the offset direction and offset amount.

[0078] As a non-limiting example, as Figure 8 shown, for the spot beam emitted by a certain light source, the spot beam reflected by the target object is imaged on the pixel array, and the imaging spot of this light source in the spot imaging diagram is Figure 8 as shown by the solid circle 81 in [Figure], while the spot in the expected standard spot diagram is Figure 8 as shown by the dashed circle 82 in [Figure]. It is determined that the solid circle 81 is shifted to the left by one pixel relative to the dashed circle 82, as shown by the black arrow 83 in the figure. Therefore, the emission angle of the light source can be adjusted to shift the imaging spot to the right by one pixel, so that the solid circle 81 and the dashed circle 82 basically coincide, that is, there is basically no difference between the two. Record the current emission angle of the light source at this time as the calibrated emission angle.

[0079] In some embodiments, the light source can adjust the emission angle under the control of the control and processing circuit and the drive of the drive circuit. In other embodiments, reflection optical elements such as MEMS mirrors are arranged on the light output path of the light source array, and the reflection optical elements such as MEMS mirrors adjust the emission angle of the light source under the control of the control and processing circuit.

[0080] It should be noted that the imaging position change trends of the spots of different light sources in the light source array are basically the same. Therefore, the calibrated emission angles of other unactivated light sources can be calibrated by analogy according to the calibrated emission angle of the activated light source, so that all light sources can be calibrated at one time. During the detection process after calibration, if different working light sources are switched to emit spot beams to detect the target object, calibration may not be required anymore. Instead, the light sources to be activated, that is, the working light sources, can be activated according to the previous calibration results and the calibrated emission angles.

[0081] Figure 9 Shown is a calibration method for a detection system based on DToF provided by another embodiment of the present application. As Figure 9As shown, the calibration method of the DToF-based detection system may include the following steps S91 to S95. It should be noted that the same parts as those in the foregoing embodiments are not described herein again.

[0082] S91, controlling the emitter to emit a speckle beam towards the target according to a preset projection pattern.

[0083] S92, synchronously activating the pixel array in the collector to receive at least part of the speckle beam reflected by the target.

[0084] Wherein, at least part of the pixels in the pixel array receive at least part of the speckle beam reflected by the target and output photon detection signals to the readout circuit.

[0085] S93, generating a grayscale image according to the number of photon detection signals of the pixels in the pixel array.

[0086] In one embodiment, the readout circuit is further configured to count the number of photon detection signals generated by each pixel in the pixel array and transmit the number to the control and processing circuit. The control and processing circuit uses the number as the grayscale value of the pixel to obtain a grayscale image.

[0087] In another embodiment, the control and processing circuit is further configured to count the number of photon detection signals generated by each pixel in the pixel array. Specifically, the control and processing circuit sums up the number of photons in each time bin of the histogram corresponding to each pixel in the pixel array, and then uses the sum result as the grayscale value of the pixel to obtain a grayscale image.

[0088] S94, inputting the grayscale image into the first neural network model to obtain a speckle imaging map.

[0089] In some cases, due to the limitation of the number of samples, the robustness of the first neural network model is not strong enough, so errors may occur. To avoid the adverse effects of such errors on the calibration result, the embodiment of the present application sets a difference threshold. When it is determined that the difference between the speckle imaging map and the expected standard speckle map is less than the threshold, it indicates that the result has a high accuracy, and step S95 is continued; otherwise, it indicates that there may be an error in the result, and a new preset projection pattern needs to be obtained to repeat the calibration steps, that is, steps S96 to S98 are continued.

[0090] It should be noted that the difference may include an offset, etc. The threshold may be an empirical value, and the embodiment of the present application does not limit this. In addition, it should be understood that in other equivalent solutions, when it is determined that the difference is equal to the threshold, step S95 may also be continued.

[0091] S95. If it is determined that the difference between the spot imaging diagram and the expected standard spot diagram is less than the threshold, adjust the emission angle of the light source until there is no difference between the newly generated spot imaging diagram and the expected standard spot diagram, and record the current emission angle of the light source as the calibrated emission angle.

[0092] S96. If it is determined that the difference between the spot imaging diagram and the expected standard spot diagram is greater than or equal to the threshold, obtain a new preset projection pattern. If it is determined that the number of times of obtaining the preset projection pattern is less than or equal to the preset number of times, continue to execute step S97; if it is determined that the number of times of obtaining the preset projection pattern is greater than the preset number of times, continue to execute S98.

[0093] S97. If it is determined that the number of times of obtaining the preset projection pattern is less than or equal to the preset number of times, return to execute step S91 and the subsequent steps.

[0094] S98. If it is determined that the number of times of obtaining the preset projection pattern is greater than the preset number of times, end.

[0095] Specifically, when it is determined that the number of times of obtaining the preset projection pattern is greater than the preset number of times, the process can be ended. At this time, the detection result of the damage of the detection system can also be output because there are large deviations between the spot imaging diagrams of several preset projection patterns and the expected standard spot diagram, which is very likely due to damage to the detection system caused by dropping, collision, etc. during use. When it is determined that the difference between the new spot imaging diagram and the expected standard spot diagram is less than the threshold, adjust the emission angle of the light source according to the comparison result between the new spot imaging diagram and the expected standard spot diagram.

[0096] It should be noted that the preset number of times can be set to any integer number of times such as 3 times or more than 3 times.

[0097] An embodiment of the present application also provides an electronic device. Refer to Figure 10 As shown, the electronic device includes: a processor 1000, a memory 1001, and a computer program 1002 stored in the memory 1001 and executable on the processor 1000, such as a calibration program for a DToF-based detection system. When the processor 1000 executes the computer program 1002, it implements the steps in the calibration method embodiment of the DToF-based detection system in any of the above embodiments, such as Figure 3 the steps S31 to S33 shown.

[0098] Exemplarily, the computer program 1002 can be divided into one or more modules / units, which are stored in the memory 1001 and executed by the processor 1000 to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program 1002 in the electronic device.

[0099] Those skilled in the art can understand that Figure 10 merely examples of electronic devices, which do not constitute a limitation on the electronic devices. An electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different components. For example, an electronic device may also include input / output devices, network access devices, buses, etc.

[0100] The so-called processor 1000 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0101] The memory 1001 may be an internal storage unit of the electronic device, such as the hard disk or memory of the electronic device. The memory 1001 may also be an external storage device of the electronic device, such as a plug-in hard disk equipped on the electronic device, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 1001 may also include both the internal storage unit and the external storage device of the electronic device. The memory 1001 is used to store the computer program and other programs and data required by the electronic device. The memory 1001 may also be used to temporarily store data that has been output or will be output.

[0102] An embodiment of the present application further provides an electronic device. The electronic device includes the DToF-based detection system according to any one of the foregoing embodiments, wherein the transmitter and the collector of the detection system are arranged on the same side of the electronic device body.

[0103] As a non-limiting example, the electronic device may be a lidar, etc.

[0104] An embodiment of the present application provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps in the above-described embodiments of the calibration method for each DToF-based detection system can be implemented.

[0105] An embodiment of the present application provides a computer program product, and when the computer program product runs on an electronic device, the electronic device can implement the steps in the above-described embodiments of the calibration method for each DToF-based detection system.

[0106] In the above embodiments, the descriptions of the various embodiments have their own emphases. For parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0107] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application.

[0108] In the embodiments provided by the present application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be in electrical, mechanical or other forms.

[0109] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0110] In addition, the functional units in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0111] When the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-described embodiment methods of this application, it can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, ROM, RAM, electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice within the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0112] The foregoing are only the preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent replacements, and improvements made within the spirit and principle of this application shall be included within the protection scope of this application.

Claims

1. A calibration method based on a DToF system, characterized in that, Including: S1. Control a transmitter to emit a speckle beam towards a target according to a preset projection pattern; the transmitter includes a light source array composed of a plurality of light sources; The preset projection pattern identifies the working light source; S2. Synchronously activate a pixel array in a collector to receive at least part of the speckle beam reflected by the target; at least part of the pixels in the pixel array receive the speckle beam and output photon detection signals; S3. Generate a grayscale image according to the number of the photon detection signals, and determine a calibrated emission angle of the light source according to a comparison result between the grayscale image and an expected standard speckle pattern; wherein, the predicted imaging position of the speckle beam emitted by each light source on the pixel array is theoretically calculated in advance according to parameters of a detection system, and the expected standard speckle pattern is determined according to the light sources turned on in the preset projection pattern, and the expected standard speckle pattern identifies the predicted imaging position of the working light source in the pixel array; Determining the calibrated emission angle of the light source according to the comparison result between the grayscale image and the expected standard speckle pattern includes: Input the grayscale image into a first neural network model to obtain a speckle imaging map, compare the speckle imaging map with the expected standard speckle pattern, and if it is determined that there is no difference between the speckle imaging map and the expected standard speckle pattern, record the current emission angle of the light source as the calibrated emission angle; If it is determined that the difference between the speckle imaging map and the expected standard speckle pattern is less than a threshold, adjust the emission angle of the light source until there is no difference between the newly generated speckle imaging map and the expected standard speckle pattern, and record the current emission angle of the light source as the calibrated emission angle; If it is determined that the difference between the speckle imaging map and the expected standard speckle pattern is greater than or equal to the threshold, obtain a new preset projection pattern, and return to execute step S1 and subsequent steps until the number of times of obtaining the preset projection pattern is greater than a preset number, or until it is determined that the difference between the new speckle imaging map and the expected standard speckle pattern is less than the threshold, adjust the emission angle of the light source until there is no difference between the newly generated speckle imaging map and the expected standard speckle pattern, and record the current emission angle of the light source as the calibrated emission angle.

2. The calibration method according to claim 1, wherein Adjusting the emission angle of the light source includes: Determine the offset direction of the actual imaging pixel of the light source in the speckle imaging map relative to the expected imaging pixel, and adjust the emission angle of the light source according to the offset direction; or Determine the offset direction and offset amount of the actual imaging pixel of the light source in the speckle imaging map relative to the expected imaging pixel, and adjust the emission angle of the light source according to the offset direction and the offset amount.

3. The calibration method according to any one of claims 1 to 2, characterized in that, It further includes: Control the working light sources in the light source array to emit detection beams towards the target according to the calibrated emission angle; Synchronously control the working pixels corresponding to the working light sources in the pixel array to be turned on and receive the detection beams to output photon detection signals to a readout circuit; the readout circuit is used to generate a time-of-flight value according to the photon detection signals output by the working pixels.

4. A detection system based on DToF, characterized in that, Including: A transmitter, a collector, and a control and processing circuit The emitter includes a light source array composed of a plurality of light sources, and the emitter is configured to emit a speckle beam towards the target object according to a preset projection pattern; The preset projection pattern identifies the working light source; The collector includes a pixel array composed of a plurality of pixels, and the collector is configured to activate the pixel array under the control of the control and processing circuit to receive at least part of the speckle beam reflected back by the target object. At least part of the pixels in the pixel array receive the speckle beam and output photon detection signals; The control and processing circuit is configured to generate a grayscale image according to the number of the photon detection signals, and determine the calibrated emission angle of the light source according to the comparison result between the grayscale image and an expected standard speckle pattern. The expected standard speckle pattern is determined according to the preset projection pattern and identifies the predicted imaging position of the working light source in the pixel array; Determining the calibrated emission angle of the light source according to the comparison result between the grayscale image and the expected standard speckle pattern includes: Inputting the grayscale image into a first neural network model to obtain a speckle imaging map, comparing the speckle imaging map with the expected standard speckle pattern. If it is determined that there is no difference between the speckle imaging map and the expected standard speckle pattern, record the current emission angle of the light source as the calibrated emission angle; If it is determined that the difference between the speckle imaging map and the expected standard speckle pattern is less than a threshold, adjust the emission angle of the light source until there is no difference between the newly generated speckle imaging map and the expected standard speckle pattern, and record the current emission angle of the light source as the calibrated emission angle; If it is determined that the difference between the speckle imaging map and the expected standard speckle pattern is greater than or equal to the threshold, obtain a new preset projection pattern, and return to execute step S1 and subsequent steps until the number of times of obtaining the preset projection pattern is greater than a preset number, or until it is determined that the difference between the new speckle imaging map and the expected standard speckle pattern is less than the threshold, adjust the emission angle of the light source until there is no difference between the newly generated speckle imaging map and the expected standard speckle pattern, and record the current emission angle of the light source as the calibrated emission angle.

5. The DToF-based detection system according to claim 4, characterized in that, It further includes a readout circuit. The readout circuit or the control and processing circuit includes a counting circuit, and the counting circuit is used to count the number of photon detection signals output by each pixel; The readout circuit further includes a TDC circuit, a histogram circuit and a readout unit. The TDC circuit is used to generate a time signal according to the photon detection signals output by the correspondingly connected pixels. The histogram circuit is used to generate a histogram according to the time signal. The readout unit is used to generate a time-of-flight value according to the histogram; The control and processing circuit is further configured to control the working light source in the light source array to emit a detection beam towards the target object according to the calibrated emission angle; synchronously control the working pixels corresponding to the working light source in the pixel array to be turned on and receive the detection beam to output photon detection signals to the readout circuit, and the readout circuit is used to generate a time-of-flight value according to the photon detection signals output by the working pixels.

Images