Detection device and terminal equipment

Abstract

A detection device and terminal device can be applied to fields such as autonomous driving or intelligent driving. The detection device may include: a transmitting component that emits a first light beam; a spectroscopic component that changes the optical path of an echo light signal from a detection area to obtain a first echo light signal and a second echo light signal, and then transmits the first echo light signal to a first detection component and the second echo light signal to a second detection component; the first detection component detects the first echo light signal to obtain a first electrical signal, and the second detection component detects the second echo light signal to obtain a second electrical signal, and the first and second electrical signals are used to determine the distance information of the target; when the detection distance is less than a preset value, the detection accuracy of the first detection component is greater than the detection accuracy of the second detection component; when the detection distance is not less than the preset value, the detection accuracy of the first detection component is not greater than the detection accuracy of the second detection component. In this way, the detection device can achieve high-precision detection over the entire detection range.

Description

Technical Field

[0001] The present application relates to the field of detection technology, and in particular to a detection device and terminal equipment. Background Art

[0002] With the development of informatization, smart terminals are gradually becoming part of people's daily lives. Sensing systems are playing an increasingly important role in smart terminals. Currently, sensing systems are widely applied in a variety of fields, such as industrial production, space exploration, ocean exploration, environmental protection, resource surveys, medical diagnosis, and bioengineering. Among them, three-dimensional (3D) sensing systems are a current research hotspot in the field of sensing systems. They can acquire complete geometric information in a 3D scene and accurately digitize the scene using images with depth information, thereby enabling high-precision recognition, positioning, reconstruction, and scene understanding.

[0003] Technologies suitable for 3D sensing systems primarily include stereo imaging, structured light, and time-of-flight (TOF) technology. TOF is a key technology used in 3D sensing systems due to its advantages, such as long detection range and high resolution. TOF technology measures the round-trip time between the transmitting component, the target, and the receiving component, and obtains accurate distance information based on this round-trip time and the speed of light. TOF technology is primarily divided into two categories: direct time of flight (d-TOF) and indirect time of flight (i-TOF). The depth measurement accuracy of d-TOF technology is independent of detection distance and is typically used for long-range detection. However, when applied to short-range detection scenarios (e.g., less than 0.5m), the measurement accuracy falls far short of the required level. The depth measurement accuracy of i-TOF technology decreases with increasing detection distance, and its accuracy is also low when applied to long-range detection scenarios.

[0004] In summary, how to achieve high-precision measurement within the entire detection distance range is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] The present application provides a detection device and a terminal device for achieving high-precision measurement within the full detection distance range.

[0006] In a first aspect, the present application provides a detection device, which may include a transmitting component and a receiving component, the receiving component including a spectroscopic component, a first detection component and a second detection component; when the detection distance is less than a preset value, the detection accuracy of the first detection component is greater than the detection accuracy of the second detection component, and when the detection distance is not less than the preset value, the detection accuracy of the first detection component is not greater than the detection accuracy of the second detection component; the transmitting component is used to transmit a first light beam; the spectroscopic component is used to change the propagation path of the echo light signal for the first light beam from the detection area, obtain the first echo light signal and the second echo light signal, and propagate the first echo light signal to the first detection component and propagate the second echo light signal to the second detection component; the first detection component is used to detect the received first echo light signal to obtain a first electrical signal; the second detection component is used to detect the received second echo light signal to obtain a second electrical signal; wherein the first electrical signal and the second electrical signal are used to determine the distance information of the target in the detection area.

[0007] Based on this solution, the optical splitter changes the propagation path of the return light signal from the detection area, thereby transmitting the first return light signal to the first detection component and the second return light signal to the second detection component. Furthermore, since the detection accuracy of the first detection component is greater than that of the second detection component when the detection distance is less than a preset value, and the detection accuracy of the first detection component is no greater than that of the second detection component when the detection distance is not less than the preset value, the detection accuracy of the first detection component is not greater than that of the second detection component when the detection distance is less than the preset value. In this way, when the detection distance is less than the preset value, the first detection component is used to detect the target; when the detection distance is not less than the preset value, the second detection component is used to detect the target, thus achieving high-precision detection across the entire detection range.

[0008] In a possible implementation, the first detection component includes an i-TOF image sensor, and the second detection component includes a d-TOF image sensor.

[0009] The i-TOF image sensor is suitable for close-range detection and has higher detection accuracy at close range. The d-TOF image sensor is suitable for long-range detection and has higher detection accuracy than the i-TOF image sensor at long range. The i-TOF and d-TOF image sensors work together to achieve high-precision detection across the entire detection range.

[0010] In a possible implementation, the optical splitting component is configured to propagate the first echo optical signal to the first detection component at the second timing, or propagate the second echo optical signal to the second detection component at the third timing, according to the received second control signal.

[0011] By transmitting the corresponding echo optical signals to different detection components at different time sequences through the optical splitting component, the first detection component can work at the second time sequence and the second detection component can work at the third time sequence, that is, the two detection components work at different time sequences.

[0012] Furthermore, optionally, the second time sequence and the third time sequence are arranged alternately. It can also be understood that the first detection component and the second detection component work alternately.

[0013] Because the first and second detection assemblies operate alternately, this helps improve the dynamic detection capabilities of the detection device. This can reduce the offset between the first and second detection assemblies, especially for high-speed moving objects, helping to avoid issues such as smearing in the fused image that affect image quality.

[0014] The time-division-based optical splitting component can be, for example, liquid crystal on silicon (LCOS), an optical switch, an optical fiber circulator, or a digital micromirror device (DMD).

[0015] In a possible implementation, the transmitting component is used to transmit the first light beam according to a received first control signal, and the first control signal is used to control a first timing of the transmitting component to transmit the first light beam.

[0016] By controlling the timing of the emission of the first light beam by the emission component, the emission component can be synchronized with the first detection component and the second detection component to further improve the detection accuracy of the detection device.

[0017] Further, optionally, the transmitting component is used to transmit a first light beam according to a received first modulation signal, wherein the first modulation signal can be a pulse wave or a continuous wave.

[0018] In a possible implementation, the optical splitting component is used to split the echo optical signal to obtain a first echo optical signal and a second echo optical signal, and propagate the first echo optical signal to the first detection component and the second echo optical signal to the second detection component.

[0019] By performing spatial division on the received echo light signal through the light splitting component, the first detection component and the second detection component can be operated simultaneously, thereby helping to increase the imaging speed of the depth image of the target.

[0020] The light splitting component based on space division can be, for example, a beam splitter or a diffractive optical device.

[0021] In a possible implementation, the transmitting component is configured to transmit the first light beam according to a received second modulation signal, where the second modulation signal includes a pulse wave.

[0022] In this way, the first echo optical signal and the second echo optical signal can both be pulse waves, so that the first detection component and the second detection component can be in the working mode at the same time.

[0023] In one possible implementation, the detection device further includes a processing control component for receiving a first electrical signal from the first detection component and a second electrical signal from the second detection component; and determining distance information of the target based on the first electrical signal and the second electrical signal.

[0024] Further, optionally, the processing and control component can be used to determine the phase difference between the emission of the first light beam and the reception of the first echo light signal based on the first electrical signal, and determine the first time difference between the emission of the first light beam and the reception of the first echo light signal based on the phase difference; determine the second time difference between the emission of the first light beam and the reception of the second echo light signal based on the second electrical signal; determine the first distance information of the target based on the first time difference; and determine the second distance information of the target based on the second time difference.

[0025] In one possible implementation, the processing and control component is further configured to determine error distance information corresponding to a multipath interference region or an intensity error region in the first distance information; determine target distance information corresponding to the error distance information in the second distance information; and replace the error distance information with the target distance information. This can also be understood as removing the error distance information corresponding to the multipath interference region or the intensity error region in the first distance information and replacing it with the target distance information corresponding to the error distance information in the second distance information.

[0026] Since the i-TOF image sensor will produce multipath interference areas or intensity error areas, by eliminating the error distance information corresponding to the multipath interference areas or intensity error areas in the first distance information corresponding to the i-TOF image sensor and replacing it with the target distance information corresponding to the error distance information in the second distance information, it helps to improve the accuracy of the distance information of the target detected by the detection device, and thus improve the quality of the depth image formed of the target.

[0027] In one possible implementation, the processing and control component is further used to eliminate distance information in the first distance information whose distance is greater than a preset value, obtain third distance information, and generate a first image based on the third distance information; eliminate distance information in the second distance information whose distance is not greater than the preset value, obtain fourth distance information, and generate a second image based on the fourth distance information; and fuse the first image and the second image to obtain a depth image of the target.

[0028] Since the first distance information is obtained based on the first electrical signal of the first detection component, and the second distance information is obtained based on the second electrical signal of the second detection component, and when the detection distance is less than the preset value, the detection accuracy of the first detection component is greater than that of the second detection component, therefore, by eliminating the distance information in the first distance information whose distance is greater than the preset value, a third distance information with higher accuracy can be obtained; when the detection distance is not less than the preset value, the detection accuracy of the first detection component is not greater than the detection accuracy of the second detection component, therefore, by eliminating the distance information in the second distance information whose distance is not greater than the preset value, a high-precision fourth distance information is obtained, and the depth image of the target obtained based on the third distance information and the fourth distance information has a high detection accuracy within the entire detection distance range.

[0029] In one possible implementation, the processing and control component is further used to multiply the distance information in the first distance information whose distance is greater than a preset value by a first confidence level, and multiply the distance information in the first distance information whose distance is not greater than the preset value by a second confidence level, to obtain fifth distance information, and generate a third image based on the fifth distance information, wherein the second confidence level is greater than the first confidence level; multiply the distance information in the second distance information whose distance is not greater than the preset value by a third confidence level, and multiply the distance information in the second distance information whose distance is greater than the preset value by a fourth confidence level, to obtain sixth distance information, and generate a fourth image based on the sixth distance information, wherein the fourth confidence level is greater than the third confidence level; and fuse the third image and the fourth image to obtain a depth image of the target.

[0030] Since the first distance information is obtained based on the first electrical signal of the first detection component, and the second distance information is obtained based on the second electrical signal of the second detection component, and when the detection distance is less than a preset value, the detection accuracy of the first detection component is greater than that of the second detection component, and when the detection distance is not less than the preset value, the detection accuracy of the first detection component is not greater than that of the second detection component, therefore, the first distance information is divided into two parts, the part not greater than the preset value is multiplied by the larger second confidence level, and the part greater than the preset value is multiplied by the smaller first confidence level, to obtain the fifth distance information; the second distance information is also divided into two parts, the part not greater than the preset value is multiplied by the smaller third confidence level, and the part greater than the preset value is multiplied by the larger fourth confidence level, to obtain the sixth distance information. The depth image of the target obtained based on the fifth and sixth distance information has high detection accuracy throughout the entire detection distance range.

[0031] In one possible implementation, the processing and control component is further used to multiply the first distance information by a fifth confidence level to obtain seventh distance information, and generate a fifth image based on the seventh distance information, where the fifth confidence level is negatively correlated with the first distance information; multiply the second distance information by a sixth confidence level to obtain eighth distance information, and generate a sixth image based on the eighth distance information, where the sixth confidence level is positively correlated with the second distance information; and fuse the fifth image and the sixth image to obtain a depth image of the target.

[0032] Since the first distance information is obtained based on the first electrical signal of the first detection component, and the second distance information is obtained based on the second electrical signal of the second detection component, and when the detection distance is less than a preset value, the detection accuracy of the first detection component is greater than that of the second detection component, and when the detection distance is not less than the preset value, the detection accuracy of the first detection component is not greater than that of the second detection component. Therefore, as the distance of the first distance information increases, the fifth confidence factor multiplied by the first distance information decreases. Conversely, as the distance of the second distance information increases, the sixth confidence factor multiplied by the second distance information decreases. In this way, the sixth distance information can occupy a larger proportion at close distances and a smaller proportion at long distances; conversely, the eighth distance information can occupy a smaller proportion at close distances and a larger proportion at long distances; thereby, the depth image of the target obtained based on the seventh distance information and the eighth distance information can have a high detection accuracy throughout the entire detection distance range.

[0033] In a second aspect, the present application provides a terminal device comprising the above-mentioned first aspect or any one detection device in the first aspect and a processor, wherein the processor can be used to control the detection device to detect a detection area.

[0034] In one possible implementation, the terminal device may be, for example, a radar (such as a lidar), a smart phone, a vehicle, a smart home device, a smart manufacturing device, a robot, a drone, or a smart transportation device.

[0035] The technical effects that can be achieved in the second aspect can be referred to the description of the beneficial effects in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1a A schematic diagram of the ranging principle of the d-TOF technology provided in this application;

[0037] Figure 1b A schematic diagram of the ranging principle of the i-TOF technology provided in this application;

[0038] Figure 1c A schematic diagram of the principle of multipath interference provided by this application;

[0039] Figure 2a A schematic diagram of possible application scenarios of a detection device provided in this application;

[0040] Figure 2b A schematic diagram of a possible application scenario of another detection device provided in this application;

[0041] Figure 2c A schematic diagram of a possible application scenario of another detection device provided in this application;

[0042] Figure 3 A schematic structural diagram of a detection device provided in this application;

[0043] Figure 4 A schematic diagram of timing control of a first light beam emitted by a light source provided in this application;

[0044] Figure 5a A schematic diagram of the structure of an LCoS provided in this application;

[0045] Figure 5b A schematic diagram of the structure of an optical switch provided in this application;

[0046] Figure 5c A schematic structural diagram of another optical switch combination provided in this application;

[0047] Figure 5d A schematic structural diagram of a fiber optic circulator provided in this application;

[0048] Figure 6 A schematic structural diagram of a beam splitter provided in this application;

[0049] Figure 7a A schematic structural diagram of an i-TOF image sensor provided in this application;

[0050] Figure 7b A schematic structural diagram of a d-TOF image sensor provided in this application;

[0051] Figure 8 A schematic diagram of the ranging range of two image sensing components provided in this application;

[0052] Figure 9a A schematic structural diagram of another detection device provided in this application;

[0053] Figure 9b A schematic structural diagram of another detection device provided in this application;

[0054] Figure 10 A schematic diagram of the structure of a terminal device provided in this application. DETAILED DESCRIPTION

[0055] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0056] The following is an explanation of some of the terms used in this application. It should be noted that these explanations are for the purpose of facilitating understanding by those skilled in the art and do not limit the scope of protection claimed in this application.

[0057] 1. Detection accuracy

[0058] Detection accuracy refers to the minimum distance at which two different targets can be distinguished.

[0059] 2. Image fusion

[0060] Image fusion is an image processing technology that refers to the process of extracting the beneficial information from each channel to the maximum extent through image processing and calculation of specific algorithms from image data of the same target collected by multiple source channels, and finally fusing high-quality images (such as brightness, clarity, and color). The fused image has a higher resolution than the original image.

[0061] The foregoing text introduces some of the terms used in this application. The following text introduces the technical features involved in this application. It should be noted that these explanations are for the purpose of facilitating understanding by those skilled in the art and do not limit the scope of protection claimed by this application.

[0062] like Figure 1a Figure 1 is a schematic diagram of the distance measurement principle of a d-TOF technology provided by this application. The d-TOF technology directly measures the difference between the emission time t1 of the signal light and the reception time t2 of the echo light signal (i.e., t2-t1). The echo light signal is the reflection of the signal light by the target in the detection area. The target distance information is then calculated according to d = C × (t2-t1), where d represents the target distance and C represents the speed of light.

[0063] It should be noted that the signal light is typically pulsed laser light. Due to laser safety restrictions and the power consumption limits of the detection device, the energy of the emitted signal light is limited. However, the complete detection area must be covered. Therefore, when the signal light is reflected by the target and the return signal to the receiver, energy loss is significant. Furthermore, ambient light, acting as noise, interferes with the receiver's detection and restoration of the return signal. Therefore, d-ToF technology requires a highly sensitive detector to detect the return signal. For example, a single-photon avalanche diode (SPAD) has the sensitivity to detect single photons. In operation, a SPAD is a diode biased by a high reverse voltage. This reverse bias creates a strong electric field within the device. When a photon is absorbed by the SPAD and converted into a free electron, this free electron is accelerated by the internal electric field, gaining sufficient energy to collide with other atoms to produce free electron-hole pairs. The newly generated carriers are further accelerated by the electric field, generating more carriers. This geometrically amplified avalanche effect gives the SPAD nearly infinite gain, resulting in a high current pulse output, enabling the detection of single photons.

[0064] like Figure 1b As shown, it is a schematic diagram of the ranging principle of an i-TOF technology provided by the present application. The i-TOF technology uses phase to indirectly obtain the time difference between the emission time of the signal light and the reception time of the echo light signal. Specifically, by collecting the charge value transferred by the current-assisted photonic demodulator (CAPD) in different time windows (i.e., the C1 time window and the C2 time window), the phase delay between the echo light signal and the signal light is analyzed, and the distance information of the target is calculated according to d=1 / 2*c*ΔT*Q2 / (Q1+Q2), where d represents the distance of the target, C represents the speed of light, ΔT represents the pulse width of the light signal emitted by the light source, Q1 represents the charge value transferred by the current-assisted photonic demodulator in the C1 time window, Q2 represents the charge value transferred by the current-assisted photonic demodulator in the C2 time window, and ΔT*Q2 / (Q1+Q2) represents the time difference. Generally, i-ToF technology can be categorized into continuous wave (CW) modulation and pulse (PL) modulation based on the modulation method. Continuous wave modulation transmits a continuous sinusoidal signal, and distance is determined by analyzing the sinusoidal signal phase. Pulse modulation transmits a repetitive pulse signal, and distance is determined by analyzing the pulse signal phase.

[0065] In actual application scenarios, there are complex diffuse reflections and even specular reflections. Multipath interference (MPI) will in principle increase the distance measurement value, thus affecting the effect of 3D reconstruction. Figure 1c As shown, it is a schematic diagram of the principle of multipath interference provided by this application. Figure 1c Taking the measurement of the base of a wall as an example, the light emitted by the transmitting component to the left (dashed line) is reflected twice and is received by the image sensor at the same time as the light emitted to the right (solid line). The double depth information causes errors in the distance information measured by the image sensor.

[0066] It should be noted that the detected intensity error may also affect the 3D reconstruction effect. For example, it can manifest as areas with different reflectivity on the same plane appearing at different distances. For example, if the detected target is a black and white checkerboard, the detection result may be uneven.

[0067] Based on the above, the following are possible application scenarios of the detection device in this application. For example, the detection device can be a camera module (such as a depth camera), which can be installed on a vehicle (such as an unmanned vehicle, a smart car, an electric car, a digital car, etc.) as a vehicle-mounted camera. Figure 2a . On-board cameras can obtain measurement information such as the distance of surrounding objects in real time or periodically, thereby providing necessary information for operations such as lane correction, distance keeping, and reversing. Because on-board cameras can achieve: a) target recognition and classification, such as various lane line recognition, traffic light recognition, and traffic sign recognition, etc.; b) traversable space detection (FreeSpace), for example, the safe boundary (drivable area) for vehicle driving can be divided, mainly dividing vehicles, ordinary road edges, curb edges, boundaries without visible obstacles, unknown boundaries, etc.; c) the ability to detect laterally moving targets, such as the detection and tracking of pedestrians and vehicles crossing the intersection; d) positioning and map creation, such as positioning and map creation based on visual simultaneous localization and mapping (SLAM) technology.

[0068] The detection device can also be a laser radar, which can also be installed on a vehicle as a vehicle-mounted laser radar. Figure 2bOn-board LiDAR can acquire real-time or periodic measurement information such as the vehicle's latitude and longitude, speed, direction, and distance to surrounding objects. Based on this information and in conjunction with the Advanced Driving Assistant System (ADAS), it can implement assisted or autonomous driving. For example, the vehicle's position can be determined using longitude and latitude, or its future direction and destination can be determined using speed and direction, or the number and density of obstacles around the vehicle can be determined using the distance to surrounding objects. On-board LiDAR can also implement mapping functions.

[0069] Combined with the above Figure 2b The figure shows a schematic diagram of the detection principle of a laser radar provided by this application. The laser radar emits a laser beam in a certain direction. If there is a target within a certain distance along the emission direction of the laser beam, the target can reflect the received laser beam back to the laser radar (called an echo light signal). The laser radar can determine the target information based on the echo light signal, such as the distance to the target, the target's moving speed, the target's posture or point cloud map, etc. It should be understood that this example uses an example of a laser radar deployed at the front end of a vehicle. The laser radar can sense the fan-shaped area shown in the dotted box, which can be called the detection area of ​​the laser radar.

[0070] Detection devices have been widely used in the fields of unmanned driving, automatic driving, assisted driving, intelligent driving, connected vehicles, security monitoring, surveying and mapping. It should be noted that the application scenarios shown above are only examples, and the detection device provided in this application can also be used in a variety of other scenarios, not limited to the scenarios exemplified above. For example, the detection device can also be applied to a terminal device or a component provided in a terminal device. The terminal device can be, for example, a smart phone, smart home device, smart manufacturing equipment, a robot, a drone or an intelligent transportation equipment (such as an automated guided vehicle (AGV) or an unmanned transport vehicle, etc.). For another example, the detection device can also be installed on a drone as an airborne detection device, etc. For another example, the detection device can also be installed on a roadside traffic device (such as a road side unit (RSU)) as a roadside traffic detection device, see Figure 2c , thus realizing intelligent vehicle-road collaboration.

[0071] As described in the background art, the detection device in the prior art cannot achieve high-precision measurement within the entire detection distance range.

[0072] In view of this, the present application provides a detection device that can achieve high-precision measurement within the full detection distance range.

[0073] The following is combined with Figure 3 To the attached Figure 9b , the detection device proposed in this application is explained in detail.

[0074] Based on the above, if Figure 3 FIG2 is a schematic diagram of the structure of a detection device provided by the present application. The detection device may include a transmitting component 301 and a receiving component 302. The receiving component 302 includes a spectroscopic component 3021, a first detection component 3022, and a second detection component 3023. When the detection distance is less than a preset value, the detection accuracy of the first detection component 3022 is greater than that of the second detection component 3023; when the detection distance is not less than the preset value, the detection accuracy of the first detection component 3022 is not greater than that of the second detection component 3023. The transmitting component 301 is configured to emit a first light beam. The spectroscopic component 3021 is configured to change the propagation path of the echo light signal for the first light beam from the detection area to obtain a first echo light signal and a second echo light signal, and then propagate the first echo light signal to the first detection component and the second echo light signal to the second detection component. The first detection component 3022 is configured to detect the received first echo light signal and obtain a first electrical signal. Alternatively, the first detection component 3022 may be configured to perform photoelectric conversion on the received first echo light signal to obtain the first electrical signal. Second detection component 3023 is used to detect the received second echo light signal and obtain a second electrical signal. Alternatively, second detection component 3033 is used to perform photoelectric conversion on the received second echo light signal to obtain a second electrical signal. The first and second electrical signals are used to determine the distance information of the target in the detection area.

[0075] Based on the above-described detection device, the propagation path of the echo light signal from the detection area is altered by the spectroscopic component, thereby enabling the propagation of the first echo light signal to the first detection component and the second echo light signal to the second detection component. Furthermore, since the detection accuracy of the first detection component is greater than that of the second detection component when the detection distance is less than a preset value, and the detection accuracy of the first detection component is no greater than that of the second detection component when the detection distance is not less than the preset value, the detection accuracy of the first detection component is no greater than that of the second detection component when the detection distance is less than the preset value. In this way, when the detection distance is less than the preset value, the target is detected based on the first detection component; when the detection distance is not less than the preset value, the target is detected based on the second detection component, achieving high-precision detection across the entire detection range. Furthermore, when three-dimensional modeling is performed based on the obtained high-precision distance information, an accurate three-dimensional model can be obtained.

[0076] Here, the first detection component is suitable for short-range detection, and the second detection component is suitable for long-range detection. In one possible implementation, the first detection component includes an i-TOF image sensor, and the second detection component includes a d-TOF image sensor.

[0077] In one possible implementation, the preset value may be the distance at which the detection accuracy of the first detection component is the same as the detection accuracy of the second detection component. For example, if the detection accuracy of the first detection component is 2% of the distance and the detection accuracy of the second detection component is 2 nanoseconds (ns), that is, a constant detection distance of c*t / 2=0.3 meters (m), then the preset value may be 15m, where c is the speed of light. Figure 8 , Z1 represents the preset value.

[0078] In a possible implementation, the intensities of the first echo optical signal and the second echo optical signal may be the same or different, which is not limited in this application.

[0079] It should be noted that when the detection device is a camera module, the detection area can also be understood as the field of view of the camera module. The target object includes but is not limited to a single object. For example, when photographing a person, the target object includes the person and the environment around the person, that is, the environment around the person is also part of the target object.

[0080] Below Figure 3 Each functional component shown is introduced and explained separately to provide an exemplary specific implementation scheme. For the convenience of explanation, the transmitting component, receiving component, the optical splitting component included in the receiving component, the first detection component and the second detection component are not labeled below.

[0081] 1. Launch Components

[0082] In a possible implementation, the emitting component may include a light source. It can also be understood that the detection device may include a self-luminous light source.

[0083] In one possible implementation, the light source may be a single light source or a light source array consisting of multiple light sources. For example, the light source may be a vertical cavity surface emitting laser (VCSEL) or an edge emitting laser (EEL). EEL-based light sources can be independently addressable, meaning that any light source in the light source array can be independently strobed (or lit, turned on, or powered on).

[0084] In one possible implementation, the first light beam emitted by the light source may be visible light, infrared light, or ultraviolet light. For example, the wavelength of the first light beam emitted by the light source may be in the range of 905 nanometers (nm). Furthermore, optionally, the transmitting module may also include a diffuser (also known as a beam expander or a light homogenizing element). The diffuser may be used to expand the first light beam emitted by the light source into a uniform first light beam and project the uniform first light beam into the detection area. Here, the uniform first light beam may also be referred to as a flood beam.

[0085] In one possible implementation, the light source may control the first timing of emitting the first light beam based on a received first control signal. Furthermore, optionally, the light source may also determine the waveform of the first light beam based on a received first modulation signal. The first modulation signal may be a pulse wave or a continuous wave, for example, a sine wave or a square wave.

[0086] like Figure 4 , which is a schematic diagram of timing control of a first light beam emitted by a light source provided in the present application. Figure 4 The first time sequence includes a T1 period, a T2 period, a T3 period, a T4 period, a T5 period, a T6 period, a T7 period, and a T8 period. A first light beam modulated by a first modulation signal can be emitted during the T1 period, the T2 period, the T3 period, the T4 period, the T5 period, the T6 period, the T7 period, and the T8 period.

[0087] Combined with the above Figure 4 If the i-TOF image sensor is working in the detection device during the T1, T3, T5 and T7 periods, the first modulation signal in the T1, T3, T5 and T7 periods may be any waveform of a sine wave, a square wave, a pulse wave or a continuous wave; if the d-TOF image sensor is working in the detection device during the T2, T4, T6 and T8 periods, the first modulation signal in the T2, T4, T6 and T8 periods is a pulse wave.

[0088] In another possible implementation, the light source may emit the first light beam according to the received second modulation signal, where the second modulation signal includes a pulse wave. Figure 4 The second modulation signals in the T1 period, T2 period, T3 period, T4 period, T5 period, T6 period, T7 period, and T8 period are all pulse waves. It can also be understood that the light source can continuously emit pulse waves according to the received second modulation signal.

[0089] It should be noted that each time period may include multiple modulation signals. For example, the T1 time period may include multiple pulse waves, and the T2 time period may also include multiple pulse waves.

[0090] 2. Spectral Components

[0091] As follows, different scenarios are introduced based on whether the light splitting component splits light in space or in time.

[0092] In the first scenario, the light splitting component performs light splitting in time (time division method).

[0093] Based on this scenario, the optical splitter component can transmit the first echo optical signal to the first detection component at the second timing sequence, or transmit the second echo optical signal to the second detection component at the third timing sequence, based on the received second control signal. The second control signal can be transmitted from the processing and control component to the optical splitter component. For details, refer to the description of the processing and control component below and are not repeated here. Furthermore, optionally, the second timing sequence and the third timing sequence are arranged alternately.

[0094] Combined with the above Figure 4 As shown, Figure 4 (a) in the equation can represent the second time series, Figure 4 (b) in the figure may represent the third time sequence. Specifically, during time periods T1, T3, T5, and T7, the optical splitter component transmits the received first echo optical signal to the first detection component. That is, during time periods T1, T3, T5, and T7, the detection device operates in the first detection component mode. During time periods T2, T4, T6, and T8, the optical splitter component transmits the received second echo optical signal to the second detection component. That is, during time periods T2, T4, T6, and T8, the detection device operates in the second detection component mode.

[0095] It should be noted that the timing of the optical splitting components can be controlled according to actual needs. Figure 4 This is just an example. In addition, the timing of the spectroscopic component is synchronized with the first timing of the light source emitting the first light beam. Specifically, in the T1 period, the light source starts to emit the first light beam (i.e., the rising edge of the light source signal), and the spectroscopic component synchronously propagates the received first echo light signal to the first detection component; in the T2 period, the light source starts to emit the first light beam again (i.e., the rising edge of the light source signal), and the spectroscopic component synchronously propagates the received second echo light signal to the second detection component; in the T3 period, the light source starts to emit the first light beam again (i.e., the rising edge of the light source signal), and the spectroscopic component synchronously propagates the received first echo light signal to the first detection component; and so on. It should be understood that the synchronization between the light source and the spectroscopic component is usually a synchronization with a period of the order of tens of milliseconds (ms) (depending on the frame rate of the detection device).

[0096] It should also be noted that the T1 period is the second time sequence, the T2 period is the third time sequence, and the arrangement of the T1 period and the T2 period is alternating, and so on. In addition, the T1 period and the T2 period are a detection cycle of the detection device, and the distance information detected in the T1 period and the T2 period can be fused to form a depth image of the target.

[0097] As shown below, four possible structures of a light splitting component for performing light splitting in time are exemplified. It can also be understood that the structures of the four light splitting components shown below can control light switching in a timing manner.

[0098] Structure 1, DMD.

[0099] For example, when the DMD is at an angle α, the DMD may propagate a first echo optical signal to the first detection component; and when the DMD is at an angle β, the DMD may propagate a second echo optical signal to the second detection component.

[0100] In one possible implementation, based on the received second control signal, the DMD can be controlled to be at an angle α at the second timing sequence and at an angle β at the third timing sequence, so as to propagate the first echo optical signal to the first detection component at the second timing sequence and the second echo optical signal to the second detection component at the third timing sequence. It can also be understood that by controlling the angle of the DMD, light splitting can be achieved in time.

[0101] Combined with the above Figure 4 , during the T1, T3, T5, and T7 periods, the DMD is at an angle α; during the T2, T4, T6, and T8 periods, the DMD is at an angle β.

[0102] Structure 2, LCOS.

[0103] See Figure 5a , a schematic diagram of the structure of an LCoS provided in this application. This LCoS can be a phase-type LCoS. A voltage or current signal applied to the LCoS changes the orientation of the long axis of the liquid crystal molecules, thereby changing the refractive index of the LCoS. This can alter the phase of light passing through the LCoS, and thus the angle at which light exits the LCoS. This is equivalent to rotating the light exit angle using phase delay, thereby achieving temporal light splitting.

[0104] In one possible implementation, a first phase diagram may be applied to the LCoS at a second timing based on a received second control signal. The first phase diagram may control the voltage of each pixel in the LCoS at the second timing, so that the LCoS transmits a first echo light signal to the first detection component at the second timing. A second phase diagram may be applied to the LCoS at a third timing. The second phase diagram may control the voltage of each pixel in the LCoS at the third timing, so that the LCoS transmits a second echo light signal to the second detection component at the third timing.

[0105] Combined with the above Figure 4 , during the T1 period, the T3 period, the T5 period, and the T7 period, the first phase diagram is applied to the LCoS; during the T2 period, the T4 period, the T6 period, and the T8 period, the second phase diagram is applied to the LCoS.

[0106] Structure three, optical switch.

[0107] An optical switch is an optical circuit conversion device with one or more selectable transmission ports. Its function is to physically switch or perform logical operations on optical signals in an optical transmission line or integrated optical circuit. Optical switches can be traditional mechanical optical switches, micromechanical optical switches, thermo-optic switches, liquid crystal optical switches, electro-optic switches, and acousto-optic switches.

[0108] See also Figure 5b , is a structural schematic diagram of an optical switch provided in the present application. The optical switch can be in the form of 1×2, that is, 1 input end and 2 output ends. One input end of the off switch is used to receive the first echo optical signal and the second echo optical signal from the detection area, the output end 1 of the off switch is connected to the first detection component, and the output end 2 of the off switch is connected to the second detection component. When the output end 1 is connected to the input end, the link of the output end 1 is in the on state. At this time, the input end receives the first echo optical signal from the detection area and transmits the first echo optical signal to the first detection component. When the output end 2 is connected to the input end, the link of the output end 2 is in the on state. At this time, the input end receives the second echo optical signal from the detection area and transmits the second echo optical signal to the second detection component.

[0109] In a possible implementation, according to the second control signal, the output terminal 1 is connected to the input terminal at the second timing control, and the output terminal 2 is connected to the input terminal at the third timing control.

[0110] Combined with the above Figure 4 During the T1 period, T3 period, T5 period, and T7 period, the control output terminal 1 is connected to the input terminal; during the T2 period, T4 period, T6 period, and T8 period, the control output terminal 2 is connected to the input terminal.

[0111] See also Figure 5c, is a structural schematic diagram of another optical switch combination provided by the present application. The optical switch can be in the form of 2×2, that is, 2 input ports and 2 output ports. Input port 1 is used to connect to output port 1, and input port 2 is used to connect to output port 2. When input port 1 is connected to output port 1, the link of input port 1 is in the connected state. At this time, input port 1 receives the first echo optical signal from the detection area and transmits the first echo optical signal to the first detection component. When input port 2 is connected to the output port, the link of input port 2 is in the connected state. At this time, input port 2 receives the second echo optical signal from the detection area and transmits the second echo optical signal to the second detection component.

[0112] In a possible implementation, according to the second control signal, the output terminal 1 is connected to the input terminal 1 at the second timing sequence, and the output terminal 2 is connected to the input terminal 2 at the third timing sequence.

[0113] Combined with the above Figure 4 During the T1 period, T3 period, T5 period, and T7 period, the control output terminal 1 is connected to the input terminal 1; during the T2 period, T4 period, T6 period, and T8 period, the control output terminal 2 is connected to the input terminal 2.

[0114] Structure four, fiber optic circulator.

[0115] Fiber circulator is a multi-port non-reciprocal optical device, in which optical signals can only propagate in one direction. Figure 5d The figure shows the structure of a fiber circulator provided by this application. If an optical signal is input from port 1, it will be output from port 2; if an optical signal is input from port 2, it will be output from port 3, and the output loss is very small. If an optical signal is input from port 2, the loss is very large when it is output from port 1. Similarly, if the light is input from port 3, the loss is very large when it is output from either port 1 or port 2.

[0116] In a possible implementation, according to the second control signal, the first echo optical signal may be controlled to be input from port 1 at the second timing, and the second echo optical signal may be controlled to be input from port 3 at the third timing.

[0117] It should be understood that the above-mentioned Structure 1, Structure 2, Structure 3 and Structure 4 are all examples and this application does not limit them.

[0118] In the second scenario, the light splitting component performs light splitting in space (space separation method).

[0119] Based on this second scenario, the optical splitter component can split the echo light signal from the detection area into two, obtaining a first echo light signal and a second echo light signal, and transmit the first echo light signal to the first detection component and the second echo light signal to the second detection component. For example, the optical splitter component can split the echo light signal from the detection area into the first echo light signal and the second echo light signal according to an intensity ratio; wherein the intensity ratio can be 1:1, i.e., the intensity of the first echo light signal is the same as the intensity of the second echo light signal; or the intensity ratio can be other possible ratios, for example, the intensity of the first echo light signal is greater than the intensity of the second echo light signal, or the intensity of the first echo light signal is less than the intensity of the second echo light signal.

[0120] Two possible structures of a light splitting component for performing spatial light splitting are shown below as examples.

[0121] Structure A, beam splitter.

[0122] In one possible implementation, the beam splitter can be, for example, a beam splitter (BS) or a beam splitter plate. The beam splitter is formed by coating one or more thin films (i.e., beam splitter films) on the surface of the prism, and the beam splitter plate is formed by coating one or more thin films (i.e., beam splitter films) on one surface of a glass plate. Both the beam splitter prism and the beam splitter plate utilize the different transmittance and reflectivity of the thin film to the incident light to split the echo light signal from the detection area into two, thereby obtaining a first echo light signal and a second echo light signal. For example, the beam splitter is a polarizing beam splitter. The polarizing beam splitter can be composed of two polarizing beam splitters (PBS), and the inclined surfaces of the two PBSs are bonded together by an adhesive layer (see Figure 6 PBS is achieved by coating one or more thin films on the inclined surface of a right-angle prism, and then fusing them together to form a cubic structure through an adhesive layer. When a light beam is incident at the Brewster angle, the transmittance of P-polarized light is 1, while the transmittance of S-polarized light is less than 1. After the light beam passes through the multi-film structure multiple times at the Brewster angle, the P-polarized light is completely transmitted, while the vast majority of the S-polarized light is reflected (at least 90%).

[0123] For example, the polarization beam splitter can separate the incident echo light signal (P-polarized light and S-polarized light) from the detection area into horizontally polarized light (i.e., S-polarized light) and vertically polarized light (i.e., P-polarized light), namely the first echo light signal and the second echo light signal. The P-polarized light passes completely, while the S-polarized light is reflected at a 45-degree angle, and the outgoing direction of the S-polarized light forms a 90-degree angle with the outgoing direction of the P-polarized light.

[0124] It should be noted that Figure 6The example in which the first optical echo signal is P-polarized light and the second optical echo signal is S-polarized light is used for illustration. It should be understood that the first optical echo signal may also be S-polarized light, and correspondingly, the second optical echo signal may be P-polarized light.

[0125] Structure B, diffractive optical elements (DOE).

[0126] In one possible implementation, the diffraction optical device can split the echo light signal from the detection area into two to obtain a first echo light signal and a second echo light signal, and propagate the first echo light signal to the first detection component and the second echo light signal to the second detection component.

[0127] It should be understood that the above-mentioned structure A and structure B are examples, and this application does not limit them.

[0128] It should be noted that if the spectroscopic component is based on the structure of the above-mentioned scenario one, the waveform of the first light beam emitted by the emitting component is related to whether the detection device operates in the d-TOF image sensor mode or the i-TOF image sensor mode. If the detection device operates in the i-TOF image sensor mode, the waveform of the first light beam can be any one of a sine wave, a square wave, a pulse wave, or a continuous wave; if the detection device operates in the d-TOF image sensor mode, the waveform of the first light beam can be a pulse wave. If the spectroscopic component is based on the structure of the above-mentioned scenario two, regardless of whether the detection device operates in the d-TOF image sensor mode or the i-TOF image sensor mode, the waveform of the first light beam emitted by the emitting component is a pulse wave.

[0129] 3. First Detection Component

[0130] In a possible implementation, the first detection component may be an i-TOF image sensor. The i-TOF image sensor may be used to perform photoelectric conversion on the received first echo light signal to obtain a first electrical signal. Figure 7a FIG2 is a schematic diagram of the structure of an i-TOF image sensor provided by the present application. The i-TOF image sensor may include a CAPD array, which may include a photon detector (PD), a capacitor A, and a capacitor B. The PD may perform photoelectric conversion on a received first echo light signal, calculate and analyze the phase delay between the echo light signal and the signal light through the charge difference between capacitors A and B, and thus calculate the first distance information of the target.

[0131] Furthermore, optionally, the d-TOF image sensor may further include a memory and a control circuit. The control circuit may store the time of flight detected by CAPD in the memory.

[0132] It should be noted that the i-TOF image sensor is suitable for close-range detection, that is, the detection accuracy of the i-TOF image sensor is higher when the detection distance is less than a preset value.

[0133] 4. Second Detection Component

[0134] In a possible implementation, the second detection component may be a d-TOF image sensor, which may be configured to perform light point conversion on the received second echo light signal to obtain a second electrical signal.

[0135] like Figure 7b As shown in the figure, it is a schematic diagram of the structure of a d-TOF image sensor provided by the present application. The d-TOF image sensor may include a SPAD array and a time to digital converter (TDC) array. In this example, a 5×5 SPAD array and a 5×5 TDC array are taken as an example. When a certain TDC detects the first pulse signal, timing starts. After one of the at least one SPAD corresponding to the TDC that starts timing receives an echo light signal, the TDC stops timing. In this way, the time difference between the emission and reception of the light signal, that is, the flight time of the light signal, can be detected.

[0136] Furthermore, optionally, the d-TOF image sensor may further include a memory and a control circuit. The control circuit may store the time of flight detected by the SPAD / TDC in the memory.

[0137] It's important to note that the depth measurement accuracy of d-TOF technology is independent of detection distance and relies primarily on the time-to-digital converter (TDC) in the d-TOF sensor module. d-TOF is suitable for long-distance detection, and its accuracy is high when the detection distance is at least a preset value.

[0138] In one possible implementation, the resolution range of the d-TOF image sensor may be [8 megapixels, 48 ​​megapixels], and the resolution range of the i-TOF image sensor may also be [8 megapixels, 48 ​​megapixels]. For example, the resolution of the d-TOF image sensor may be 8 megapixels, 12 megapixels, 20 megapixels, or 48 megapixels; the resolution of the i-TOF image sensor may be 8 megapixels, 12 megapixels, 20 megapixels, or 48 megapixels. It should be understood that the resolution of the d-TOF image sensor may also be greater than 48 megapixels, for example, 52 megapixels, 60 megapixels, 72 megapixels, etc.; the resolution of the i-TOF image sensor may also be greater than 48 megapixels, for example, 52 megapixels, 60 megapixels, 72 megapixels, etc.

[0139] In a possible implementation, the detection device may further include a processing control component, which is described in detail below.

[0140] 5. Processing Control Components

[0141] In one possible implementation, the processing control component may be connected to the transmitting component and the receiving component respectively to control the transmitting component and the receiving component, which are described below.

[0142] In one possible implementation, the processing and control component may generate a first control signal and transmit the first control signal to the transmitting component (e.g., a light source in the transmitting component) to control a first timing of the light source emitting a first light beam. Furthermore, optionally, the processing and control component may generate a first modulation signal and transmit the first modulation signal to the light source to control a waveform of the first light beam emitted by the light source. The waveform may be, for example, a pulse wave or a continuous wave.

[0143] In another possible implementation, in combination with situation one in the above-mentioned spectroscopic component, the processing control component can also generate a second control signal and send the second control signal to the spectroscopic component to enable the spectroscopic component to transmit the first echo optical signal to the first detection component in the second timing, or to transmit the second echo optical signal to the second detection component in the third timing.

[0144] In a possible implementation, the second time sequence and the third time sequence are arranged alternately, which means that the first detection component and the second detection component work alternately. Figure 4 , the first detection component works during the T1 period, the second detection component works during the T2 period, the first detection component works during the T3 period, the second detection component works during the T4 period, and so on.

[0145] Further, optionally, the processing control component may receive a first electrical signal from the i-TOF image sensor, determine a phase difference between the emission of the first light beam and the reception of the first echo light signal by the i-TOF image sensor based on the first electrical signal, and calculate a first time difference (i.e., a flight time of the first light beam) between the emission of the first light beam and the reception of the first echo light signal by the i-TOF image sensor based on the phase difference, and determine the first distance information of the target based on the first time difference. For the specific process, please refer to the aforementioned Figure 1b The principle of .

[0146] Accordingly, the processing control component may receive a second electrical signal from the d-TOF image sensor, determine a second time difference between the emission of the first light beam and the receipt of the second echo light signal by the d-TOF image sensor based on the second electrical signal, and determine the second distance information of the target based on the second time difference. For details, please refer to the aforementioned Figure 1a Introduction.

[0147] It should be noted that the processing control component also needs to control the first timing of the light source emitting the first light beam and the TDC synchronization in the d-TOF image sensor. For example, the processing control component controls the light source to emit the first pulse of light and controls the TDC to start timing at the same time. After the SPAD receives the second echo light signal, the TDC stops timing and waits; when the processing control component controls the light source to emit the second pulse of light and controls the TDC to start timing for the second time, similarly, after the SPAD receives the second echo light signal, the TDC stops timing for the second time and waits, and so on. It should be understood that the range of the TDC is smaller than the period of the pulse light. Typically, the synchronization of the TDC in the d-TOF image sensor with the light source emitting the first light beam is based on a period of hundreds of nanoseconds (ns). It should be understood that the synchronization period of the TDC and the light source emitting the first light beam is related to the detection distance.

[0148] In a possible implementation, the processing control component may generate distance information of the target according to the first distance information and the second distance information.

[0149] Since the i-TOF image sensor will produce multipath interference areas or intensity error areas, the distance information detected in some areas will be incorrect, seriously affecting the image quality. Figure 8 The elliptical area within the range of Z0 to Z1 in the figure represents the area where multipath interference or intensity error occurs. In order to ensure high-quality imaging within the full detection distance range, the distance information corresponding to the elliptical area can be eliminated and replaced (or supplemented) with the distance information corresponding to the elliptical area in the d-TOF image sensor.

[0150] In a possible implementation, the processing control component may剔除 the error distance information corresponding to the multipath interference area or intensity error area in the first distance information, obtain the target distance information corresponding to the error distance information from the second distance information, and supplement the error distance information with the target distance information. That is to say, the processing control component may determine the error distance information corresponding to the multipath interference area or intensity error area in the first distance information, determine the target distance information corresponding to the error distance information in the second distance information, and replace the error distance information with the target distance information.

[0151] Further, optionally, the processing control component may generate a first image based on the first distance information, generate a second image based on the second distance information, and fuse the first image and the second image to obtain the depth image of the target. Optionally, the first distance information for generating the first image here may be the first distance information after replacing the error distance information with the target distance information as described above. In this way, it helps to improve the accuracy of the distance information of the target detected by the detection device, and thus can improve the quality of the depth image of the target formed.

[0152] In a possible implementation, the processing control component may generate the depth image of the target based on the first electrical signal from the first detection component and the second electrical signal from the second detection component. For the convenience of explaining the solution, as follows, the high-precision measurement distance range applicable to the i-TOF image sensor is from Z0 to Z1, and the high-precision measurement distance range applicable to the d-TOF image sensor is from ZI to Z2, where Z0 < Z1 < Z2, and reference can be made to Figure 8 .

[0153] The following exemplarily shows two possible implementation manners of image fusion to achieve high-precision detection within the full detection distance range.

[0154] Implementation manner 1,剔除 the inaccurate distance information.

[0155] In a possible implementation, the processing control component may剔除 the distance information in the first distance information that is greater than the preset value to obtain the third distance information, and generate a first image based on the third distance information;剔除 the distance information in the second distance information that is not greater than the preset value to obtain the fourth distance information, and generate a second image based on the fourth distance information; fuse the first image and the second image to obtain the depth image of the target.

[0156] Combined with the above Figure 8 It should be noted that the Chinese character "剔除" in the original text is not a standard English word. Here, a more appropriate English expression can be used according to the context, such as "exclude" or "remove". The above translation uses "剔除" for the sake of consistency with the original text. You can adjust it according to the actual situation.The processing and control component can calculate the distance information within the full detection range, i.e., the distance information within the range of Z0 to Z2, based on the first electrical signal from the i-TOF image sensor; eliminate the distance information with a distance greater than a preset value from the distance information within the full detection range, i.e., eliminate the distance information within the range of Z1 to Z2, to obtain third distance information, i.e., obtain the distance information within the range of Z0 to Z1. The processing and control component can calculate the distance information within the full detection range, i.e., the distance information within the range of Z0 to Z2, based on the second electrical signal from the d-TOF image sensor; eliminate the distance information not greater than the preset value from the distance information within the full detection range, i.e., eliminate the distance information within the range of Z0 to Z1, to obtain fourth distance information, i.e., the distance information within the range of Z1 to Z2. The processing and control component can generate a first image based on the distance information within the range of Z0 to Z1, generate a second image based on the distance information within the range of Z1 to Z2, and fuse the first image and the second image to obtain a depth image of the target.

[0157] Since the i-TOF image sensor has high accuracy at close range and its accuracy decreases linearly with increasing distance, the d-TOF image sensor maintains the same accuracy within the measurement range, so the accuracy can be maintained at a high level at long distances. However, the accuracy for nearby targets often cannot meet the requirements. By using the third distance information of the i-TOF image sensor at close range to generate a first image, and using the fourth distance information of the d-TOF image sensor at long range to generate a second image, the first and second images are then fused to obtain a depth image of the target, thereby obtaining high-precision imaging over the entire detection range.

[0158] Implementation method 2: weighted average based on confidence.

[0159] In one possible implementation, the confidence level can be designed based on the relative detection accuracy of the i-TOF image sensor and the d-TOF image sensor. For the i-TOF image sensor, the confidence level is lowered as the distance value increases; for the d-TOF image sensor, the confidence level is higher as the distance value increases. Alternatively, a relationship between the first distance information and the confidence level can be configured. For example, the greater the distance corresponding to the first distance information, the lower the confidence level; and the greater the distance corresponding to the second distance information, the higher the confidence level.

[0160] Example 1: Multiply the distance information in the first distance information whose distance is greater than a preset value by the first confidence level, and multiply the distance information in the first distance information whose distance is not greater than the preset value by the second confidence level to obtain fifth distance information, and generate a third image based on the fifth distance information, wherein the second confidence level is greater than the first confidence level; multiply the distance information in the second distance information whose distance is not greater than the preset value by the third confidence level, and multiply the distance information in the second distance information whose distance is greater than the preset value by the fourth confidence level to obtain sixth distance information, and generate a fourth image based on the sixth distance information, wherein the fourth confidence level is greater than the third confidence level; and fuse the third image and the fourth image to obtain a depth image of the target.

[0161] Combined with the above Figure 8 The processing and control component can calculate the distance information within the full detection distance range, that is, the distance information within the range of Z0 to Z2, based on the first electrical signal from the i-TOF image sensor; multiply the distance information in the full detection distance range that is greater than a preset value (Z1) (that is, the distance information in the range of Z1 to Z2) by the first confidence level, and multiply the distance information in the full detection distance range that is not greater than the preset value (that is, the distance information in the range of Z0 to Z1) by the second confidence level to obtain fifth distance information, and generate a third image based on the fifth distance information, wherein the second confidence level is greater than the first confidence level. The processing and control component can calculate the distance information within the full detection distance range (i.e., the distance information within the range of Z0 to Z2) based on the second electrical signal from the d-TOF image sensor; multiply the distance information within the full detection distance range whose distance is not greater than a preset value (Z1) (i.e., the distance information within the range of Z0 to Z1) by a third confidence level, and multiply the distance information within the full detection distance range whose distance is greater than the preset value (i.e., the distance information within the range of Z1 to Z2) by a fourth confidence level to obtain sixth distance information, and generate a fourth image based on the sixth distance information, where the fourth confidence level is greater than the third confidence level; and fuse the third image and the fourth image to obtain a depth image of the target.

[0162] By dividing the first distance information into two parts, multiplying the portion not greater than the preset value by the larger second confidence level, and the portion greater than the preset value by the smaller first confidence level, that is, the portion of the first distance information not greater than the preset value has a larger proportion, and the portion greater than the preset value has a smaller proportion; similarly, dividing the second distance information into two parts, multiplying the portion not greater than the preset value by the smaller third confidence level, and the portion greater than the preset value by the larger fourth confidence level. The depth image of the target obtained based on the fifth and sixth distance information has high accuracy across the entire detection range.

[0163] In Example 2, the first distance information is multiplied by the fifth confidence level to obtain seventh distance information, and a fifth image is generated based on the seventh distance information. The fifth confidence level is negatively correlated with the first distance information, i.e., the fifth confidence level decreases as the first distance increases. The second distance information is multiplied by the sixth confidence level to obtain eighth distance information, and a sixth image is generated based on the eighth distance information. The sixth confidence level is positively correlated with the second distance information, i.e., the sixth confidence level increases as the second distance increases. The fifth and sixth images are fused to obtain a depth image of the target.

[0164] It should be noted that the specific functional relationship between the fifth confidence level and the first distance can be linear or nonlinear, and the specific functional relationship between the sixth confidence level and the second distance can be linear or nonlinear, and this application does not limit this.

[0165] Combined with the above Figure 8 When both the first and second distance information are Z1, the fifth and sixth confidence levels are the same. For example, when the first distance information is Z0, the fifth confidence level is 1; when the first distance information is Z1, the fifth confidence level is 0.5. This means that the fifth confidence level decreases as the first distance increases. When the second distance information is Z1, the sixth confidence level is 0.5; when the second distance information is Z2, the sixth confidence level is 1. This means that the sixth confidence level increases as the second distance increases.

[0166] It should be noted that in the above two image fusion implementation methods, the error distance information corresponding to the elliptical area within the range of Z0 to Z1 in the first distance information can be replaced by the target distance information corresponding to the error distance in the second distance information.

[0167] In one possible implementation, the first processing component can be a processing control component such as a processor, a microprocessor, or a processor, for example, a general-purpose central processing unit (CPU), a general-purpose processor, a digital signal processing (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a transistor logic device, a hardware component or any combination thereof.

[0168] It should be noted that if the detection device does not include the aforementioned processing and control component, the functions of the processing and control component may be performed by a processor in a terminal device in which the detection device is used. For example, when the detection device is used in a vehicle, the functions of the processing and control component may be performed by a main processor in the vehicle. For another example, when the detection device is used in a smartphone, the functions of the processing and control component may be performed by a CPU in the smartphone.

[0169] The receiving assembly in the detection device in any of the above embodiments may further include a lens assembly, which is used to converge the echo light signal received from the detection area to the d-TOF image sensor and / or the i-TOF image sensor as much as possible. Further, optionally, the receiving assembly may also include an infrared radiation (IR) filter, which may be located between the lens assembly and the spectroscopic assembly. The IR filter can be used to block or absorb infrared rays to prevent damage to the d-TOF image sensor and the i-TOF image sensor. Exemplarily, the material of the IR filter may be glass or a glass-like resin, such as blue glass.

[0170] Based on the above content, two specific examples of the above detection device are given below in combination with specific hardware structures to facilitate further understanding of the structure of the above detection device.

[0171] like Figure 9a FIG2 is a schematic diagram of the structure of another detection device provided by the present application. The detection device may include an emitting component, a spectroscopic component, a first detection component, a second detection component, and a processing and control component. The spectroscopic component is a spectroscopic component based on the above-mentioned scenario 1. For an introduction to the emitting component, the spectroscopic component based on scenario 1, the first detection component, the second detection component, and the processing and control component, please refer to the above-mentioned related descriptions and will not be repeated here.

[0172] like Figure 9b , which is a schematic diagram of the structure of another detection device provided by the present application. The detection device may include an emitting component, a spectroscopic component, a first detection component, a second detection component, and a processing and control component. Among them, the spectroscopic component is a spectroscopic component based on the above-mentioned scenario 2. For an introduction to the emitting component, the spectroscopic component based on scenario 2, the first detection component, the second detection component, and the processing and control component, please refer to the above-mentioned related descriptions and will not be repeated here.

[0173] It should be noted that the transmitting module in the detection device can reuse the transmitting module, lens assembly, etc. of the detection device in the prior art.

[0174] Based on the structure and functional principles of the detection device described above, the present application may also provide a camera, which may include the detection device in any of the above embodiments. It can also be understood that the detection device in any of the above embodiments can function as a camera alone. Furthermore, the camera may optionally generate grayscale images or may be an infrared camera.

[0175] Based on the structure and functional principles of the detection device described above, the present application may also provide a terminal device, which may include the detection device and a processor in any of the above embodiments, wherein the processor is configured to control the detection device to detect the detection area. Furthermore, the terminal device may optionally include a memory configured to store programs or instructions; the processor is configured to invoke the programs or instructions to control the detection device to detect the detection area. It is understood that the terminal device may also include other components, such as a wireless communication device, a touch screen, and a display screen.

[0176] like Figure 10 As shown in FIG, a schematic diagram of the structure of a terminal device provided by the present application. In this example, the detection device takes a camera module as an example. The terminal device 1000 may include a processor 1001, a memory 1002, a camera module 1003 and a display screen 1004. It should be understood that Figure 10 The hardware structure shown is only an example. The terminal device to which this application is applicable may have more Figure 10 The terminal devices shown in FIG. 3 may include more or fewer components, may combine two or more components, or may have a different configuration of components. Figure 10 The various components shown in the drawings may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0177] The processor 1001 may include one or more processing units. For example, the processor 1001 may include an application processor 1001 (AP), a graphics processing unit 1001 (GPU), an image signal processor 1001 (ISP), a controller, a digital signal processor 1001 (DSP), etc. The different processing units may be independent devices or integrated into one or more processors 1001.

[0178] The memory 1002 may be a random access memory (RAM), a flash memory, a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a register, a hard disk, a mobile hard disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary memory 1002 is coupled to the processor 1001 so that the processor 1001 can read information from the memory 1002 and write information to the memory 1002. Of course, the memory 1002 may also be an integral part of the processor 1001. Of course, the processor 1001 and the memory 1002 may also exist in the terminal device as discrete components.

[0179] The camera module 1003 can be used to capture dynamic and static images, etc. In some embodiments, the terminal device may include one or N camera modules 1003, where N is a positive integer. The introduction of the camera module 1003 can be found in the description of the previous embodiment, and will not be repeated here.

[0180] When the camera module 1003 is applied as a vehicle-mounted camera module, it can be divided into a driving assistance camera module, a parking assistance camera module and an in-vehicle driver monitoring camera module based on the function of the vehicle-mounted camera module. The driving assistance camera module is used for driving recording, lane departure warning, door opening warning, blind spot monitoring and traffic sign recognition, etc. The driving assistance camera module includes intelligent front vision (such as monocular / binocular / trinocular), which can be used for dynamic object detection (vehicles, pedestrians), static object detection (traffic lights, traffic signs, lane lines, etc.) and passable space division, etc.; side view assistance (such as wide angle), which is used to monitor dynamic targets in the blind spot of the rearview mirror during driving; night vision assistance (such as night vision camera), which can be used to better detect target objects at night or other conditions with poor lighting. Parking assist camera modules can be used for reversing images and 360° surround view. 360° surround view (such as wide-angle / fisheye) is primarily used for low-speed, close-range sensing, creating a seamless 360-degree bird's-eye view of the vehicle's surroundings. In-vehicle driver monitoring camera modules primarily provide single- or multi-layered warnings for dangerous situations such as driver fatigue, distraction, and irregular driving. Based on their installation location within the terminal device, in-vehicle camera modules can be categorized as front-view, side-view, rear-view, and built-in cameras.

[0181] The display screen 1004 can be used to display images, videos, etc. The display screen 1004 may include a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLed, a Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device may include one or Q display screens 1004, where Q is a positive integer greater than 1. For example, the terminal device may implement the display function through a GPU, the display screen 1004, and the processor 1001.

[0182] Exemplarily, the terminal device may be a radar (such as a lidar), a vehicle, a smart phone, a smart home device, a smart manufacturing device, a robot, a drone, or a smart transportation device (such as an AGV or an unmanned transport vehicle, etc.).

[0183] In the various embodiments of the present application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced by each other. The technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0184] In this application, "uniform" does not mean absolute uniformity, and a certain degree of error is allowed. "Vertical" does not mean absolute verticality, and a certain degree of engineering error is allowed. "At least one" means one or more, and "more" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. In the text description of this application, the character " / " generally indicates that the previous and next associated objects are in an "or" relationship. In the formulas of this application, the character " / " indicates that the previous and next associated objects are in a "division" relationship. In this application, the symbol "[a, b]" represents a closed interval, the range is greater than or equal to a and less than or equal to b; in addition, in this application, the word "exemplarily" is used to indicate an example, illustration or explanation. Any embodiment or design scheme described as an "example" in this application should not be interpreted as being more preferred or more advantageous than other embodiments or design schemes. Alternatively, it can be understood that the use of the word "example" is intended to present concepts in a specific way and does not limit the present application.

[0185] It will be appreciated that the various numerical numbers involved in this application are merely for the purpose of describing the distinctions made, and are not intended to limit the scope of the embodiments of the present application. The size of the sequence numbers of the above-mentioned processes does not imply the order of execution, and the order of execution of each process should be determined by its function and inherent logic. Terms such as "first", "second", and similar expressions are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, comprising a series of steps or units. Methods, systems, products, or devices are not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or that are inherent to these processes, methods, products, or devices.

[0186] Although the present application has been described with reference to specific features and embodiments thereof, it is apparent that various modifications and combinations thereof may be made without departing from the spirit and scope of the present application. Accordingly, this specification and the drawings are intended to be illustrative only of the solutions defined by the appended claims and are to be construed as covering any and all modifications, variations, combinations or equivalents within the scope of the present application.

[0187] Obviously, those skilled in the art may make various changes and modifications to the present application without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations.

Claims

1. A detection device, characterized in that: The system comprises a transmitting component, a receiving component and a processing and control component, wherein the receiving component comprises a light splitting component, a first detection component and a second detection component; when the detection distance is less than a preset value, the detection accuracy of the first detection component is greater than the detection accuracy of the second detection component; when the detection distance is not less than the preset value, the detection accuracy of the first detection component is not greater than the detection accuracy of the second detection component; The emitting component is used to emit a first light beam; The optical splitter component is used to change the propagation optical path of the echo optical signal for the first light beam in the detection area to obtain a first echo optical signal and a second echo optical signal, and propagate the first echo optical signal to the first detection component and the second echo optical signal to the second detection component. The optical splitter component operates in a time division mode and a space division mode. The time division mode is that the optical splitter component switches the propagation optical path of the echo optical signal of the first light beam at a specified timing according to a control signal, and the space division mode is that the optical splitter component splits the echo optical signal of the first light beam into two. The first detection component is used to detect the first echo optical signal to obtain a first electrical signal; The second detection component is used to detect the second echo light signal to obtain a second electrical signal; the first detection component includes an indirect time-of-flight i-TOF image sensor, and the second detection component includes a direct time-of-flight d-TOF image sensor; The processing and control component is configured to receive the first electrical signal from the first detection component and the second electrical signal from the second detection component; generate target distance information based on the first electrical signal and the second electrical signal; adjust, based on the preset value, the detection weights of the first detection component and the second detection component for detection distances less than the preset value and for detection distances not less than the preset value, respectively; and replace error data corresponding to the first detection component with data corresponding to the second detection component; Among them, the detection weight of the first detection component for the detection distance less than the preset value is greater than the detection weight for the detection distance not less than the preset value, and the detection weight of the second detection component for the detection distance not less than the preset value is greater than the detection weight for the detection distance less than the preset value.

2. The detection device according to claim 1, wherein The light splitting component is used for: According to the received second control signal, the first echo optical signal is propagated to the first detection component in a second timing sequence, or the second echo optical signal is propagated to the second detection component in a third timing sequence.

3. The detection device according to claim 2, characterized in that The second timing sequence and the third timing sequence are arranged alternately.

4. The detection device according to claim 2, wherein: The optical splitting component includes any one of the following: Liquid crystal on silicon (LCOS), optical switches, fiber optic circulators or digital micromirror devices (DMDs).

5. The detection device according to any one of claims 1 to 4, characterized in that: The transmitting assembly is used to: The first light beam is emitted according to a received first control signal, where the first control signal is used to control a first timing of the emission component to emit the first light beam.

6. The detection device according to any one of claims 1 to 4, characterized in that: The transmitting assembly is used to: The first light beam is emitted according to the received first modulation signal, where the first modulation signal includes any one of the following: Pulse wave or continuous wave.

7. The detection device according to any one of claims 1 to 4, characterized in that: The optical splitting component includes any one of the following: Beam splitters or diffractive optics.

8. The detection device according to claim 7, characterized in that The transmitting assembly is used to: The first light beam is emitted according to a received second modulation signal, where the second modulation signal includes a pulse wave.

9. The detection device according to any one of claims 1 to 4, characterized in that: The processing control component is used to: determining a phase difference between transmitting the first light beam and receiving the first echo light signal according to the first electrical signal, and determining a first time difference between transmitting the first light beam and receiving the first echo light signal according to the phase difference; determining a second time difference between transmitting the first light beam and receiving the second echo light signal according to the second electrical signal; The first distance information of the target is determined according to the first time difference; and the second distance information of the target is determined according to the second time difference.

10. The detection device according to claim 9, characterized in that When the processing control component replaces the error data corresponding to the first detection component with the data corresponding to the second detection component, the processing control component is specifically configured to: Determining error distance information corresponding to the multipath interference area or the intensity error area in the first distance information; determining target distance information corresponding to the error distance information in the second distance information; The error distance information is replaced with the target distance information.

11. The detection device according to claim 9, characterized in that The processing control component is further configured to: Eliminating distance information whose distance is greater than the preset value from the first distance information to obtain third distance information, and generating a first image according to the third distance information; Eliminating distance information whose distance is not greater than the preset value from the second distance information to obtain fourth distance information, and generating a second image according to the fourth distance information; The first image and the second image are fused to obtain a depth image of the target.

12. The detection device according to claim 9, wherein: The processing control component is further configured to: multiplying distance information in the first distance information whose distance is greater than the preset value by a first confidence level, and multiplying distance information in the first distance information whose distance is not greater than the preset value by a second confidence level, to obtain fifth distance information, and generating a third image based on the fifth distance information, wherein the second confidence level is greater than the first confidence level; multiplying distance information in the second distance information where the distance is not greater than the preset value by a third confidence level, and multiplying distance information in the second distance information where the distance is greater than the preset value by a fourth confidence level to obtain sixth distance information, and generating a fourth image based on the sixth distance information, where the fourth confidence level is greater than the third confidence level; The third image and the fourth image are fused to obtain a depth image of the target.

13. The detection device according to claim 9, wherein: The processing control component is further configured to: multiplying the first distance information by a fifth confidence level to obtain seventh distance information, and generating a fifth image based on the seventh distance information, wherein the fifth confidence level is negatively correlated with the first distance information; multiplying the second distance information by a sixth confidence level to obtain eighth distance information, and generating a sixth image based on the eighth distance information, wherein the sixth confidence level is positively correlated with the second distance information; The fifth image and the sixth image are fused to obtain a depth image of the target.

14. A terminal device, characterized in that: It comprises a processor and the detection device according to any one of claims 1 to 13, wherein the processor is used to control the detection device to detect the detection area.

15. The terminal device according to claim 14, wherein: The terminal device includes any one of the following: Radar, smartphones, vehicles, smart home devices, smart manufacturing equipment, robots, drones, or smart transportation equipment.

Images