Three-dimensional sensing device, detection method and mobile equipment

Abstract

The invention is suitable for the technical field of sensors, and provides a three-dimensional sensing device, a detection method and mobile equipment. The sensor comprises an emission assembly used for emitting detection light; the receiving assembly is used for receiving the first echo light and the second echo light; the control assembly is used for obtaining detection information of the target detection object according to the first echo light and the second echo light; the first echo light is an electromagnetic wave obtained after the detection light is reflected by the target detection object; the second echo light is an electromagnetic wave obtained after diffuse reflection of the ambient light on the target detection object; the detection information comprises a three-dimensional position of the target detection object and image information corresponding to the three-dimensional position. According to the sensor, the position information and the image information of the target detection object with correlation can be obtained through calculation, better sensing precision can be achieved, and the problems that time consumption is long and reliability cannot be expected due to complex processing of multi-sensor fusion are solved.

Description

Technical Field

[0001] The present application belongs to the field of sensor technology, and in particular, relates to a three-dimensional sensing device, a detection method and a movable device. Background Art

[0002] Unmanned vehicles or robots (i.e., movable devices) need to obtain relatively accurate three-dimensional perception results of the surrounding environment, including the characteristics (e.g., object classification, people, vehicles, roads, etc.) and position of the target detection objects. Based on these three-dimensional perception results, the unmanned vehicles or robots can automatically generate control decisions.

[0003] It can be seen that three-dimensional perception is of great significance for unmanned vehicles or robots. However, existing perception solutions have some limitations, resulting in defects in their perception accuracy or perception speed, such as:

[0004] In pure vision-based perception solutions, the type of obstacle is usually determined based on machine learning methods, and its distance is estimated based on its size. This method is similar to human judgment, but there is a certain probability of misjudgment when it is implemented based on visual sensors, which may lead to serious consequences.

[0005] In the solution based on multiple sensor fusion, the visual sensor can be mainly responsible for providing semantic information to determine the attributes of the object and the approximate distance. The radar provides accurate distance measurement or serves as redundancy to prevent the camera from making misjudgments. The limitations of this solution are: first, the entire system is relatively complex and costly; second, the system relies on information fusion between multiple sensors, which is complex to process (reliability cannot be predicted, and it is time-consuming and cannot adapt to high-speed scenarios).

[0006] Therefore, how to provide a three-dimensional sensing device, detection method and movable device with satisfactory sensing accuracy and sensing speed has become a technical problem that urgently needs to be solved in the industry. Summary of the invention

[0007] The embodiments of the present application provide a three-dimensional sensing device, a detection method and a movable device, which can solve the problem of defects in sensing accuracy or sensing speed in the prior art.

[0008] In a first aspect, an embodiment of the present application provides a three-dimensional sensing device, including:

[0009] An emitting component, the emitting component is used to emit detection light;

[0010] A receiving component, the receiving component is used to receive the first echo light and the second echo light; the receiving component includes a plurality of regularly arranged sensors;

[0011] as well as,

[0012] A control component, wherein the control component is used to obtain detection information of a target detection object according to the first echo light and the second echo light;

[0013] Among them, the first echo light refers to the electromagnetic wave after the detection light is reflected by the target detection object; the second echo light refers to the electromagnetic wave after the ambient light is diffusely reflected on the target detection object; and the detection information includes the position information of the target detection object and the image information of the target detection object.

[0014] The above-mentioned three-dimensional sensing device receives the first echo light and the third light beam, calculates the correlated target detection object position information and image information, and thus obtains the perception results with position information (distance information) and image information through a single sensor, which provides a good foundation for the sensing of target detection objects, can achieve better perception accuracy, and reduces the problems of long time and unpredictable reliability caused by the complex processing of multi-sensor fusion.

[0015] In a possible implementation manner of the first aspect, the receiving component includes a first sensor group and a second sensor group, and a first detection area corresponding to the first sensor group at least partially overlaps with a second detection area corresponding to the second sensor group;

[0016] The first sensor group is used to receive the first echo light and obtain the distance information of the object according to the first echo light;

[0017] The second sensor group is used to receive the second echo light and obtain image information of the object according to the second echo light.

[0018] The above three-dimensional sensing device

[0019] In a possible implementation manner of the first aspect, the receiving component includes a three-stage filter, which is arranged before the second sensor group, and obtains image information of the target detection object by filtering the second echo light.

[0020] The above-mentioned three-dimensional sensing device realizes the separation of the first echo light and the second echo light through a filter element, thereby avoiding interference between the mixed received first echo light and the second echo light during the operation process of the control component, thereby providing a good data basis for better perception accuracy.

[0021] In a possible implementation manner of the first aspect, the first sensor group and the second sensor group are not coplanar;

[0022] The receiving assembly includes a beam splitter;

[0023] The beam splitter is disposed between the first sensor group and the second sensor group;

[0024] The beam splitter is used for:

[0025] transmitting the first echo light so that the first echo light is received by the first sensor group; reflecting the second echo light so that the second echo light is received by the second sensor group;

[0026] or,

[0027] The first echo light is reflected so that the first echo light is received by the first sensor group; and the second echo light is transmitted so that the second echo light is received by the second sensor group.

[0028] The three-dimensional sensing device achieves separation of the first echo light and the second echo light through a beam splitter, thereby avoiding interference between the mixed received first echo light and the second echo light during the operation of the control component, thereby providing a good data basis for better sensing accuracy.

[0029] In a possible implementation manner of the first aspect, the first sensor group and the second sensor group are located in the same plane.

[0030] In a possible implementation manner of the first aspect, a wavelength of the detection light is not equal to a wavelength of the second echo light beam.

[0031] In a possible implementation manner of the first aspect, the first sensor is a sensor corresponding to the wavelength of the first outgoing light beam, and the second sensor is a CMOS image sensor.

[0032] In a possible implementation manner of the first aspect, the three-dimensional sensing device further includes a scanning component, and the scanning component is used to:

[0033] The emission angle of the detection light is adjusted within a preset sensing angle range.

[0034] In a second aspect, an embodiment of the present application provides a detection method, including:

[0035] The transmitting component transmits the detection light;

[0036] The receiving component receives the first echo light and the second echo light; the receiving component includes a plurality of regularly arranged sensors;

[0037] The control component obtains detection information of the target detection object according to the first echo light and the second echo light;

[0038] Among them, the first echo light refers to the beam of the detection light after being reflected by the target detection object; the second echo light refers to the electromagnetic wave after the ambient light is diffusely reflected on the target detection object; and the detection information includes the position information of the target detection object and the image information of the target detection object.

[0039] The above method receives the first echo light and the second echo light, calculates the correlated target detection object position information and image information, and thus obtains the perception result with position information (distance information) and image information through a single sensor, which provides a good foundation for the sensing of the target detection object, can achieve better perception accuracy, and reduces the problems of long time consumption and unpredictable reliability caused by the complex processing of multi-sensor fusion.

[0040] In a third aspect, an embodiment of the present application provides a movable device, comprising the sensing device described in the first aspect and a component to be installed.

[0041] In a fourth aspect, an embodiment of the present application provides a three-dimensional perception device control component, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the three-dimensional perception method described in the second aspect when executing the computer program.

[0042] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the three-dimensional perception method described in the second aspect is implemented.

[0043] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device executes the three-dimensional perception method described in the second aspect above.

[0044] It can be understood that the beneficial effects of the third to fifth aspects mentioned above can be found in the relevant descriptions of the first and second aspects mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 is a schematic diagram of the structure of a three-dimensional sensing device provided in an embodiment of the present application;

[0047] Figure 2 is a schematic diagram of an application scenario of a three-dimensional sensing device provided in an embodiment of the present application;

[0048] Figure 3 is a flow chart of a detection method provided in an embodiment of the present application;

[0049] Figure 4 This is a schematic diagram of a first optional implementation method of the three-dimensional sensing device provided in an embodiment of the present application;

[0050] Figure 5 is a schematic diagram of a second optional implementation method of the three-dimensional sensing device provided in an embodiment of the present application;

[0051] Figure 6 is a schematic diagram of a third optional implementation method of the three-dimensional sensing device provided in an embodiment of the present application;

[0052] Figure 7 is a schematic diagram of a fourth optional implementation method of the three-dimensional sensing device provided in an embodiment of the present application;

[0053] Figure 8 is a schematic diagram of a fifth optional implementation of the three-dimensional sensing device provided in an embodiment of the present application;

[0054] Reference numerals:

[0055] Transmitting component 10; receiving component 20; first sensor group 21; second sensor group 22; beam splitter 23; filter 24; control component 30; target detection object 40; scanning component 50; first field of view area 61; second field of view area 62; detection light 100; first echo light 200; second echo light 300; ambient light 400. DETAILED DESCRIPTION

[0056] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0057] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0058] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0059] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0060] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0061] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0062] Three-dimensional perception is of great significance for unmanned vehicles or robots. Perception operations performed only through visual sensors are limited by less input information and usually cannot obtain more accurate perception results. Therefore, in some implementations, three-dimensional perception is achieved through the fusion of multiple sensors.

[0063] For example, 3D perception in autonomous driving scenarios can be achieved through any combination of the following sensors:

[0064] Visual (camera) sensor: The camera can capture images and videos of the surroundings and provide high-resolution visual information. Through computer vision technology, key information such as road signs, traffic signals, pedestrians, and vehicles can be detected and recognized from images.

[0065] LIDAR sensor: LIDAR measures the distance and direction of surrounding objects by emitting laser beams, and can accurately draw a 3D point cloud map around the vehicle. This sensor is very effective for obstacle detection and ranging.

[0066] Radar sensor: Radar uses radio waves to detect surrounding objects, which can provide a long detection distance and stability in complex weather conditions. It is often used to assist lidar and cameras in target recognition and tracking.

[0067] In addition, you can also consider adding additional sensors like the ones described below to improve the effect of fusion perception.

[0068] Ultrasonic sensors: Ultrasonic sensors are used for close-range obstacle detection, such as avoiding collisions with surrounding objects when parking.

[0069] GPS (Global Positioning System): GPS is used for vehicle positioning and navigation. With the support of high-precision maps, it can help vehicles accurately understand their own position and driving direction.

[0070] The advantage of multi-sensor fusion is that the data from different sensors can be mutually corrected and verified, thus improving the reliability and robustness of the perception system. For example, when the lidar detects an obstacle, the camera can provide more detailed information to help distinguish whether it is a pedestrian or other vehicles.

[0071] These advantages can be fully demonstrated in a relatively ideal environment. However, for unmanned vehicles or robots in actual operation, they may work in a relatively complex environment, which brings some risks, including:

[0072] Data processing complexity:

[0073] Multi-sensor fusion requires highly complex processing and fusion of data from different sensors. Sensors may use different data formats, sampling rates, and coordinate systems. Therefore, data synchronization, calibration, and alignment are required before combining them. Data synchronization is the process of ensuring that data collected by different sensors have the same timestamp. Data calibration is the mapping of measurements from different sensors to a common reference coordinate system in order to correctly fuse the data. Data alignment is the matching of sensor data in time and space to ensure that they describe the same scene at the same moment. These steps require precise algorithms and computing resources, and the processing complexity increases with the number of sensors.

[0074] Reliability and security:

[0075] The perception scheme of multi-sensor fusion must deal with the uncertainty and failure of sensors. Since the performance of different sensors in different environments will vary, the perception system needs to be able to adapt and robustly cope with various situations. In addition, sensors may be subject to external interference, failure or damage, so the perception system needs to have a redundant design to ensure that even if one of the sensors fails, the entire system can continue to work and maintain high reliability. Fault detection and fault tolerance mechanisms are also necessary. Faults can be detected in time by monitoring abnormal conditions in sensor outputs, and appropriate measures can be taken to avoid wrong decisions. At the same time, considering that autonomous vehicles are a field involving public safety, the safety of the perception system is of vital importance, and high standards of safety specifications must be followed to ensure that the system is not maliciously attacked or manipulated.

[0076] In addition, hardware cost is also an important factor that cannot be ignored. The perception solution of multi-sensor fusion requires the use of multiple different types of sensors, such as cameras, lidar, radar, ultrasonic sensors, etc. Each sensor has its own hardware cost, and these sensors are usually highly professional and require high precision, so the cost is relatively high. At the same time, in order to ensure the reliability and redundancy of the system, it may be necessary to use multiple sensors of the same type to achieve redundancy to prevent the failure of a sensor from causing the perception system to fail. This increases the hardware cost of the entire system.

[0077] Therefore, in order to overcome the above problems and control the above risks, the present application embodiment provides a three-dimensional sensing device, such as Figure 1 As shown, including:

[0078] An emitting component, the emitting component is used to emit detection light;

[0079] A receiving component, the receiving component is used to receive the first echo light and the second echo light; the receiving component includes a plurality of regularly arranged sensors;

[0080] as well as,

[0081] A control component, wherein the control component is used to obtain detection information of a target detection object according to the first echo light and the second echo light;

[0082] Among them, the first echo light refers to the electromagnetic wave after the detection light is reflected by the target detection object; the second echo light refers to the electromagnetic wave after the ambient light is diffusely reflected on the target detection object; and the detection information includes the position information of the target detection object and the image information of the target detection object.

[0083] In this embodiment, in order to adapt to different target detection object detection needs, the emitting component can be a laser that emits highly parallel linear lasers, such as a VCSEL laser, or a component that provides a surface light source through an optical structure, such as an emitting component that emits light sufficient to cover the field of view FOV through a laser and a lens; it can also be a light source array (for emitting surface light sources, or for achieving direction / intensity / irradiation area modulation of detection light through the coherence / diffraction / optical properties of the light).

[0084] Correspondingly, the detection light can be an electromagnetic beam in the form of linear light, spherical light, plane light, etc., and in some cases, a vibration beam such as sound waves can also be added. It is worth noting that the light mentioned in the embodiments of the present application is not limited to visible light, but can also be electromagnetic waves such as infrared light, ultraviolet light, millimeter waves or alpha rays.

[0085] In this embodiment, the first echo light is a beam obtained after the detection light is reflected on the target detection object, and therefore, its form and properties are correlated with the detection light, wherein the detection light and the first echo light are usually in the same form.

[0086] Unlike the detection light which has many possible forms, the second echo light is a beam of ambient light after diffuse reflection on the target detection object. Its form is usually visible light, and in some cases it may also be infrared light and (near) ultraviolet light (formed by reflection or radiated by the target detection object).

[0087] The second echo light in the form of visible light will be able to provide image information at a specified position of the target detection object, thereby providing information similar to that of a visual sensor. In some optional embodiments, the second echo light in the form of infrared and ultraviolet light can also provide other information in addition to image information, such as temperature information at a specified position of the target detection object.

[0088] Similarly, other information including the velocity of the target detection object at a specified position can be additionally obtained through the difference in properties between the first echo light and the detection light.

[0089] In this embodiment, depending on the different types of detection light, the receiving component may have different structures. For example, for a transmitting component in the form of a light source, the receiving component may include a photon counter for receiving photons in the first echo light and photons in the second echo light, or may include an electromagnetic wave detector (for example, an electromagnetic wave detection device / circuit using a magnetic induction coil / inductor coil); for another example, for a detection light with an attached vibration wave, the receiving component may be a vibrometer.

[0090] Figure 2FIG. 1 shows an optional application scenario of the present embodiment, in which a portion of the diffusely reflected light of the ambient light on the target detection object (i.e., the second echo light) at a specified angle is received by the receiving component. Figure 2 In the figure, it is represented by a solid arrow from the target detection object to the receiving component. The reflected light of the detection light on the target detection object (i.e., the first echo light) is also received by the receiving component. This part of the light is Figure 2 The dotted arrow between the target detection object and the receiving component is represented in FIG.

[0091] It is worth noting that due to the unpredictable reflection angle of the target detection object, the first echo light and the detection light are not necessarily Figure 2 In the case of an ideal plane shown in FIG, the first echo light may be a portion of a specified angle of the diffusely reflected light of the detection light on the target detection object. This specified angle can be adjusted by setting the receiving angle of the sensor. In some cases, it may be consistent with or close to the incident angle of the detection light.

[0092] It can be understood that, in an optional embodiment, the receiving component includes a first sensor group 21 and a second sensor group 22, and a first detection area corresponding to the first sensor group 21 and a second detection area corresponding to the second sensor group 22 at least partially overlap;

[0093] The first sensor group 21 is used to receive the first echo light and obtain the distance information of the object according to the first echo light;

[0094] The second sensor group 22 is used to receive the second echo light and obtain image information of the object according to the second echo light.

[0095] Further, in a preferred embodiment, the first sensor is a sensor corresponding to the wavelength of the first outgoing light beam, and the second sensor is a CMOS image sensor.

[0096] In addition, since the acquisition of the first echo light depends on the emission of the detection light, and the acquisition of the second echo light depends on the ambient light, it can be understood that in a preferred embodiment, reference Figure 2 As shown, the number of first echo lights received (eg, the number of sensors used to receive the first echo lights) is smaller than the number of second echo lights received (eg, the number of sensors used to receive the second echo lights).

[0097] Furthermore, in order to reduce the amount of calculation required in the fusion process, at least a part of the receiving angle of the second echo light can be consistent with the receiving angle of the first echo light, and the sensors that have the same receiving angles but are used to receive the first echo light and the second echo light respectively can be arranged adjacent to each other. In this case, part of the image information based on the second echo light can be corresponded to the position information based on the first echo light with a relatively small error, that is, in the detection information, the corresponding image information (for example, color or grayscale) can be attached to the data point corresponding to each position information.

[0098] Exemplarily, due to the limitations of sensor structure and sensing principle, the number of second sensors in the second sensor group 22 may be much higher than the number of first sensors in the first sensor group 21. In this case, each first sensor may correspond to multiple second sensors to fuse the information of the two. For example, the number of second sensors is 9 times the number of first sensors, and the first sensors are arranged in an array. Then, the distance information and image information received by the first sensor (denoted as A) and the second sensor (denoted as B1) with the same receiving angle (or within the allowable error range) can obviously be directly fused. At the same time, the distance information provided by sensor A can also be fused with the image information provided by the other 8 second sensors adjacent to sensor B5 (denoted as B2 to B9 respectively), so that the image information provided by the nine second sensors B1 to B9 (for example, an image unit containing nine pixels) has the same distance information.

[0099] In some optional embodiments, the first sensor group 21 constructs a first field of view (to obtain information within the first detection area), and the three-dimensional perception device in the first field of view can effectively obtain the distance information of the object (of course, some areas outside the first field of view may also obtain distance information, but this information may produce some acceptable or unacceptable distortion). Correspondingly, the second sensor group 22 constructs a second field of view (to obtain information within the second detection area), and the three-dimensional perception device in the second field of view can effectively obtain the image information of the object.

[0100] In actual applications, the first field of view and the second field of view may not completely overlap. For example, the second field of view may be smaller than the first field of view. In this case, the second field of view can be set in the central area of ​​the first field of view so that the overlapping area of ​​the first field of view and the second field of view can obtain the best three-dimensional perception effect.

[0101] That is to say, for the first field of view and the second field of view, the overlapping area of ​​the two can be set as a preset region of interest, and the overlapping area here may be the entire area of ​​either the first field of view or the second field of view.

[0102] The beneficial effects of this embodiment are:

[0103] By receiving the first echo light and the third light beam, the correlated position information and image information of the target detection object are calculated, so that the perception result with position information (distance information) and image information is obtained through a single sensor, which provides a good foundation for the sensing of the target detection object, can achieve better perception accuracy, and avoids the problems of long time and unpredictable reliability caused by the complex processing of multi-sensor fusion.

[0104] According to the above embodiment, in another embodiment, the receiving component includes a three-stage filter (a type of filter 24), and the three-stage filter is arranged before the second sensor group 22, and the image information of the target detection object is obtained by filtering the second echo light.

[0105] In an optional embodiment, the wavelength of the detection light is not equal to the wavelength of the second echo light beam.

[0106] It can be understood that in the solution provided in this embodiment, in order to better obtain the image information of the target detection object, the interference of the first echo light and the detection light should be eliminated as much as possible, so a three-stage filter is set in front of the second sensor group 22 to prevent the detection light and the first echo light from passing through.

[0107] As for the reception of the first echo light, as mentioned above, the form and property of the detection light can be pre-modulated, and the first sensor group 21 can be set corresponding to the modulation mode of the detection light, so that it can be better received without filtering.

[0108] In addition, in an optional implementation manner of this embodiment, the first sensor group 21 and the second sensor group 22 are located in the same plane.

[0109] In another embodiment parallel to this embodiment:

[0110] The receiving assembly includes a filter element 24 and a sensor element;

[0111] The filter element 24 comprises:

[0112] A first filter 24 for allowing the first echo light to pass therethrough; and

[0113] A second filter 24 for allowing the second echo light to pass through;

[0114] The sensing element comprises:

[0115] a distance measuring part provided on the forward path of the first echo light passing through the first filter element 24, the distance measuring part being used for obtaining the distance information of the target detection object at a specified angle according to the first echo light and the detection light; and,

[0116] A color measuring part is arranged on the forward path of the second echo light passing through the second filter element 24, and is used for obtaining the image information of the target detection object at the specified angle according to the electromagnetic wavelength of the second echo light.

[0117] It is worth noting that the color measurement part can be used to obtain the RGB value of the target detection object, and can also be used to obtain the grayscale value of the target detection object.

[0118] In these two parallel embodiments, the function of the filter element 24 is to separate the first echo light and the second echo light at the same angle to obtain the coordinate information and image information of the specific position of the target detection object at that angle, thereby obtaining correlated color and position perception results without fusion.

[0119] In a specific implementation, the first filter element 24 and the second filter element 24 can be two independent structures that are separately arranged (in some embodiments, the two independent structures can be integrally formed or connected to each other), and the position of the first filter element 24 is determined according to the emission angle of the detection light. The position of the second filter element 24 can be set adjacent to the first filter element 24 to obtain a second echo light that is slightly deviated from the first echo light. Although this setting slightly loses the strict correspondence between the image information (second echo light) and the position information (first echo light), the overall reliability of the device is better and has better cost compression potential.

[0120] In another optional embodiment, the first filter element 24 and the second filter element 24 can be two states of an independent structure under different conditions, that is, the filter element 24 as a whole is a device with variable optical properties, and the optical properties of the filter element 24 are adjusted by control components or other components to obtain image information (second echo light) and position information (first echo light) at the same angle in different time periods. In this embodiment, under the premise of ignoring the movement of the target detection object and the sensor itself, the image information (second echo light) and the position information (first echo light) have strict correspondence, and when the collection time interval between the first echo light and the second echo light is small enough, the errors caused by the movement of the target detection object and the sensor itself will not have a recognizable impact on the perception results.

[0121] The specific implementation of this implementation method can be based on the following strategies:

[0122] At the first moment, the detection light is emitted, and the optical property of the filter element 24 is adjusted to the state of the second filter element 24, that is, the second echo light is allowed to pass through, thereby completing the reception of the second echo light;

[0123] At the second moment, the optical property of the filter element 24 is adjusted to the state of the first filter element 24, that is, the first echo light is allowed to pass through, thereby completing the reception of the first echo light;

[0124] In some implementations based on scanning perception, the following further includes:

[0125] At the third moment, the emission angle of the detection light is adjusted;

[0126] The first moment, the second moment, and the third moment are used as loop units and executed in a loop until all preset scanning angles (i.e., the emission angles of the detection light) have completed the scanning. Then, the current scanning ends and the perception result of the current scanning is obtained.

[0127] It is worth noting that in the application scenario, the three-dimensional sensing device has a preset scanning frequency, such as a scanning frequency of 1 Hz. In this case, it can be understood that the above-mentioned "loop execution with the first moment, the second moment, and the third moment as the loop unit until all the preset scanning angles (that is, the emission angle of the detection light) have completed the scan" action will be executed once per second.

[0128] In particular, in order to better achieve the above-mentioned filtering, the properties of the first echo light (actually depending on the optical properties of the detection light) can be set to have a clearly distinguishable difference from the optical properties of the second echo light. For example, in a preferred embodiment, considering that the main purpose of the second echo light is to obtain image information of the target detection object, the visible part of the second echo light (for example, electromagnetic waves with a wavelength of 400nm to 760nm) will be identified as key information. In order to distinguish it, infrared or ultraviolet electromagnetic waves can be selected as the form of detection light, or vibration waves can be selected as the form of detection light. For example, in another optional embodiment, polarization or other optical properties can be considered to distinguish the first echo light and the second echo light, which are both electromagnetic waves.

[0129] The beneficial effects of this embodiment are:

[0130] The first echo light and the second echo light are separated by the filter element 24, thereby avoiding interference between the mixed received first echo light and the second echo light during the operation of the control component, thereby providing a good data basis for better perception accuracy.

[0131] According to any of the above embodiments, in yet another embodiment:

[0132] The first sensor group 21 and the second sensor group 22 are not coplanar;

[0133] The receiving assembly includes a beam splitter 23;

[0134] The beam splitter 23 is disposed between the first sensor group 21 and the second sensor group 22;

[0135] The beam splitter 23 is used for:

[0136] transmitting the first echo light so that the first echo light is received by the first sensor group 21; reflecting the second echo light so that the second echo light is received by the second sensor group 22;

[0137] or,

[0138] The first echo light is reflected so that the first echo light is received by the first sensor group 21 ; and the second echo light is transmitted so that the second echo light is received by the second sensor group 22 .

[0139] The above provides a solution of this embodiment. In practical applications, this embodiment can be specifically as follows:

[0140] The receiving assembly includes a beam splitter 23 and a sensor;

[0141] The beam splitter 23 is used for:

[0142] receiving the first echo light and the second echo light;

[0143] deriving the first echo light via a first path; and

[0144] deriving the second echo light via a second path;

[0145] The sensing element comprises:

[0146] a distance measuring part arranged on the first path, the distance measuring part being used to obtain distance information of the target detection object at a specified angle according to the first echo light and the detection light; and,

[0147] A color measuring part is arranged on the second path, and the color measuring part is used to obtain the image information of the target detection object at the specified angle according to the electromagnetic wavelength of the second echo light.

[0148] In an optional embodiment, the transmitting component is configured to transmit the detection light in the form of electromagnetic waves, so that the beam splitter 23 can perform wave selection and beam splitting for the same electromagnetic waves. In this case, the beam splitter 23 can be an optical element that transmits one of the first echo light and the second echo light and reflects the other of the first echo light and the second echo light.

[0149] In another optional embodiment, the transmitting component is configured to transmit detection light in the form of vibration waves. In this case, the beam splitter 23 can be an element that can transmit vibration waves and refract / reflect electromagnetic waves.

[0150] In this embodiment, since the beam splitter 23 has separated the first echo light and the second echo light, the setting positions of the distance measuring part and the color measuring part of the receiving component are determined based on the derivation paths of the beam splitter 23 for the first echo light and the second echo light.

[0151] In addition, the description of the switching of the scanning frequency and the detection light emission angle in the above-mentioned embodiment is also applicable to the present embodiment and will not be repeated here.

[0152] Different from the above-mentioned embodiments, the beam splitter 23 in this embodiment can separate the first echo light and the second echo light and guide them to different paths. Therefore, the control logic of time-sharing reception of the first echo light and the second echo light in the above embodiment can be omitted in this embodiment.

[0153] The beneficial effects of this embodiment are:

[0154] The separation of the first echo light and the second echo light is achieved by the beam splitter 23, thereby avoiding interference between the mixed received first echo light and the second echo light during the operation process of the control component, thereby providing a good data basis for better perception accuracy.

[0155] According to any of the above embodiments, in yet another embodiment:

[0156] The receiving component includes a sensor element, and the sensor element is a sensor array, and the sensor array includes:

[0157] A first area for receiving the first echo light obtained after the detection light within a preset sensing angle range is reflected by the target detection object; and,

[0158] A second area for receiving the second echo light within a preset second angle range;

[0159] The second angle range is not less than: the angle range of the first echo light obtained after the detection light within the first specified angle range is reflected by the target detection object.

[0160] In this embodiment, the perception of the position information and image information of the target detection object with a corresponding relationship can be achieved without adding the filter element 24 and / or the beam splitter 23 provided in the above embodiment. Specifically:

[0161] In an embodiment where the first filter element 24 and the second filter element 24 are two independent structures that are separately arranged, the second filter element 24 is arranged at an adjacent position to the first filter element 24 to obtain a second echo light that is slightly deviated from the first echo light. In this manner, the receiving components of the first echo light and the second echo light may be the same (e.g., components for detecting electromagnetic waves) or different. That is, the perception of the first echo light and the second echo light is based on the filter element 24.

[0162] In the present embodiment, the sensing array in the receiving component is divided into a first area and a second area. The separation of the first echo light and the second echo light is achieved by using different receiving components in the first area and the second area (for example, a vibration wave receiving element array is provided in the first area to realize the reception of the first echo light in the form of vibration waves, and an electromagnetic wave receiving element array is provided in the second area to realize the reception of the second echo light in the form of electromagnetic waves), or by collecting beams of different properties in the first area and the second area (for example, the first area and the second area are both electromagnetic wave receiving element arrays, but in the first area only non-visible light consistent with the detection light is collected / the visible light signal is filtered out after collecting the electromagnetic wave, and in the second area only visible light is collected / the non-visible light signal is filtered out after collecting the electromagnetic wave).

[0163] That is to say, this embodiment realizes the separate reception of the first echo light and the second echo light through a receiving component with a large field of view. At the same time, the function of this embodiment is not limited to this, and will be specifically described below from the perspective of three-dimensional perception realization.

[0164] In order to achieve three-dimensional perception of the target detection object, it is necessary to perceive the target detection object based on the first echo light and the second echo light, wherein the perception of the second echo light is based on the diffuse reflection of ambient light, so the appropriate second echo light signal can be received at any specified angle; and the first echo light is formed by the reflection of the detection light on the target detection object, so there will be some optional forms and schemes based on the scanning of the detection light / first echo light.

[0165] First of all, it should be made clear that since the shape and angle of the target detection object are possible in many ways, the first echo light formed after the detection light is reflected by the target detection object should be understood as a beam returned along the detection light emission path (or, as mentioned above, a return beam with a slight deviation within the allowable range), rather than a total reflection case with equal incident angle and exit angle (because the beam angle after total reflection cannot be predicted). Therefore, the first echo light can be understood to be an echo similar to diffuse reflection to some extent, and it can be expected that the intensity of the first echo light will be significantly weakened compared to the detection light.

[0166] Therefore, in the first solution, in order to improve the intensity of the first echo light, thereby improving the collection accuracy of the first echo light and providing a basis for improving the accuracy of the subsequent perception results, the transmitting component is configured as a laser of a linear light source (usually can provide better laser intensity than a surface light source). However, for a linear light source, the signal returned at a specific angle is a point signal. In order to meet the three-dimensional perception requirements of the target detection object, it is necessary to collect multiple point signals to form a point cloud to describe the overall picture of the target detection object. In an optional embodiment, the three-dimensional perception device further includes a scanning component 50, and the scanning component 50 is used to:

[0167] Adjusting the emission angle of the detection light within a preset sensing angle range;

[0168] Furthermore, since an additional scanning component is introduced to adjust the emission angle of the first light beam, the receiving component will also face angle changes when receiving the second light beam reflected from the first light beam emitted at different angles. Therefore, the scanning component 50 can be used for:

[0169] The first echo light and the second echo light are guided to the filter element 24, the beam splitter 23 or the sensor array.

[0170] Alternatively, by setting a first area with a large field of view and / or a second area with a large field of view of the sensing array, the first echo light returned at each angle can be received by the designated position of the first area, and the second echo light returned at each angle can be received by the designated position of the second area; this method can further reflect the advantages of this embodiment.

[0171] The specific structure of the scanning component 50 will be exemplarily described below.

[0172] In an optional embodiment, the scanning component 50 can be a mechanical component, such as a mechanical rotating component, which can adjust the emission angle of the detection light by changing the orientation of the transmitting component. Correspondingly, the receiving component can be the large field of view receiving component mentioned above, or it can be a small field of view receiving component also provided on the above mechanical component (for example, the transmitting component and the receiving component whose angles have been adjusted on the mechanical rotating structure can realize coaxial transmission and reception).

[0173] In another optional embodiment, the scanning component 50 can be an adjustable optical component, such as MEMS (Micro-Electro-Mechanical System). MEMS integrates a compact micro-vibration mirror on a silicon-based chip, and its core structure is a very small cantilever beam. By controlling the tiny mirror translation and torsional reciprocating motion, the laser is reflected to different angles to complete the scanning, while the laser generator itself is fixed. The drive of the cantilever beam can be achieved based on electrostatic, electromagnetic, electrothermal or piezoelectric methods. Similarly, the angle adjustment of the laser emitted by a fixed laser can be achieved by a rotating mirror assembly or a prism, that is, the laser is guided by rotating the optical element.

[0174] In another optional embodiment, the scanning component 50 can be an electrically controlled component that utilizes the coherence / diffraction characteristics of electromagnetic waves, such as OPA (Optical Phase Array), which controls the current phase of each element of the phased planar array and utilizes the phase difference to allow wave sources at different positions to interfere with each other (similar to when two circles of water waves are superimposed on each other, some directions will cancel each other out, while others will enhance each other), thereby pointing in a specific direction, and reciprocating control can achieve a scanning effect.

[0175] The beneficial effects of the above scheme are:

[0176] The scanning component 50 is introduced to adjust the emission angle of the detection light and receive the sensing beams (the first echo light and the second echo light) at different angles by the receiving component, thereby reducing the influence of background noise on the accuracy of the sensor.

[0177] Different from the above-mentioned scheme based on linear light sources, the second scheme omits the scanning component 50 through the surface light source, and can obtain the position information and image information of the target detection object in a single acquisition with a light source with a larger radiation surface. In this scheme, although the single acquisition is limited by the problem of the intensity of the surface light source (in fact, in order to achieve the resolution improvement of the surface light source, the surface light source cannot be achieved only by the laser array, and the gaps between the multiple linear light sources emitted by the laser array still need to be filled by optical means, so the achievable peak intensity of the surface light source is still weaker than that of the linear light source), which may lead to a decrease in perception accuracy, the acquisition frequency in the second scheme is better than that in the first scheme due to the elimination of the scanning process, and thus the impact of errors can be reduced by increasing the acquisition frequency.

[0178] In the second solution without the need for the scanning component 50, a receiving component with a large field of view will be necessary, that is, by setting a first area with a large field of view and / or a second area with a large field of view of the sensing array, the first echo light returned at each angle can be received by the designated position of the first area, and the second echo light returned at each angle can be received by the designated position of the second area; this method can further reflect the advantages of this embodiment.

[0179] The beneficial effects of this embodiment are:

[0180] Through a sensing array with a large field of view, a first area for receiving the first echo light and a second area for receiving the second echo light are provided (in some embodiments, the large field of view can also be understood as the first area and the second area having a large field of view at the same time, that is, the field of view area of ​​the sensing array satisfies: being able to respectively receive the first echo light of different angles and the second echo light of different angles through different areas), thereby achieving separate reception of the first echo light and the second echo light, thereby avoiding interference between the mixed received first echo light and the second echo light during the calculation process of the control component, and thus providing a good data basis for better perception accuracy.

[0181] The above provides structural embodiments of the three-dimensional sensing device of the present application. Although these embodiments provide some instructions related to the usage methods, these instructions correspond to the above-mentioned sensor structure. The following will provide a series of method embodiments from the perspective of the perception principle. These method embodiments can be applied to the above-mentioned sensor structure, but this does not mean that only the above-mentioned sensor structure can implement the following method embodiments.

[0182] The present application embodiment provides a detection method, such as Figure 3 As shown, including:

[0183] Step 602, the transmitting component transmits detection light;

[0184] Step 604: a receiving component receives the first echo light and the second echo light; the receiving component includes a plurality of regularly arranged sensors;

[0185] Step 606: the control component obtains detection information of the target detection object according to the first echo light and the second echo light;

[0186] Among them, the first echo light refers to the beam of the detection light after being reflected by the target detection object; the second echo light refers to the electromagnetic wave after the ambient light is diffusely reflected on the target detection object; and the detection information includes the position information of the target detection object and the image information of the target detection object.

[0187] In an optional implementation, the detection method may also be implemented independently of the above components, namely:

[0188] Step 6020, receiving a sensing beam at a specified angle; the sensing beam includes a first echo light and a second echo light, the first echo light refers to a beam obtained by reflection of the target detection object after the detection light is emitted, and the second echo light refers to an electromagnetic wave obtained by diffuse reflection of the ambient light on the target detection object;

[0189] Step 6040: Acquire detection information of the target detection object according to the first echo light and the second echo light, wherein the detection information includes a three-dimensional position of the target detection object and image information corresponding to the three-dimensional position.

[0190] Similar to the above-mentioned structural embodiment, the first echo light is a beam obtained after the detection light is reflected on the target detection object. Therefore, its form (vibration wave, electromagnetic wave, etc.) and properties (phase, wavelength, frequency, etc.) are correlated with the detection light. Among them, the detection light and the first echo light are usually in the same form, and the properties may change slightly due to environmental factors. Such changes will be regarded as errors and are situations that can be allowed and recognized by the receiving component.

[0191] The detection light can be an electromagnetic beam in the form of linear light, spherical light, plane light, etc., or a vibration beam such as sound waves. It is worth noting that the light mentioned in the embodiments of the present application is not limited to visible light, but can also be electromagnetic waves such as infrared light, ultraviolet light or α rays.

[0192] Unlike the detection light which has many possible forms, the second echo light is a beam of ambient light after diffuse reflection on the target detection object. Its form is usually visible light, and in some cases it may also be infrared light and (near) ultraviolet light (formed by reflection or radiated by the target detection object).

[0193] The second echo light in the form of visible light will be able to provide image information at a specified position of the target detection object, thereby providing information similar to that of a visual sensor. In some optional embodiments, the second echo light in the form of infrared and ultraviolet light can also provide other information in addition to image information, such as temperature information at a specified position of the target detection object.

[0194] Similarly, other information including the velocity of the target detection object at a specified position can be additionally obtained through the difference in properties between the first echo light and the detection light.

[0195] When the method of this embodiment is applied to the three-dimensional sensing device provided by the above-mentioned structural embodiment, the execution subject of this embodiment can be understood as the control component or the processing unit in the control component.

[0196] When the method of this embodiment is applied to sensors of other structures, the execution subject of this embodiment can be understood as a processing unit disposed on the sensor or in communication connection with the sensor.

[0197] The beneficial effects of this embodiment are:

[0198] By receiving the first echo light and the third light beam, the correlated position information and image information of the target detection object are calculated, so that the perception result with position information (distance information) and image information is obtained through a single sensor, which provides a good foundation for the sensing of the target detection object, can achieve better perception accuracy, and avoids the problems of long time and unpredictable reliability caused by the complex processing of multi-sensor fusion.

[0199] According to the above method embodiment, in yet another method embodiment:

[0200] The step of obtaining detection information of the target detection object according to the first echo light and the second echo light comprises:

[0201] Calculate the three-dimensional position of the target detection object at the specified angle according to at least one of the flight time, the phase difference and the frequency difference, and determine the color of the three-dimensional position of the target detection object at the specified angle according to the electromagnetic wavelength of the second echo light at the specified angle;

[0202] Among them, the flight time refers to the time from the emission moment of the detection light emitted at the specified angle to the moment when the first echo light at the specified angle is received; the phase difference refers to the phase difference between the detection light emitted at the specified angle and the first echo light at the specified angle; the frequency difference refers to the frequency difference between the detection light emitted at the specified angle and the first echo light at the specified angle.

[0203] For the scheme of obtaining the three-dimensional position according to the flight time, the processing unit can collect the emission time of the detection light and the reception time of the first echo light through a timer or other methods, and then calculate the distance of the target detection object relative to the emission position of the emission detection light (or the first echo light reception position) at the specified angle according to the propagation speed of the detection light in the specific medium, and calculate the three-dimensional position coordinates of the reflection point of the target detection object according to the specified angle and the distance;

[0204] For the scheme based on phase difference and / or frequency difference, the processing unit can record the transmission signal after generating a transmission signal for controlling the transmitting component (for example, a laser, sound source or other vibration source, etc.) to transmit the specified detection light, and after receiving the first echo light, convert it into a corresponding receiving signal, and mix the transmission signal and the receiving signal (for example, this can be done through a mixer), and then perform low-pass filtering and analog-to-digital conversion on the mixed signal for processing to obtain the information that needs to be perceived.

[0205] It is worth noting that the comparison between the detection light and the first echo light can obtain not only the position information but also other information including the speed information.

[0206] The beneficial effects of this embodiment are:

[0207] Based on at least one of the flight time, phase difference and frequency difference between the detection light and the first echo light, the distance information of the detection light reflection point of the target detection object relative to the sensor is obtained, and the image information of the detection light reflection point of the target detection object is obtained through the electromagnetic wavelength of the second echo light, thereby obtaining the distance and image information of the same position of the target detection object without fusion.

[0208] According to any of the above method embodiments, in yet another method embodiment:

[0209] Before the step of calculating the three-dimensional position of the target detection object at the specified angle according to at least one of the flight time, the phase difference and the frequency difference, the step further includes:

[0210] Separating the sensing beam to obtain the first echo light and the second echo light;

[0211] At least one of the flight time, the phase difference and the frequency difference is calculated based on the separated first echo light and the detection light.

[0212] In this embodiment, considering that there are a first echo light formed by the detection light reflected on the target detection object and a second echo light formed by the ambient light reflected on the target detection object in the perception beam, the two may interfere with each other, which increases the difficulty of subsequent data processing (to obtain three-dimensional position information, image information and other information of the target detection object). Therefore, before performing specific data processing, the two are separated (for example, by wavelet selection or filtering) to obtain better perception accuracy.

[0213] The beneficial effects of this embodiment are:

[0214] The separation of the first echo light and the second echo light is achieved, thereby avoiding interference between the mixed received first echo light and the second echo light during the operation process of the control component, thereby providing a good data basis for better perception accuracy.

[0215] According to any of the above method embodiments, in yet another method embodiment:

[0216] The step of receiving a sensing beam at a specified angle comprises:

[0217] Transmitting the detection light at a preset sensing angle, and receiving the sensing beam returned at the sensing angle;

[0218] The sensing angle is updated to any angle within a preset first angle range, and the step of emitting the detection light at the preset sensing angle is returned to until a preset stop condition is met.

[0219] The step of receiving the sensing beam returned at the sensing angle comprises:

[0220] Receive the perception beam returned at the perception angle through a receiving component whose posture angle is consistent with the perception angle; or,

[0221] The sensing beam returned at the sensing angle is received in an area of ​​the sensing array corresponding to the sensing angle.

[0222] The step of emitting the detection light at a preset sensing angle comprises:

[0223] Determine that the posture angle of the scanning component 50 is the sensing angle, and then derive the detection light emitted by the emitting component at the sensing angle through the scanning component 50;

[0224] Correspondingly, the step of updating the perception angle to any angle within a preset first angle range includes:

[0225] The posture angle of the scanning component 50 is adjusted to any angle within a preset first angle range.

[0226] In the above description from the perspective of three-dimensional perception implementation, the scanning methods for the corresponding cases of large field of view receiving components and small field of view receiving components are discussed and explained. The contents and ideas of these discussions are also applicable to the present embodiment and will not be repeated here.

[0227] It is worth mentioning that the setting of the first angle range usually depends on the specific structure of the scanning component 50. For example, for a mechanically rotating scanning component 50, its theoretical first angle range can be 360° in the horizontal plane, and the first angle range outside the horizontal plane depends on the pitch angle range of its mechanical structure (scanning components 50 with other structures are similar and have a range determined by the physical structure). However, in actual application, the 360° horizontal plane angle range may not meet the needs. For example, for a three-dimensional perception device installed at the front end of an autonomous driving vehicle, it only needs to focus on the target detection object in front of the vehicle. Therefore, in this case, the first angle range can be set to 120° (or other angle ranges) directly in front.

[0228] Therefore, from another perspective, when designing the scanning component 50 of the three-dimensional sensing device, the first angle range (and the second angle range) can be determined according to the needs, and a matching scanning component 50 structure can be selected on this basis.

[0229] The beneficial effects of this embodiment are:

[0230] The receiving component receives the sensing beams (first echo light and second echo light) at different angles, thereby reducing the influence of background noise on the accuracy of the sensor.

[0231] Alternatively, through a sensing array with a large field of view, the first echo light and the second echo light can be received separately, thereby avoiding interference between the mixed received first echo light and the second echo light during the operation of the control component, thereby providing a good data basis for better perception accuracy.

[0232] In addition, the emission angle of the detection light is adjusted by the scanning component 50. Based on the scanning component 50 that integrates the emission angle adjustment and the receiving beam guidance, on the one hand, the perception angle range (i.e., the perception field of view angle FOV) is improved, and on the other hand, the introduction of unnecessary structures is reduced, providing better reliability.

[0233] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0234] The following provides specific embodiments from the perspective of application.

[0235] This embodiment proposes a method for implementing a three-dimensional sensing device, which outputs the color and distance information of an object in a sensor, wherein the color part can be used to judge the characteristics of the object, and combined with the distance information can be directly used to judge the route.

[0236] The three main modules of the three-dimensional sensing device provided in this embodiment are:

[0237] (1) Transmitter module (transmits laser for distance measurement);

[0238] (2) Distance measurement module (receives laser for distance measurement);

[0239] (3) Image information measurement module (receives ambient light for color measurement);

[0240] In addition to these three main modules, the sensor can also include a scanning module, a main control module, etc.

[0241] This embodiment provides a variety of optional implementations to better illustrate the solution of the present application.

[0242] like Figure 4 As shown, in the configuration of the first three-dimensional sensing device: the transmitting device (at least a part of the transmitting component) emits a laser (for example, an infrared laser), which enters the ambient space through the scanning module (at least a part of the scanning component 50), and the scanning module simultaneously receives the laser reflected light of the object and the ambient light. The echo passes through a beam splitting system (at least a part of the beam splitting element 23), which can transmit the laser (corresponding to the wavelength band of the transmitting module, such as the infrared laser corresponding to the above example), and reflect the visible light. The laser enters the sensor such as APD or SiPM (at least a part of the first sensor group 21), and the time of its echo is measured to obtain the distance of the object. The visible light part is reflected to another module (at least a part of the second sensor group 22) to measure the RGB color of the object (similar to a camera). The measurement results of these two parts are fed back to the main control (at least a part of the control component) for fusion, so as to obtain the detection information of the object (position + color);

[0243] like Figure 5 As shown, the second three-dimensional sensing device configuration is similar to the first three-dimensional sensing device configuration, except that there may be no wavelength selection and beam splitting module (i.e., beam splitter 23). The filter element 24 disposed upstream of the second sensor group 22 is directly used to filter out the laser emitted by the emitting device, thereby obtaining a visible light result. At the same time, the first sensor is configured as a receiving device corresponding to the laser emission parameters of the emitting device, so that it can only receive the laser emitted by the emitting device (for example, the first sensor group 21 is configured to only respond to the state of emitting laser wavelengths during emission, thereby obtaining position information).

[0244] The third solution is similar to Figure 5The scheme shown does not realize reception through two independent receiving structures of the first sensor group 21 and the second sensor group 22, but adopts a single sensor array. The sensor array uses the same type of sensors, which can respond to both visible light and infrared light (such as Si-based sensors). The received light is received by different parts of the sensor through two different groups of filters (at least a part of the filter element 24), one part measures the time information of the infrared light, and the other part is used to measure the image information of the visible light (the ratio of the three primary colors of RGB), thereby obtaining the detection information of the object.

[0245] The fourth option is Figure 6 As shown, the first solution and the second solution are combined, and the filter element 24 and the beam splitter 23 are used in conjunction with each other.

[0246] like Figure 7 As shown, in the fifth three-dimensional sensing device configuration, there may be no scanning module, and two sets of sensor arrays may be used to directly obtain distance information and image information respectively, and fuse them to obtain 3D information.

[0247] Similarly, Figure 7 In the scheme shown, the two groups of sensor arrays can also be replaced by a scheme similar to the third scheme of a single sensor array combined with a filter element 24, thereby forming a sixth three-dimensional sensing device configuration.

[0248] Alternatively, Figure 7 In the illustrated solution, an additional filter element 24 is added upstream of the second sensor group 22 to better obtain the reflection information of the ambient light, forming Figure 8 The seventh three-dimensional sensing device configuration is shown.

[0249] Further, Figure 7 and Figure 8 The preferred method for the field of view is also shown. Specifically, the second sensor group 22 is used as a receiving device for the visible light reflected light, and the second field of view constructed by it is composed of Figure 7 and Figure 8 The first sensor is used as a receiving device for the laser reflected light, and the first field of view constructed by the first sensor is composed of Figure 7 and Figure 8 The two first viewing areas 61 and the second viewing area 62 shown in FIG. 6 together constitute. In the first viewing area, the area corresponding to the second viewing area 62 is the central viewing area (generally considered to have better information acquisition accuracy), and the area corresponding to the two first viewing areas 61 is the edge viewing area.

[0250] That is to say, Figure 7 and Figure 8In the illustrated solution, the second field of view is smaller than the first field of view, and a more preferred solution can be obtained by overlapping the second field of view with the central field of view area of ​​the first field of view.

[0251] Corresponding to the three-dimensional perception method described in the above embodiment, the application embodiment also provides a three-dimensional perception system, which includes:

[0252] A receiving module, used to receive a sensing beam at a specified angle; the sensing beam includes a first echo light and a second echo light, wherein the first echo light refers to a beam obtained by reflection of the target detection object after the detection light is emitted, and the second echo light refers to an electromagnetic wave obtained by diffuse reflection of the ambient light on the target detection object;

[0253] A calculation module is used to obtain detection information of a target detection object according to the first echo light and the second echo light, wherein the detection information includes a three-dimensional position of the target detection object and image information corresponding to the three-dimensional position.

[0254] In an optional implementation, the operation module includes:

[0255] a comparison calculation unit, configured to calculate the three-dimensional position of the target detection object at the specified angle according to at least one of the flight time, the phase difference and the frequency difference, and determine the color of the three-dimensional position of the target detection object at the specified angle according to the electromagnetic wavelength of the second echo light at the specified angle;

[0256] Among them, the flight time refers to the time from the emission moment of the detection light emitted at the specified angle to the moment when the first echo light at the specified angle is received; the phase difference refers to the phase difference between the detection light emitted at the specified angle and the first echo light at the specified angle; the frequency difference refers to the frequency difference between the detection light emitted at the specified angle and the first echo light at the specified angle.

[0257] In an optional implementation, the operation module includes:

[0258] A separation unit, used for separating the sensing beam to obtain the first echo light and the second echo light;

[0259] A separation calculation unit is used to calculate at least one of the flight time, the phase difference and the frequency difference according to the separated first echo light and the detection light.

[0260] In an optional implementation, the receiving module includes:

[0261] A transmitting and receiving unit, configured to transmit the detection light at a preset sensing angle, and receive the sensing beam returned at the sensing angle;

[0262] The iterative unit is used to update the sensing angle to any angle within a preset first angle range, and return to the step of emitting the detection light at the preset sensing angle until a preset stop condition is met.

[0263] In an optional embodiment, the transmitting and receiving unit includes:

[0264] An angle receiving subunit, configured to receive the perception beam returned at the perception angle through a receiving component whose posture angle is consistent with the perception angle; or

[0265] The large field of view receiving subunit is used to receive the sensing beam returned at the sensing angle in the area of ​​the sensing array corresponding to the sensing angle.

[0266] In an alternative embodiment:

[0267] The transmitting and receiving unit comprises:

[0268] An angle emission subunit, used to determine that the posture angle of the scanning component 50 is the sensing angle, and then derive the detection light emitted by the emission component at the sensing angle through the scanning component 50;

[0269] Correspondingly, the iteration unit includes:

[0270] The angle adjustment subunit is used to adjust the posture angle of the scanning component 50 to any angle within a preset first angle range.

[0271] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0272] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0273] An embodiment of the present application also provides a terminal device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor implements the steps in any of the above-mentioned method embodiments when executing the computer program.

[0274] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0275] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0276] An embodiment of the present application provides a vehicle, including the above-mentioned three-dimensional perception device, the above-mentioned three-dimensional perception system, the above-mentioned three-dimensional perception device control component or the above-mentioned computer-readable storage medium.

[0277] An embodiment of the present application provides a robot, comprising the above-mentioned three-dimensional perception device, the above-mentioned three-dimensional perception system, the above-mentioned three-dimensional perception device control component or the above-mentioned computer-readable storage medium.

[0278] An embodiment of the present application provides a movable device, including the above-mentioned three-dimensional sensing device and a part to be installed.

[0279] Among them, the part to be installed includes a connecting part and a communication part. The connecting part is used to physically connect the three-dimensional sensing device to a designated position on the main body of the movable device, and the communication part is used to communicate and connect the signal of the three-dimensional sensing device with the processing unit of the movable device. It is worth noting that in this case, the control component can be at least a part of the processing unit in the movable device (for example, when the movable device is a vehicle, the control component can be a vehicle computer; when the movable device is a robot, the control component can be a robot processor).

[0280] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0281] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0282] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0283] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0284] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0285] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A three-dimensional sensing device, characterized in that: include: An emitting component, the emitting component is used to emit detection light; A receiving component, the receiving component is used to receive the first echo light and the second echo light; The receiving component includes a plurality of regularly arranged sensors; as well as, A control component, wherein the control component is used to obtain detection information of a target detection object according to the first echo light and the second echo light; Among them, the first echo light refers to the electromagnetic wave after the detection light is reflected by the target detection object; the second echo light refers to the electromagnetic wave after the ambient light is diffusely reflected on the target detection object; and the detection information includes the position information of the target detection object and the image information of the target detection object.

2. The three-dimensional sensing device according to claim 1, characterized in that: The receiving component includes a first sensor group and a second sensor group, and a first detection area corresponding to the first sensor group at least partially overlaps with a second detection area corresponding to the second sensor group; The first sensor group is used to receive the first echo light and obtain the distance information of the object according to the first echo light; The second sensor group is used to receive the second echo light and obtain image information of the object according to the second echo light.

3. The three-dimensional sensing device according to claim 2, characterized in that: The receiving component includes a three-stage filter, which is arranged before the second sensor group and obtains image information of the target detection object by filtering the second echo light.

4. The three-dimensional sensing device according to claim 2, characterized in that: The first sensor group and the second sensor group are not coplanar; The receiving assembly includes a beam splitter; The beam splitter is disposed between the first sensor group and the second sensor group; The beam splitter is used for: transmitting the first echo light so that the first echo light is received by the first sensor group; reflecting the second echo light so that the second echo light is received by the second sensor group; or, The first echo light is reflected so that the first echo light is received by the first sensor group; and the second echo light is transmitted so that the second echo light is received by the second sensor group.

5. The three-dimensional sensing device according to claim 2, characterized in that: The first sensor group and the second sensor group are located in the same plane.

6. The three-dimensional sensing device according to any one of claims 1 to 5, characterized in that: The wavelength of the detection light is not equal to the wavelength of the second echo light beam.

7. The three-dimensional sensing device according to claim 2, characterized in that: The first sensor is a sensor corresponding to the wavelength of the first outgoing light beam, and the second sensor is a CMOS image sensor.

8. The three-dimensional sensing device according to any one of claims 1, 2, 3, 4, 5, and 7, characterized in that: The three-dimensional sensing device further includes a scanning component, which is used to: The emission angle of the detection light is adjusted within a preset sensing angle range.

9. A detection method, characterized in that: The method comprises: The transmitting component transmits the detection light; The receiving component receives the first echo light and the second echo light; the receiving component includes a plurality of regularly arranged sensors; The control component obtains detection information of the target detection object according to the first echo light and the second echo light; Among them, the first echo light refers to the beam of the detection light after being reflected by the target detection object; the second echo light refers to the electromagnetic wave after the ambient light is diffusely reflected on the target detection object; and the detection information includes the position information of the target detection object and the image information of the target detection object.

10. A movable device, characterized in that: It comprises the three-dimensional sensing device described in any one of claims 1 to 8 and a part to be installed.

Images