Control method and device for lidar LiDAR
By identifying vulnerable targets using cameras and adjusting the laser beam intensity and point cloud density of the LiDAR, the problem of LiDAR damaging the camera CMOS is solved, thus protecting vulnerable targets and ensuring the safety of autonomous driving.
Patent Information
- Application Number
- CN202410471679.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-18
- Publication Date
- 2025-10-24
AI Technical Summary
In existing technologies, the laser beam energy of LiDAR is high, which can easily damage the camera CMOS, especially when detecting vulnerable targets, and there is a lack of effective protection methods.
The system detects targets of interest using a camera, and adjusts the laser beam intensity and/or point cloud density of the LiDAR in the target area when the distance between the target and the vehicle is less than a safety threshold and the confidence level meets a preset standard, in order to protect vulnerable targets.
It effectively protects vulnerable targets, such as pedestrians, oncoming vehicles, and people's eyes, from damage caused by the laser beam, while not affecting the normal operation of autonomous driving functions.
Smart Images
Figure CN120831671A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of control of a laser radar LiDAR, and more particularly, to a control method and device for a laser radar LiDAR, a computer program product and a domain controller. BACKGROUND
[0002] With the development of automatic driving technology, the automatic driving function of a vehicle uses a laser radar LiDAR more and more frequently, but there is no good protection method for the detection object of the laser radar LiDAR at present, for example, it has been found that some cameras CMOS are damaged by the laser beam of the LiDAR. SUMMARY
[0003] The inventors of the present application realize that the reason why the camera CMOS is damaged by the laser beam of the LiDAR is that the laser beam energy of the laser radar LiDAR is high, and when the CMOS of the camera is exposed to the laser beam at a close distance, it may cause permanent damage.
[0004] Based on the above realization of the inventors, the present application proposes to adjust the laser beam intensity and / or point cloud density of the laser radar in a specific area based on target detection of a camera to protect the detected target.
[0005] According to one aspect of the present application, a control method for a laser radar LiDAR is proposed, the method comprising: receiving surrounding scene information of a vehicle from a camera; identifying a target of interest based on the surrounding scene information of the vehicle; and when the distance between the target of interest and the vehicle is less than a safety threshold and the confidence of the target of interest meets a preset standard, adjusting the intensity and / or point cloud density of the laser radar LiDAR in the area where the target of interest is located.
[0006] As a supplement or alternative to the above scheme, in the above method, the target of interest includes a pedestrian being photographed, an oncoming vehicle and an eye of a person.
[0007] As a supplement or alternative to the above scheme, in the above method, when the motion attribute of the target of interest meets the prerequisite condition for activation of the automatic driving function and the distance between the target of interest and the vehicle is less than the safety threshold, the intensity and / or point cloud density of the laser radar LiDAR in the area where the target of interest is located is reduced.
[0008] As a supplement or alternative to the above scheme, in the above method, the intensity and / or point cloud density of the laser radar LiDAR in the area where the target of interest is located is reduced by individually controlling each laser emission of the laser radar LiDAR to form a variable point cloud.
[0009] According to another aspect of the present application, a control device for a LiDAR is provided, the device comprising: receiving means for receiving surrounding scene information of a vehicle from a camera; identifying means for identifying a target of interest based on the surrounding scene information of the vehicle; and adjusting means for adjusting intensity and / or point cloud density of the LiDAR in a region where the target of interest is located when a distance between the target of interest and the vehicle is less than a safety threshold and a confidence of the target of interest meets a preset criterion.
[0010] As a supplement or alternative to the above-mentioned solutions, in the above-mentioned device, the target of interest comprises a pedestrian being photographed, an oncoming vehicle, and an eye of a person.
[0011] As a supplement or alternative to the above-mentioned solutions, in the above-mentioned device, the adjusting means is configured to reduce the intensity and / or point cloud density of the LiDAR in the region where the target of interest is located when a motion attribute of the target of interest meets a prerequisite for activation of an autonomous driving function and the distance between the target of interest and the vehicle is less than the safety threshold.
[0012] As a supplement or alternative to the above-mentioned solutions, in the above-mentioned device, the adjusting means is configured to reduce the intensity and / or point cloud density of the LiDAR in the region where the target of interest is located by individually controlling each laser emission of the LiDAR.
[0013] According to yet another aspect of the present application, a computer-readable storage medium is provided, the medium comprising instructions that, when executed, perform the method as described above.
[0014] According to yet another aspect of the present application, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method as described above.
[0015] According to yet another aspect of the present application, a domain controller is provided, the domain controller comprising the device as described above.
[0016] The control scheme for a LiDAR of embodiments of the present application can adjust the intensity of laser beams and the point cloud density of a specific region at a lower cost by receiving surrounding scene information of a vehicle from a camera and identifying a target of interest based on the surrounding scene information of the vehicle (using a target recognition algorithm), thereby protecting the target detected by the LiDAR. BRIEF DESCRIPTION OF DRAWINGS
[0017] The above and other objects and advantages of the present application will become more apparent by describing in detail the following embodiments thereof with reference to the attached drawings, wherein like reference numerals refer to like elements throughout.
[0018] Figure 1 A flowchart of a control method for a LiDAR is shown according to an embodiment of the present application;
[0019] Figure 2 A structural diagram of a control device for a LiDAR is shown according to an embodiment of the present application;
[0020] Figure 3 A flowchart of a method for protecting a detected object (vulnerable object) is shown according to an embodiment of the present application; and
[0021] Figure 4 A structural diagram of a system for protecting a detected object (vulnerable object) is shown according to an embodiment of the present application. DETAILED DESCRIPTION
[0022] In the following, the control scheme for a LiDAR according to exemplary embodiments of the present application will be described in detail with reference to the accompanying drawings.
[0023] Figure 1 A flowchart of a control method 1000 for a LiDAR is shown according to an embodiment of the present application. As shown in Figure 1 The control method 1000 for a LiDAR includes the following steps:
[0024] In step S110, scene information around the ego vehicle is received from a camera;
[0025] In step S120, a target of interest is identified based on the scene information around the ego vehicle; and
[0026] In step S130, the intensity and / or point cloud density of the LiDAR in the region where the target of interest is located is adjusted when the distance between the target of interest and the ego vehicle is less than a safety threshold and the confidence of the target of interest meets a preset criterion.
[0027] In the context of the present application, the term "LiDAR" is the acronym for Light Detection and Ranging, i.e. laser detection and ranging, also known as optical radar. LiDAR mainly consists of a laser emitting unit and a laser receiving unit. The working mode of the emitting unit is to emit laser beam layers outward. The more layers, the higher the accuracy, but it also means that the sensor size is larger. After the laser is emitted by the emitting unit, when the laser encounters an obstacle, it will be reflected and thus received by the receiver. The receiver creates a set of point clouds according to the time (or frequency) of each laser emission and return. For example, LiDAR outputs point clouds by analyzing the information such as the size of the reflected energy, the amplitude, frequency and phase of the reflected spectrum of the target object surface, thereby presenting the precise three-dimensional structure information of the target object.
[0028] In step S110, the self-surrounding scene information is received from the camera. Here, the camera can be a vehicle-mounted camera, which has, for example, a 360-degree panoramic view and is capable of identifying target types and target positions. In addition, the term "self-surrounding scene information" means the scene information near the (LiDAR system-equipped) ego vehicle, which can be used to determine or identify whether the detection object of the LiDAR is a target of interest.
[0029] In step S120, the target of interest is identified based on the self-surrounding scene information. In the context of the present application, the target of interest can also be referred to as "sensitive target" or "vulnerable target", i.e. a target object that is easily damaged or affected by the LiDAR laser beam. In one embodiment, the target of interest includes a pedestrian being photographed, an oncoming vehicle, and a person's eye.
[0030] After the target of interest is determined, in step S130, the intensity and / or point cloud density of the LiDAR in the area where the target of interest is located is adjusted when the distance between the target of interest and the ego vehicle is less than a safety threshold and the confidence of the target of interest meets a preset standard. For example, if the confidence of the target object meets the needs of the autonomous driving function (e.g. the motion attribute of the target meets the precondition for activating the AEB function), and the distance is less than the safety threshold, the protection function will be triggered. After the protection function is triggered, the area that needs to be protected is divided, and the LiDAR is notified to reduce the laser intensity and / or point cloud density in this area. In this way, the protection of the target object can be achieved without affecting the operation of the autonomous driving function.
[0031] The above-mentioned control method 1000 for LiDAR can be used to protect the detected object in the scene where any camera, human eye or other vulnerable object is irradiated by laser.
[0032] Furthermore, since the detection principle of LiDAR is to emit a laser beam toward the object being measured and receive the reflected laser light to calculate the properties of the target object, in one embodiment, each laser emission of the LiDAR is individually controlled to form a variable point cloud (wherein a laser point cloud is composed of many laser beams sent in sequence), thereby reducing the intensity and / or point cloud density of the LiDAR in the area where the target of interest is located.
[0033] In one embodiment, a designated area in the sensor fusion coordinate system (i.e., the coordinate system of the camera and LiDAR) is sent to the LiDAR to notify the LiDAR to reduce the laser intensity and / or point cloud density in that area. In another embodiment, in addition to the designated area in the fusion coordinate system, setting parameters for the laser intensity and / or point cloud density of the designated area are also sent to the LiDAR. This may depend on the control command reception standard of the LiDAR.
[0034] Figure 3 FIG2 shows a flow chart of a method for protecting a detected object (a vulnerable object) according to an embodiment of the present application. Figure 3 As shown, the entire method flow starts at step S310 (for example, the method / function is triggered by the user or automatically activated by the autonomous driving system). In step S320, image input is received from the camera; then, in step S330, the target of interest (for example, the detected object to be protected) is identified; then, in step S340, it is determined whether the distance threshold and the confidence of the target meet the requirements. If satisfied, step S345 is executed to reduce the LiDAR laser emission power or point cloud density in the target area. If not satisfied, it is further determined in step S350 whether the function is stopped. If not stopped, jump back to step S320 (continue to receive image input from the camera), otherwise enter step S360 to end the entire method flow.
[0035] In addition, it will be readily understood by those skilled in the art that the control method 1000 for a LiDAR provided in one or more of the above embodiments of the present application can be implemented by a computer program. For example, the computer program is contained in a computer program product, and when the computer program is executed by a processor, the control method 1000 for a LiDAR according to one or more embodiments of the present application is implemented. For another example, when a computer-readable storage medium (such as a USB flash drive) storing the computer program is connected to a computer, running the computer program can execute the control method 1000 for a LiDAR according to one or more embodiments of the present application.
[0036] refer to Figure 2Fig. 2 shows a schematic diagram of a control device 2000 for a LiDAR according to an embodiment of the present application. As shown in Fig. 2, the control device 2000 comprises a receiving unit 210, an identifying unit 220 and an adjusting unit 230. The receiving unit 210 is configured to receive surrounding scene information of a vehicle from a camera; the identifying unit 220 is configured to identify a target of interest based on the surrounding scene information of the vehicle; and the adjusting unit 230 is configured to adjust intensity and / or point cloud density of the LiDAR in a region where the target of interest is located when a distance between the target of interest and the vehicle is less than a safety threshold and a confidence of the target of interest meets a preset criterion. Figure 2
[0037] In the context of the present application, the term “LiDAR” stands for Light Detection and Ranging, i.e. laser detection and ranging, also known as optical radar. LiDAR mainly consists of a laser emitting unit and a laser receiving unit. The working mode of the emitting unit is to emit laser beam layers outward. The more layers, the higher the accuracy, but it also means that the sensor size is larger. After the laser emitting unit emits the laser, when the laser encounters an obstacle, it will be reflected and thus received by the receiver. The receiver creates a set of point clouds according to the time (or frequency) of each laser emission and return. For example, LiDAR outputs point clouds by analyzing the information such as the size of the reflected energy, the amplitude, frequency and phase of the reflected spectrum of the target object surface, thereby presenting the precise three-dimensional structure information of the target object.
[0038] The receiving unit 210 is configured to receive surrounding scene information of a vehicle from a camera. Here, the camera can be a vehicle-mounted camera, which has, for example, a 360-degree panoramic view and is capable of identifying target types and target positions. In addition, the term “surrounding scene information of a vehicle” means scene information near the vehicle (loaded with a LiDAR system), which can be used to determine or identify whether the detection object of the LiDAR is a target of interest.
[0039] The identifying unit 220 is configured to identify a target of interest based on the surrounding scene information of the vehicle. In the context of the present application, the target of interest can also be referred to as a “sensitive target” or an “easily damaged target”, i.e. a target object that is easily damaged or affected by the LiDAR laser beam. In one embodiment, the target of interest includes a pedestrian taking a photo, an oncoming vehicle and a person's eye.
[0040] After the target of interest is determined, the adjusting device 230 is configured to adjust the intensity and / or point cloud density of the laser radar LiDAR in the region where the target of interest is located when the distance between the target of interest and the ego vehicle is less than a safety threshold and the confidence of the target of interest meets a preset criterion. For example, if the confidence of the target object meets the needs of the automatic driving function (e.g., the motion attribute of the target meets the precondition for activating the AEB function), and the distance is less than the safety threshold, the protection function will be triggered. After the protection function is triggered, the adjusting device 230 is configured to divide the region that needs to be protected and notify the laser radar LiDAR to reduce the laser intensity and / or point cloud density in the region. In this way, the protection of the target object can be achieved without affecting the operation of the automatic driving function.
[0041] In one embodiment, the adjusting device 230 is configured to form a variable point cloud by individually controlling each laser emission of the laser radar LiDAR, thereby reducing the intensity and / or point cloud density of the laser radar LiDAR in the region where the target of interest is located.
[0042] Reference Figure 4 It shows a schematic diagram of protecting a detected object (vulnerable object) according to an embodiment of the present application. As shown in Figure 4 The device 410 for protecting a detected object (e.g., including the control device 2000 for the laser radar LiDAR) receives the surrounding scene information of the ego vehicle from the camera 420 and identifies a target of interest 440 based on the received surrounding scene information of the ego vehicle. When the distance between the target of interest 440 and the ego vehicle is less than a safety threshold and the confidence of the target of interest 440 meets a preset criterion, the device 410 for protecting a detected object divides the working region (of the laser radar LiDAR) (wherein the region that needs to be protected is region 432, and the region that does not need additional protection is region 434), and adjusts the intensity and / or point cloud density of the laser radar 430 in the region where the target of interest is located (i.e., region 432).
[0043] In one or more embodiments, the aforementioned control device 2000 for the laser radar LiDAR can be integrated in various types of domain controllers (e.g., an automatic driving domain controller).
[0044] In summary, the control scheme for the laser radar LiDAR of the embodiments of the present application can adjust the laser beam intensity and point cloud density of a specific region at a lower cost by receiving the surrounding scene information of the ego vehicle from the camera and identifying the target of interest based on the surrounding scene information of the ego vehicle (using a target recognition algorithm), thereby protecting the target detected by the laser radar LiDAR.
[0045] The above examples mainly illustrate the control scheme of the embodiments of the present application for laser radar LiDAR. Although only some of the embodiments of the present application are described, it should be understood by those skilled in the art that the present application can be implemented in many other forms without departing from the spirit and scope thereof. Therefore, the examples and embodiments shown are considered illustrative rather than limiting, and the present application can encompass various modifications and alternatives without departing from the spirit and scope of the present application as defined in the claims.
Claims
1. A control method for a LiDAR, characterized by, The method comprises: receiving, from a camera, information about a scene surrounding a vehicle; identifying a target of interest based on the information about the scene surrounding the vehicle; and adjusting intensity and / or point cloud density of a LiDAR in a region where the target of interest is located when a distance between the target of interest and the vehicle is less than a safety threshold and a confidence of the target of interest meets a preset criterion.
2. The method of claim 1, wherein, The target of interest includes a pedestrian taking a photo, an oncoming vehicle, and an eye of a person.
3. The method of claim 1, wherein, The intensity and / or point cloud density of the LiDAR in the region where the target of interest is located is reduced when a motion attribute of the target of interest meets a precondition for activation of an autonomous driving function and the distance between the target of interest and the vehicle is less than the safety threshold.
4. The method of claim 3, wherein, The intensity and / or point cloud density of the LiDAR in the region where the target of interest is located is reduced by individually controlling each laser emission of the LiDAR to form a variable point cloud.
5. A control device for a LiDAR, characterized in that The device comprises: receiving means for receiving, from a camera, information about a scene surrounding a vehicle; identifying means for identifying a target of interest based on the information about the scene surrounding the vehicle; and adjusting means for adjusting intensity and / or point cloud density of a LiDAR in a region where the target of interest is located when a distance between the target of interest and the vehicle is less than a safety threshold and a confidence of the target of interest meets a preset criterion.
6. The apparatus of claim 5, wherein, The target of interest includes a pedestrian taking a photo, an oncoming vehicle, and an eye of a person.
7. The apparatus of claim 5, wherein, The adjusting means is configured to reduce the intensity and / or point cloud density of the LiDAR in the region where the target of interest is located when a motion attribute of the target of interest meets a precondition for activation of an autonomous driving function and the distance between the target of interest and the vehicle is less than the safety threshold.
8. The apparatus of claim 7, wherein, The adjusting means is configured to reduce the intensity and / or point cloud density of the LiDAR in the region where the target of interest is located by individually controlling each laser emission of the LiDAR to form a variable point cloud.
9. A computer program product comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the method of any one of claims 1 to 4.
10. A domain controller, characterized by The domain controller comprises the device of any one of claims 5 to 8.