System and method for anti-dazzling smart vehicle lamp processing

Abstract

The invention discloses a system and method for anti-dazzling intelligent vehicle lamp processing. The system comprises a vehicle sensor, a controller and an intelligent vehicle lamp module. Wherein the vehicle sensor comprises a laser radar and a camera; the controller is used for receiving point cloud and image data collected by the laser radar and the camera, identifying a target type according to the image data, converting detected coordinate information of pedestrians and / or vehicles into coordinate information under a vehicle lamp coordinate system, and sending the coordinate information to the vehicle lamp coordinate system. The coordinate information of the pedestrian and / or the vehicle is sent to the intelligent vehicle lamp module so as to control the intelligent vehicle lamp module to reduce the brightness of the pedestrian area and / or the vehicle area; wherein the intelligent vehicle lamp module is used for projecting coordinates of a target to an image of a vehicle lamp to obtain a projection area after receiving coordinate information, sent by the controller, of a pedestrian and / or a vehicle, and turning down the brightness of the vehicle lamp in the projection area. By adopting the scheme of the invention, the experience of pedestrians and drivers of opposite vehicles is optimized.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent vehicle lights, and in particular to a system and method for processing anti-glare intelligent vehicle lights. Background Art

[0002] With the continuous improvement of social civilization, people's evaluation of car lights is no longer just reflected in basic aspects such as appearance and brightness, but also puts forward new requirements for higher-level aspects such as intelligence and humanization of car lights.

[0003] When a vehicle is driving at night under poor lighting conditions, the headlights are brighter than the surrounding environment, which can easily cause dazzle to pedestrians in front of the vehicle and drivers of oncoming vehicles. This dazzle can make people feel blinded and uncomfortable, and in severe cases may cause people to be blinded for a moment, leading to traffic accidents. Summary of the invention

[0004] An object of the present invention is to provide a system and method for anti-glare intelligent vehicle light processing, so as to at least partially solve the above-mentioned problems of the prior art.

[0005] To achieve the above object, the present invention provides a system for anti-glare intelligent vehicle light processing, comprising:

[0006] Vehicle sensors, controllers and smart car light modules; vehicle sensors include lidar and cameras;

[0007] The controller is used to: receive the point cloud data and image data collected by the laser radar and the camera respectively, identify the target type according to the image data, identify the target 3D position according to the point cloud raw data, identify the target three-dimensional coordinates according to the laser radar point cloud raw data, fuse the target type information with the three-dimensional coordinate information, track the detected vehicle, determine whether the vehicle is an oncoming vehicle, convert the detected coordinate information of the pedestrian and / or oncoming vehicle into coordinate information in the headlight coordinate system, and send the coordinate information of the pedestrian and / or oncoming vehicle to the smart headlight module to control the smart headlight module to reduce the brightness of the pedestrian area and / or the oncoming vehicle area;

[0008] Among them, the intelligent headlight module is used to: after receiving the coordinate information of pedestrians and / or oncoming vehicles sent by the controller, project the coordinates of the target onto the image of the headlight to obtain the projection area, and lower the brightness of the headlight in the projection area.

[0009] Optionally, the vehicle sensors, controller and smart light module communicate via an industrial Ethernet connection.

[0010] Optionally, the controller is further configured to:

[0011] Pre-configure the rotation and translation transformation matrix from the laser radar coordinate system to the smart headlight coordinate system; and / or

[0012] Pre-configure the rotation and translation transformation matrix from the lidar coordinate system to the camera coordinate system.

[0013] Optionally, the controller is also used to: project the point cloud data collected by the laser radar onto the image data, and determine the target type corresponding to the point cloud data based on the projection result of the point cloud data.

[0014] Optionally, the controller is also used to: construct a 3D bounding box of the target based on the point cloud data collected by the lidar, construct a 2D bounding box of the target based on the image data collected by the camera, project the 3D bounding box onto the image collected by the camera and match it with the 2D bounding box to obtain the target type.

[0015] Optionally, the controller is further used to: convert the coordinate information of the 3D bounding box in the laser radar coordinate system into the coordinate information of the 3D bounding box in the smart headlight coordinate system;

[0016] The intelligent car light module is also used to: project the vertices of the 3D bounding box onto the image of the car light to obtain projection points according to the coordinate information of the 3D bounding box in the car light coordinate system; and construct a maximum 2D bounding box as the projection area according to the projection points.

[0017] Optionally, the controller is used to: identify target types, including pedestrians and / or vehicles, based on image data collected by the camera.

[0018] Optionally, the controller is further configured to:

[0019] If the target is a vehicle, an ID is assigned to the vehicle, the vehicle ID and movement status are saved, and the vehicle's movement speed is calculated based on the position information of the vehicle ID during the next detection. If the vehicle's movement speed is less than a threshold, the vehicle is judged to be an oncoming vehicle, and the coordinate information of the detected target vehicle is converted into the coordinate information in the vehicle light coordinate system;

[0020] Otherwise, the vehicle is judged to be a same-direction vehicle and no coordinate transformation is performed.

[0021] An embodiment of the present invention further provides a method for anti-glare intelligent vehicle light processing, which is used to execute the system for anti-glare intelligent vehicle light processing described in any of the above embodiments and optional solutions thereof, and the method includes:

[0022] Collect point cloud data and image data through laser radar and camera respectively;

[0023] Identify the target type based on the image data, identify the target 3D position based on the point cloud raw data, convert the coordinate information of the detected pedestrians and / or vehicles into coordinate information in the headlight coordinate system, project the coordinates of the pedestrians and / or vehicles onto the image of the headlight to obtain the projection area, and lower the brightness of the headlight in the projection area.

[0024] Compared with the prior art, the present invention has at least the following advantages:

[0025] By utilizing the solution of the present invention, the 3D position and bounding box size of the person in front of the vehicle and the oncoming vehicle can be detected, and the brightness of the light area projected onto these targets can be accurately lowered to achieve an anti-glare effect, thereby optimizing the experience of pedestrians and oncoming vehicle drivers, while also reducing the risk of accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A schematic diagram of the structure of a system for anti-glare intelligent vehicle light processing provided by an embodiment of the present invention.

[0027] Figure 2 This is a diagram of the operating principle of the perception module provided in an embodiment of the present invention.

[0028] Figure 3 A schematic diagram of a 3D bounding box projection provided by an embodiment of the present invention.

[0029] Figure 4 A schematic diagram of a flow chart of a method provided in an embodiment of the present invention.

[0030] Figure 5 An example of a usage scenario of the solution provided in an embodiment of the present invention.

[0031] Figure 6 A schematic diagram of a projection diagram of a two-dimensional bounding box and a 3D bounding box of a laser radar point cloud obtained by performing target detection using an image taken by a camera provided by the present invention.

[0032] Figure 7 A schematic diagram of intelligent vehicle light projection using the anti-glare intelligent vehicle light processing solution provided by the present invention. DETAILED DESCRIPTION

[0033] In the drawings, the same or similar reference numerals are used to represent the same or similar elements or elements with the same or similar functions. The embodiments of the present invention are described in detail below in conjunction with the drawings.

[0034] In the description of the present invention, the terms "center", "longitudinal", "lateral", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the scope of protection of the present invention.

[0035] The present invention provides a system for anti-glare intelligent vehicle light processing, such as Figure 1 As shown, the system includes: a vehicle sensor 11, a controller 12 and an intelligent vehicle light module 13, wherein the vehicle sensor 11 includes a laser radar 111 and a camera 112.

[0036] The controller 12 is used to: receive the point cloud data and image data collected by the laser radar 111 and the camera 112 respectively, identify the target type according to the image data, convert the coordinate information of the detected pedestrians and / or vehicles into the coordinate information in the headlight coordinate system, and send the coordinate information of the pedestrians and / or vehicles to the smart headlight module 13 to control the smart headlight module 13 to reduce the brightness of the pedestrian area and / or vehicle area;

[0037] The intelligent headlight module 13 is used to: after receiving the coordinate information of the pedestrian and / or vehicle sent by the controller 12, project the coordinates of the target onto the image of the headlight to obtain the projection area, and lower the brightness of the headlight in the projection area.

[0038] In one embodiment, the controller is further used to: track the detected vehicle and determine whether the vehicle is an oncoming vehicle; if it is not an oncoming vehicle, the vehicle coordinate information is not sent to the smart headlight module. In other words, when the detected target is a vehicle, only the headlight brightness of the oncoming vehicle area is adjusted, and for vehicles in the same direction, it may not be adjusted.

[0039] In one embodiment, the vehicle sensor 11, the controller 12 and the smart vehicle light module 13 communicate with each other via, for example, an industrial Ethernet connection.

[0040] In one embodiment, the controller 12 is further configured to:

[0041] Pre-configure the rotation and translation transformation matrix from the laser radar coordinate system to the smart headlight coordinate system; and / or

[0042] Pre-configure the rotation and translation transformation matrix from the lidar coordinate system to the camera coordinate system.

[0043] For example, after the laser radar and smart headlight module are installed and fixed on the vehicle body, they need to be calibrated to obtain the rotation and translation transformation matrix from the laser radar coordinate system to the smart headlight coordinate system.

[0044] In one embodiment, the controller 12 is further used to: project the point cloud data collected by the laser radar onto the image data, and determine the target type corresponding to the point cloud data according to the projection result of the point cloud data.

[0045] In one embodiment, the controller 12 is also used to: construct a 3D bounding box of the target based on the point cloud data collected by the lidar, construct a 2D bounding box of the target based on the image data collected by the camera, and project the 3D bounding box to the 2D bounding box to identify the target type.

[0046] In one implementation, the controller 12 is further configured to: convert the coordinate information of the 3D bounding box in the laser radar coordinate system into the coordinate information of the 3D bounding box in the smart headlight coordinate system;

[0047] The intelligent headlight module 13 is also used to: project the vertices of the 3D bounding box onto the image of the headlight to obtain projection points according to the coordinate information of the 3D bounding box in the headlight coordinate system; and construct a maximum 2D bounding box as a projection area according to the projection points.

[0048] In one embodiment, the controller 12 is further configured to: identify target types, including pedestrians and / or vehicles, based on image data collected by the camera.

[0049] In one embodiment, the controller 12 is further configured to:

[0050] If the target is a vehicle, an ID is assigned to the vehicle, the vehicle ID and movement status are saved, and the vehicle's movement speed is calculated based on the position information of the vehicle ID during the next detection. If the vehicle's movement speed is less than a threshold, the vehicle is judged to be an oncoming vehicle, and the coordinate information of the detected target vehicle is converted into the coordinate information in the vehicle light coordinate system;

[0051] Otherwise, the vehicle is judged to be a same-direction vehicle and no coordinate transformation is performed.

[0052] In one embodiment, the laser radar is used to detect the center position coordinates and the size of the 3D bounding box of the target.

[0053] In one embodiment, the smart car light module 13 can be used to: project the eight vertices of the three-dimensional bounding box of the target in the smart car light coordinate system onto the image of the car light according to the internal parameter matrix of the car light itself, obtain the maximum 2D bounding box of the eight projection points, and lower the light brightness in the area of ​​the maximum 2D bounding box.

[0054] In order to accurately achieve an anti-glare effect for pedestrians and / or oncoming vehicles in front of the vehicle, it is necessary to obtain the type, position and movement speed of the target in front of the vehicle.

[0055] In the present invention, the controller detects the 2D (two-dimensional) bounding box and type of the target in the image through a deep learning neural network, and obtains the 3D (three-dimensional) bounding box and position of the target through algorithms such as clustering or deep learning neural networks. By combining the information of the two, the 3D (three-dimensional) position coordinates (x, y, z), bounding box size (l, w, h) and type (car or person) of the target can be obtained.

[0056] In order to distinguish between vehicles in the same direction and vehicles in the opposite direction, the speed of the vehicle also needs to be known. Therefore, after obtaining the position of the target, target tracking is also required. Each target will be given an ID, and the ID and the motion state will be stored and recorded. In the new round of target detection, the stored and recorded target will be matched with the newly detected target. After the match is successful, the motion state is calculated and updated according to a certain algorithm using the detected new position information and the previously stored motion state, and the speed of each target can be obtained. If the speed of the target of the vehicle type is less than the threshold value of -2m / s, the target can be considered to be an opposite vehicle.

[0057] In addition to distinguishing between vehicles in the same direction and vehicles in the opposite direction, pedestrians in the same direction and pedestrians in the opposite direction can also be distinguished. In the same way as the above-mentioned method of distinguishing between vehicles in the same direction and vehicles in the opposite direction, an ID can also be assigned to each pedestrian target, and its movement state can be judged as the same direction or the opposite direction based on the movement state of the ID.

[0058] Finally, the controller converts the coordinates (x, y, z) of all targets of type human and oncoming vehicles of type car into the smart headlight coordinate system according to the calibrated rotation and translation transformation relationship between the lidar and the smart headlight, and sends the converted coordinates and bounding box size to the smart headlight module via Ethernet. The smart headlight module adjusts the lighting area according to the coordinates and bounding box size to reduce the light intensity in the area of ​​type human and oncoming vehicles of type car.

[0059] In one embodiment, different modules are set in the vehicle sensor 11, the controller 12 and the smart car light module 13. For example, the vehicle sensor 11 has a built-in sensor driver module, the controller 12 has a built-in perception module, and the smart car light module 13 has a built-in light control module. The sensor driver module is responsible for transmitting the point cloud and image data collected by the laser radar and the camera respectively to the perception module; the perception module first performs target detection on the point cloud and image data respectively, and then performs data fusion on the targets detected by the point cloud and the image, and sends the final target detection result to the light control module; the light control module reduces the brightness of the luminous area irradiated to pedestrians and vehicles according to the detected targets, and finally achieves an anti-glare effect.

[0060] The operations of the perception module and the lighting control module are described in detail below, wherein the situations where the brightness of the vehicle lights needs to be adjusted include pedestrians and oncoming vehicles as an example.

[0061] 1) Perception module

[0062] In order for the lighting control module to accurately achieve anti-glare effects for pedestrians and oncoming vehicles in front of the vehicle, the perception module needs to obtain the type, position and movement speed of the target in front of the vehicle. Figure 2 The operating principle diagram of the perception module is shown in FIG. Figure 2 As shown, the perception module may include an image detection module and a point cloud detection module: the image detection module detects the 2D bounding box and type of the target in the image through a deep learning neural network; the point cloud detection module obtains the 3D bounding box and position of the target through algorithms such as clustering or deep learning neural networks; by fusing the perception information of the two, the perception module can obtain the 3D position coordinates (x, y, z), bounding box size (l, w, h) and type (car or person) of the target.

[0063] In order to distinguish between vehicles in the same direction and vehicles in the opposite direction, the speed of the vehicles also needs to be known. Therefore, after obtaining the position of the target, target tracking is also required. Each target will be given an ID, and will be stored and recorded together with the motion state. In the new round of target detection, the stored and recorded targets will be matched with the newly detected targets. After the match is successful, the motion state is calculated and updated according to a certain algorithm using the detected new position information and the previously stored motion state, and the speed of each target can be obtained. If the speed of a target of type car is less than a threshold, such as -2m / s, the target can be considered to be an opposite vehicle.

[0064] Finally, the perception module converts the coordinates (x, y, z) of all targets of type human and oncoming vehicles of type car into the smart headlight coordinate system according to the calibrated rotation and translation transformation relationship between the lidar and the smart headlight, and sends the converted coordinates and bounding box size to the lighting control module via Ethernet.

[0065] 2) Lighting control module

[0066] Figure 3 3D bounding box projection diagram is shown. Figure 3 As shown in the figure, 1 is the headlight image, 2 is the projection point of the 3D bounding box vertex, 3 is the 2D bounding box, 4 is the 3D bounding box vertex, 5 is the 3D bounding box, and 6 is the projection focus. After receiving the target information of the type of person and the type of vehicle, the light control module projects the 8 vertices of the 3D bounding box of the target in the headlight coordinate system onto the headlight image according to the internal parameter matrix of the headlight itself, and then calculates the maximum 2D bounding box of the 8 projected points, and finally lowers the brightness of the bounding box area on the image to achieve an anti-glare effect.

[0067] The present invention also provides a method for anti-glare intelligent vehicle light processing, which is used to execute the system for anti-glare intelligent vehicle light processing described in any of the above embodiments and optional solutions thereof. Figure 4 A flow chart of the method is shown as follows: Figure 4 As shown, the method includes:

[0068] Step 41, collecting point cloud and image data by laser radar and camera respectively;

[0069] Step 42, identifying the target type based on the image data, converting the coordinate information of the detected pedestrians and / or vehicles into coordinate information in the headlight coordinate system, projecting the coordinates of the pedestrians and / or vehicles onto the image of the headlight to obtain a projection area, and lowering the brightness of the headlight in the projection area.

[0070] Compared with the prior art, the present invention has at least the following advantages:

[0071] By utilizing the solution of the present invention, the 3D position and bounding box size of the person in front of the vehicle and the oncoming vehicle can be detected, and the brightness of the light area projected onto these targets can be accurately lowered to achieve an anti-glare effect, thereby optimizing the experience of pedestrians and oncoming vehicle drivers, while also reducing the risk of accidents.

[0072] The solution of the present invention is described below by means of a specific example.

[0073] Figure 5 An example of a usage scenario of the solution provided by the present invention is shown. Figure 5 As shown in the figure, 1 is the vehicle, 2 is the lidar, 3 is the camera, 4 is the pedestrian, 5 is the oncoming vehicle, and 6 is the vehicle in the same direction. The rectangular block in the figure is the 3D bounding box of pedestrians, vehicles and other targets detected by the lidar. The detection result includes the center position coordinates (x, y, z) of the 3D bounding box and the size of the bounding box (l, w, h).

[0074] Figure 6 A schematic diagram showing a projection diagram of a two-dimensional bounding box and a 3D bounding box of a laser radar point cloud obtained by performing target detection using an image taken by a camera provided by the present invention. Figure 6 The solid line frame is the 2D bounding box detected by the image, and the dotted line frame is the 2D maximum bounding box of the 8 vertices of the 3D bounding box projected onto the image. According to the image detection results, the type of target 4 is human, and the types of targets 5 and 6 are cars. By fusing the target detection information of the image and point cloud, the 3D position coordinates (x, y, z), bounding box size (l, w, h) and type (car or person) of the target can be obtained. For example, the 8 vertices of the 3D bounding box are projected onto the image to obtain the 2D bounding box detected by the image that matches the 2D maximum bounding box. Based on the semantic function of the image, the target in the 2D bounding box can be distinguished as a person or a car, thereby obtaining the target type.

[0075] Through target tracking, it can be determined Figure 5 The moving direction of vehicle 6 is the same direction, so no anti-glare is required. The moving direction of vehicle 5 is the opposite direction, so anti-glare is required together with pedestrian 4.

[0076] Figure 7 FIG. 2 shows a schematic diagram of intelligent headlight projection using the anti-glare intelligent headlight processing solution provided by the present invention. Figure 7 As shown, the black box is Figure 5 After the 3D bounding box coordinates of pedestrian 4 and vehicle 5 are converted from the lidar to the headlight coordinate system, they are projected onto the projection plane of the headlight according to the internal parameters of the headlight to form a 2D maximum bounding box. By lowering the brightness of the bounding box area, an anti-glare effect can be achieved for pedestrians and oncoming vehicles.

[0077] The present invention proposes an anti-glare intelligent vehicle lamp technical solution, which can detect the 3D position and bounding box size of people in front of the vehicle and oncoming vehicles, and accurately lower the brightness of the light area projected onto these targets to achieve an anti-glare effect, thereby optimizing the experience of pedestrians and oncoming vehicle drivers, and also reducing the risk of accidents.

[0078] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Those skilled in the art should understand that the technical solutions described in the above embodiments may be modified, or some of the technical features thereof may be replaced by equivalents; these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A system for anti-glare intelligent vehicle light processing, characterized in that: include: Vehicle sensors, controllers and smart lighting modules; vehicle sensors include lidar and cameras; The controller is used to: receive the point cloud data and image data collected by the laser radar and the camera respectively, identify the target type according to the image data, identify the target 3D position according to the original point cloud data, convert the coordinate information of the detected pedestrians and / or vehicles into the coordinate information under the headlight coordinate system, and send the coordinate information of the pedestrians and / or vehicles to the smart headlight module to control the smart headlight module to reduce the brightness of the pedestrian area and / or vehicle area; Among them, the intelligent headlight module is used to: after receiving the coordinate information of pedestrians and / or vehicles sent by the controller, project the coordinates of the target onto the image of the headlight to obtain the projection area, and lower the brightness of the headlight in the projection area.

2. The system for anti-glare intelligent vehicle light processing according to claim 1, characterized in that: The controller is also used to: track the detected vehicle and determine whether the vehicle is an oncoming vehicle; if it is not an oncoming vehicle, the coordinate information of the vehicle will not be sent to the smart headlight module.

3. The system for anti-glare intelligent vehicle light processing according to claim 1, characterized in that: Vehicle sensors, controllers and smart lighting modules communicate via industrial Ethernet.

4. The system for anti-glare intelligent vehicle light processing as claimed in claim 1, characterized in that: The controller is also used to: Pre-configure the rotation and translation transformation matrix from the laser radar coordinate system to the smart headlight coordinate system; and / or Pre-configure the rotation and translation transformation matrix from the lidar coordinate system to the camera coordinate system.

5. The system for anti-glare intelligent vehicle light processing according to any one of claims 1 to 4, characterized in that: The controller is also used to: project the point cloud data collected by the laser radar onto the image data, and determine the target type corresponding to the point cloud data according to the projection result of the point cloud data.

6. The system for anti-glare intelligent vehicle light processing as claimed in claim 5, characterized in that: The controller is also used to: construct a 3D bounding box of the target based on the point cloud data collected by the lidar, construct a 2D bounding box of the target based on the image data collected by the camera, project the 3D bounding box onto the image collected by the camera and match it with the 2D bounding box to obtain the target type.

7. The system for anti-glare intelligent vehicle light processing as claimed in claim 6, characterized in that: The controller is also used to: convert the coordinate information of the 3D bounding box in the laser radar coordinate system into the coordinate information of the 3D bounding box in the smart headlight coordinate system; The intelligent car light module is also used to: project the vertices of the 3D bounding box onto the image of the car light to obtain projection points according to the coordinate information of the 3D bounding box in the car light coordinate system; and construct a maximum 2D bounding box as the projection area according to the projection points.

8. The system for anti-glare intelligent vehicle light processing according to any one of claims 1 to 4, characterized in that: The controller is used to: Identify target types, including pedestrians and / or vehicles, based on image data collected by the camera.

9. The system for anti-glare intelligent vehicle light processing according to any one of claims 1 to 4, characterized in that: The controller is also used to: If the target is a vehicle, an ID is assigned to the vehicle, the vehicle ID and movement status are saved, and the vehicle's movement speed is calculated based on the position information of the vehicle ID during the next detection. If the vehicle's movement speed is less than a threshold, the vehicle is judged to be an oncoming vehicle, and the coordinate information of the detected target vehicle is converted into the coordinate information in the vehicle light coordinate system; Otherwise, the vehicle is judged to be a same-direction vehicle and no coordinate transformation is performed.

10. A method for anti-glare intelligent vehicle light processing, characterized in that: A system for executing anti-glare intelligent vehicle light processing according to any one of claims 1 to 9, the method comprising: Collect point cloud data and image data through laser radar and camera respectively; Identify the target type based on the image data, identify the target 3D position based on the point cloud raw data, convert the coordinate information of the detected pedestrians and / or vehicles into coordinate information in the headlight coordinate system, project the coordinates of the pedestrians and / or vehicles onto the image of the headlight to obtain the projection area, and lower the brightness of the headlight in the projection area.

Images