An ADB headlight control method and system

Abstract

The present disclosure relates to the technical field of vehicle headlight control, and provides an ADB headlight control method, which at least includes the following steps: obtaining information data for controlling the headlight, at least including image data obtained by an imaging device; separating the information data to obtain at least first data and second data, inputting the obtained first data into a trained first model, and inputting the obtained second data into a trained second model. After being trained, the first model and the second model can identify first object information or second object information. According to the obtained first object information or second object information, the LED lights in the LED light group that irradiate the corresponding positions are turned off or blocked. The technical solution of the present application preprocesses the information, can remove a large amount of interference information, and finally identifies certain information for a certain area, improving the detection efficiency and accuracy, that is, having a better detection effect under extreme conditions.

Description

Technical Field

[0001] The present disclosure relates to the technical field of vehicle headlamp control, and particularly to an ADB vehicle headlamp control method and system. Background Art

[0002] Currently, with the development of automotive electronic technology, the safety and comfort of vehicles have received increasing attention. In the control of vehicle headlamps, technologies such as ADB (Adaptive Driving Beam) are adopted. This light distribution control device is used to prevent the driver of a vehicle traveling ahead, a vehicle traveling in the opposite direction, or a pedestrian in the front area of the present vehicle from being dazzled, and on the other hand, to improve the observability of this front area, thereby improving safety during driving. At present, some adaptive high-beam systems often only directly switch between the high beam and low beam of motor vehicles, ignoring the danger that the field of vision is limited during the period when the vehicle switches from high beam to low beam, which is likely to cause accidents. Therefore, it is necessary to improve the adaptive high-beam system to enhance driving safety. Existing object detection and re-identification models already have strong detection and processing capabilities for data of a single visual modality. However, in the real environment, the data that people can obtain is often of multiple modalities, and there are often complementary information between different modalities. Fully exploring the associations between modalities will help improve the performance of the model. Combining multiple modalities with domain adaptation and unsupervised techniques can make more comprehensive use of multi-source, multi-modal information and a large amount of unlabeled data in the real environment to achieve more efficient detection and re-identification.

[0003] Although general control models have achieved good results in the vehicle headlamp ADB control task, there are still some problems. On the one hand, since the camera devices generally collect camera devices on the vehicle, which belong to optical images and are easily affected by the natural environment, complex environmental factors such as different lighting conditions and bad weather seriously affect the image quality in the video and cause interference to vehicle object detection. Therefore, higher requirements are also put forward for the practicality and robustness of object detection algorithms. On the other hand, the object detection models pre-trained on public datasets, which have the ability to detect various types of objects, cannot be directly applied to vehicle detection in specific scenarios. The vehicle detection models for real-world scenarios often rely on labeled data for training. Although there are currently some publicly available labeled vehicle detection datasets for training vehicle detection models, performance degradation will also occur when they are transferred to the real environment, and it needs to be solved through domain adaptation methods. Radar data has relatively accurate information such as the vehicle position and type, while the position of the vehicle in the original camera device is unlabeled, that is, there is a radar modality with rich information and a surveillance image modality with poor information.

[0004] The detection performance of object detection models is easily affected by factors such as weather and environment. For example, under adverse weather conditions, such as heavy rain, heavy fog, etc., camera devices cannot extract accurate visual information, resulting in the object detection model being unable to accurately detect target vehicles. Object detection models based on deep learning require a large amount of labeled data for training to achieve good detection effects. However, the human and time costs of labeled data are huge. Therefore, how to use limited labeled data to train object detection models with good detection performance is an important issue.

[0005] The reliability of image data and other multi-modal data obtained in extreme environments decreases. Especially for image data, the image data obtained in weather such as heavy rain and heavy fog has more noise. In the case of more interfering pixels, it is difficult to accurately identify the target object, or the target object may be misidentified. Some objects may be misidentified or missed. If the pixels of the picture are increased, the calculation time is greatly increased. The illumination range of high beam headlights is usually a relatively flat range, which can be divided into multiple square areas, and the frequencies of different objects appearing in different positions also vary. For example, there are usually oncoming vehicles in the left front. In such a case, the risk of turning on the high beam headlights is relatively high. Therefore, it is necessary to quickly identify and turn off the high beam headlights. There are usually pedestrians on the right side of urban roads. High beam headlights can also cause discomfort to pedestrians. Especially when the face is directly facing the high beam headlights, the eyes may not be able to open due to glare.

[0006] The invention patent with the Chinese patent application number "2023101654488" and the patent name "An ADB Headlight Control Method and System" proposes an adaptive headlight control method and system. However, the detection of target vehicles based on camera devices is easily affected by factors such as weather and environment, and the obtained pictures have more noise, resulting in the object detection model being unable to extract accurate visual information and the problem of decreased detection performance. In addition, it does not consider the training efficiency problem and cannot perform fast deep learning.

[0007] The invention patent with the Chinese patent application number "2021104750488" and the patent name "A Multi-modal Small Target Detection Method Based on YOLOv5" proposes a multi-modal small target detection method, which performs multi-modal fusion on the results detected in the visible light modality and the infrared modality. Although it can have good robustness in extreme weather and environments, the defect of decreased accuracy in identifying picture information due to unclear pictures in extreme weather has not been solved.

[0008] The invention patent with the Chinese patent application number "2022115975969" and the patent name "Target detection method based on millimeter-wave radar and visible light image fusion" proposes a target detection method for image fusion, which improves the accuracy at the cost of partial recall rate. However, it has not solved the problem of how to accurately identify various objects in the picture after the picture quality deteriorates in extreme weather such as heavy rain and fog.

[0009] The invention patent with the Chinese patent application number "2024111749525" and the patent name "An automobile lighting management method, device, storage medium and equipment" proposes to judge the vehicle's environment through various modal perception data, which can effectively respond to changes in road conditions and improve driving safety. And a large model is used to perform in-depth semantic analysis on the perception data, so as to be able to generate more intelligent lighting control strategies. However, due to the large amount of data processed, the response time from obtaining data information to controlling the automobile lights is long, and it cannot quickly detect oncoming vehicles and thus quickly turn off the corresponding headlight modules. The problem of how to accurately identify various objects in the picture when the clarity of the picture deteriorates in extreme weather has not been well solved under the premise of ensuring efficiency. Summary of the Invention

[0010] The present disclosure provides an ADB headlight control method and system to at least solve the problem of decreased recognition accuracy existing in the prior art under extreme weather conditions.

[0011] According to the first aspect of the present disclosure, there is provided an ADB headlight control method, which at least includes the following steps:

[0012] Obtain information data for controlling the headlights, including image data obtained by a camera device and radar data obtained by a radar;

[0013] Separate the information data to obtain at least first data and second data, where the first data corresponds to the first illumination range of the headlights, and the second data corresponds to the second illumination range of the headlights;

[0014] Input the first data and the second data into a pre-trained first model and a second model respectively. The first model is used to identify the first object information within the first illumination range, and the second model is used to identify the second object information within the second illumination range;

[0015] According to the obtained first object information or second object information, turn off or block the LED lights in the headlights that illuminate the corresponding positions.

[0016] Compared with the prior art, the ADB headlight control method of the present disclosure has the following beneficial effects:

[0017] The technical solution of this application preprocesses image data or multi-modal data, separates the data of different regions. For example, high-resolution pictures are segmented according to regions, or different imaging devices are set for different regions, and the algorithm model is also optimized accordingly for that region. Different regions mainly identify different objects. For example, if there are many pedestrians in this region, the accuracy of pedestrian detection should be improved, that is, more face pictures are input during training. If there are many vehicles in this region, the detection ability of vehicle recognition is improved, and there is also a bias during training to accurately identify different types of vehicles. The prior art uniformly processes multi-modal data, which not only has more noise but also a large amount of information. The technical solution of this application preprocesses the information, can remove a large amount of interfering information, and finally identifies certain information for a certain region, improving the detection efficiency and accuracy, that is, having a better detection effect under extreme conditions.

[0018] In an implementable embodiment, the first irradiation range is located in front of the left side of the controlled vehicle, and the second irradiation range is located in front of the right side of the controlled vehicle. After the first model is trained relative to the second model, it can identify the position information of other vehicles in the first irradiation range faster and / or more accurately, and turn off or block the LED lights in the matrix light group that irradiate the corresponding position according to the position information of other vehicles. After the second model is trained relative to the first model, it can identify the position information of a face or a pedestrian in the second irradiation range faster and / or more accurately, and turn off or block the LED lights in the matrix light group that irradiate the corresponding position. The LED lights of the matrix light group are divided into a sensitive area and a non-sensitive area. The LED lights in the sensitive area can irradiate the area in front of the left side of the controlled vehicle, and oncoming vehicles, that is, reverse vehicles, often appear in this area. Therefore, there is a greater risk of turning on the high beam lights. Therefore, vehicles should be quickly detected in this area and the corresponding high beam lights should be turned off. Pedestrians mainly appear on the right front side, and the detection accuracy can be improved, and the response speed can be slower.

[0019] In an implementable embodiment, the information data for controlling the vehicle lights is multi-modal information data, and the multi-modal information data at least includes image data obtained through an imaging device and modal data obtained through a radar. Devices such as lidar, infrared radar, and ultrasonic radar are less affected by weather and environment, while imaging devices have different recognition effects in different environments and may even fail in extreme environments. Therefore, the multi-modal method can make up for each other's advantages.

[0020] In an implementable embodiment, the vehicle light is a matrix headlight including a plurality of LED lamp beads, or an ADB headlight, or an LED pixel light, so that the irradiation range of the vehicle light can be accurately controlled.

[0021] In an implementable embodiment, the first model is based on the YOLOv5 model. First, the YOLOv5 model is trained on the source domain using image data with vehicle annotations. When migrating to the target domain, the correspondence between the position information of other vehicles in the radar data and the image data obtained by the camera device is utilized to generate high-confidence pseudo-labels for the image data in the target domain, and the source domain dataset is further incrementally trained, thereby improving the detection performance of the model in the target domain. The YOLOv5 model has a fast calculation speed and can quickly identify vehicle information.

[0022] In an implementable embodiment, the image data in the source domain dataset has labels, denoted as , where represents the set of object labels in the source domain image data , represents the bounding box label of the object in the image data, represents the object category label corresponding to the bounding box,

[0023] Based on the YOLOv5 model, the source domain dataset is supervised-trained using the following loss function:

[0024]

[0025] represents the CIOU loss, and its calculation method is shown as follows:

[0026]

[0027]

[0028]

[0029] Among them, IOU represents the intersection over union of the predicted target box and the ground truth box, represents the Euclidean distance between the centers of the predicted target box and the ground truth box, represents the diagonal distance of the smallest closed region that can simultaneously contain the predicted box and the target box, represents the aspect ratio difference between the predicted target box and the ground truth box, represents 's weight, and represent the aspect ratios of the ground truth box and the target box respectively, represents the object detection model used,

[0030] represents the operation of binary cross-entropy loss on the classification probability and the object prediction probability, 's calculation is shown as follows:

[0031]

[0032]

[0033] Among them, represents the IOU of the predicted box and the ground truth box, is the predicted object confidence obtained through the Sigmoid function, where represents the result directly output by the model. Using this algorithm can improve the detection accuracy after training.

[0034] In an implementable manner, the second model can exchange the spatial features of the visible light image data obtained through the imaging device and the modality data obtained through the radar, realizing the complementary integration of the bimodal features, thereby enhancing the visual characteristics and detection performance, and improving the detection stability in different environments.

[0035] In an implementable manner, the training process of the second model is as follows: obtain the input of the image data , and the input of the radar data , connect their outputs through the convolutional layer and represent it as , and its calculation formula is as follows:

[0036]

[0037] Among them, 3×3 represents the size of the convolutional kernel, and Concat includes concatenation and convolution operations;

[0038] Obtain the resulting feature map divided into two parts: and , where represents the image data features converted from the radar data, represents the radar data features derived from the image data, convert the radar data features into visible light image data features through linear transformation and convert the visible light image data features into radar data features through linear transformation , as shown in the following formula:

[0039]

[0040]

[0041] Then calculate the value based on the neural network function:

[0042]

[0043] Among them, are respectively specified for calculation The value of the neural network function, respectively representing the weight parameters in the neural network, applying convolutional operations and activation functions to learn the parameters:

[0044]

[0045] Finally, the aligned and modulated features of each modality are obtained as follows:

[0046]

[0047] where: ☉ represents the Hadamard product, θ1 is the alignment and modulation parameter of the image data features, and θ2 is the alignment and modulation parameter of the radar data features, , respectively representing the image data features and radar data features after feature alignment.

[0048] In one implementable manner, first data for controlling the vehicle lamp is obtained, and it is detected and analyzed to determine whether there is a moving object within the first irradiation range;

[0049] If there is, the first model is preferentially executed to calculate and determine the position information of other vehicles, and then the second model is executed to calculate and determine the position information of the face or the position information of the pedestrian;

[0050] If not, the second model is preferentially executed to calculate and determine the position information of the face or the position information of the pedestrian, and then the first model is executed to calculate and determine the position information of other vehicles.

[0051] In one implementable manner, the first model adopts a one-stage model algorithm, and the second model adopts a two-stage model algorithm. The one-stage model algorithm is fast, and the two-stage model algorithm has high accuracy.

[0052] In one implementable manner, after the first model identifies the vehicle information, it further identifies the window position information, and closes or shields the corresponding position LED lights according to the window position information. Preventing high beam irradiation mainly means avoiding the eyes of the driver. When the distance is far, it is sufficient to identify the vehicle. When the distance is close, the detection accuracy needs to be improved, that is, the non-window position will not cause interference to the driver, so the regulation of the vehicle lamp can be not carried out. In addition, after the window is identified, face recognition can be carried out, so that the regulation of the vehicle lamp can be more accurate. Especially for some vehicles parked by the roadside, there is basically no driver, and there is no need to turn off the vehicle lamp.

[0053] According to the second aspect of the present disclosure, an ADB vehicle lamp control system is provided, including:

[0054] The main controller module is capable of exchanging information with the vehicle computer control module. The vehicle computer control module can obtain vehicle driving status and environmental information from sensors. The sensors at least include a vehicle speed sensor, a steering sensor, a photosensitive sensor, a rain sensor, an inclination sensor, and a vehicle positioning sensor;

[0055] The information acquisition module at least includes a camera device and a radar. The camera device can obtain image data of visible light, and the radar can obtain radar data;

[0056] The LED lamp group can be controlled by the main controller module to turn off or shield the lights in part of the irradiation range. The LED lamp group has a first irradiation range and a second irradiation range;

[0057] The first data is input into the first model of the main controller module for calculation, and the second data is input into the second model of the main controller module for calculation;

[0058] Separate the data obtained by the information acquisition module into first data and second data. The first irradiation range corresponds to the first data, and the second irradiation range corresponds to the second data. The main controller module controls the lights in the first irradiation range according to the calculation result of the input first data, and controls the lights in the second irradiation range according to the calculation result of the input second data. This can specifically improve the detection accuracy of different objects in the corresponding areas and reduce recognition errors on the premise of ensuring the reaction speed.

[0059] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] By referring to the drawings and reading the following detailed description, the above and other purposes, features, and advantages of the exemplary embodiments of the present disclosure will become easily understood. In the drawings, several embodiments of the present disclosure are shown in an exemplary rather than restrictive manner, where:

[0061] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0062] Figure 1 Shows a schematic diagram of the implementation process of the ADB headlight control method according to an embodiment of the present disclosure;

[0063] Figure 2 Shows a schematic diagram of the main acquisition ranges of the first data and the second data when the vehicle speed is lower than 40 km / h in the ADB headlight control method according to an embodiment of the present disclosure;

[0064] Figure 3Shows the schematic diagram of the main acquisition ranges of the first data and the second data of the ADB headlight control method according to the embodiments of the present disclosure at a vehicle speed of 60 km / h;

[0065] Figure 4 Shows the schematic diagram of the main acquisition ranges of the first data and the second data of the ADB headlight control method according to the embodiments of the present disclosure at a vehicle speed higher than 100 km / h;

[0066] Figure 5 Shows the schematic diagram of the division of the lamp bead areas when the matrix lamp group of the ADB headlight control method according to the embodiments of the present disclosure is driving normally;

[0067] Figure 6 Shows the schematic diagram of the division of the lamp bead areas when the matrix lamp group of the ADB headlight control method according to the embodiments of the present disclosure is turning;

[0068] Figure 7 Shows the schematic diagram of the division of the lamp bead areas when the matrix lamp group of the ADB headlight control method according to the embodiments of the present disclosure is tilting downwards;

[0069] Figure 8 Shows the schematic diagram of the division of the lamp bead areas when the matrix lamp group of the ADB headlight control method according to the embodiments of the present disclosure is tilting upwards;

[0070] Figure 9 Shows the schematic diagram of the structure of the ADB headlight control system according to the embodiments of the present disclosure;

[0071] Figure 10 Shows the schematic diagram of the modal alignment of the image data and the radar data of visible light according to the embodiments of the present disclosure;

[0072] Figure 11 Shows the comparison experiment result graph of the ADB headlight control method according to the embodiments of the present disclosure relative to other models. Detailed implementation manners

[0073] To make the objectives, features, and advantages of the present disclosure more obvious and understandable, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present disclosure.

[0074] Refer to Figure 1 , an ADB headlight control method, at least including the following steps:

[0075] Obtain information data for controlling vehicle lights, including at least image data obtained by a camera device and radar data obtained by radar; it can also be data obtained by other sensors. Visible light image data can be obtained by a camera device, and invisible light image data requires converting the data obtained by radar.

[0076] Separate the information data to obtain at least a first data and a second data. The first data corresponds to the first illumination range of the vehicle lights, and the second data corresponds to the second illumination range of the vehicle lights; there may be a partially overlapping area between the first illumination range and the second illumination range, or the first illumination range belongs to a part of the second illumination range. The first data and the second data may also be partially the same, or the second illumination range belongs to a part of the first illumination range. Refer to Figure 2 , this figure is from the driver's driving perspective, which is basically the same as the perspective of the camera. The range of the double-dashed curve is the illumination range of the LED light group. A first camera device and a second camera device can be set. The first camera device obtains the first data or a part of the first data of the first illumination range, that is, the position of the first frame K1 on the left side in the figure is in the front left of the vehicle to be controlled. The second camera device obtains the second data or a part of the second data of the second illumination range, that is, the position of the second frame K2 on the right side in the figure. To ensure the recognition of different objects at each position, the first frame K1 and the second frame K2 can have a partially overlapping area, which can avoid missed detection.

[0077] As another implementation, only one camera device can be set. After obtaining the image data, it is cut. That is, the position of the first frame K1 in each image is cut out as a separate picture and becomes a part of the first data. The position of the second frame K2 is also cut out as a separate picture and becomes a part of the second data, and then calculation and recognition are performed.

[0078] Input the obtained first data into a trained first model, and input the obtained second data into a trained second model. After training, the first model can recognize the first object information in the first illumination range faster and more accurately compared to the second model, and the second model can recognize the second object information in the second illumination range faster and more accurately compared to the first model. The second data can also be input into the first model to recognize the first object information, and the first data can also be input into the second model to recognize the second object information. Refer to Figure 2, the first object information can be other vehicle position information, such as the position information of cars C1 and C2 in the figure. The second object information can be face position information or pedestrian position information, such as pedestrian R1 or face L1 in the figure. When pedestrian R1 is at a relatively far distance, face L1 may not be recognized. When pedestrian R1 is at a relatively close distance, face L1 recognition is relatively easier than pedestrian R1 recognition, and the calculation running speed is also faster. Therefore, features at different distances can be recognized at different vehicle speeds. The image range of the camera device at different vehicle speeds can also be set differently. When the vehicle speed is below 40 km / h, different objects within a larger area range can be recognized, that is, refer to Figure 2 , if the vehicle speed exceeds 40 km / h, the acquisition ranges of the first data and the second data can be reduced. Detecting objects relatively close to the position of the vehicle to be controlled is of little significance because the frequency of headlight control will not be too high. Frequent flashing of the headlights not only affects the lifespan of the headlights but also interferes with the line of sight. Suppose the time interval for controlling the headlights is 1 second. When the vehicle travels more than ten meters in one second, detecting objects within ten meters of the vehicle is of little significance because when it is detected that the headlights need to be controlled, the vehicle may have already driven out of the headlight illumination range.

[0079] Refer to Figure 3 , when the vehicle speed is between 40 km / h and 80 km / h, for example, when it is 60 km / h, the acquisition range of the first data can be reduced to the image range more than 20 meters ahead, that is, the first middle frame K11 in the figure. Generally, the algorithm calculates square pictures. Therefore, multiple square first middle frames K11 can be set as needed to achieve a larger range of detection. The second data can be within the range of the second middle frame K22. In this way, the focal length of the camera device can be adjusted to achieve the clarity of more distant objects. Refer to Figure 4 , when the vehicle speed exceeds 100 km / h, the first data and the second data can only acquire a smaller range more than 30 meters directly ahead. In this way, the focal length of the camera device can be further adjusted. Of course, the minimum controllable range of the headlights is limited, and it is also of little significance to recognize objects more than 200 meters away. Therefore, it is sufficient to be able to recognize objects about 100 meters away, that is, the positions of the first small frame K12 and the second small frame K23 in the figure are more than 30 meters directly ahead. Generally, there are no pedestrians or very few pedestrians when driving at high speed. Therefore, the image data of the first small frame K12 and the second small frame K13 obtained can be considered as the first data, or can be input into the first model for calculation first, and then input into the second model for calculation. In addition, according to needs, data outside the ranges of the first small frame K12 and the second small frame K13 can be input into the third model for calculation. The third model can be used as a supplement to the first model and the second model.

[0080] In one embodiment, the vehicle lamp is a matrix headlamp including multiple LED lamp beads. The ADB headlamp can also be a DLP headlamp or an LED pixel lamp. An LED pixel lamp means that the headlamp is composed of multiple light-emitting units and can form an imaging illumination effect similar to an image, improving driving safety and visibility. Generally, a matrix headlamp has only dozens or hundreds of lamp beads, while an LED pixel lamp can have tens of thousands of light-emitting units. DLP, the full name is Digital Light Processing, which means digital light processing. Simply put, it is to first digitally process the video signal and then project it. This technology mainly relies on the digital micromirror device - DMD (Digital Micromirror Device) developed by Texas Instruments (TI) to achieve. Simply put, the core of the DLP headlamp is the digital micromirror chip, and digital optical processing is achieved through it.

[0081] According to the obtained position information of other vehicles or the position information of faces or the position information of pedestrians, turn off or shield the LED lamps in the matrix lamp group that illuminate the corresponding positions. The LED matrix lamp group 2 is provided with multiple lamp beads 1, and different LED lamp beads 1 have different illumination ranges. For example, Figure 5 As shown, multiple rows and columns of LED lamp beads 1 can be set. Different LED lamp beads 1 have different illumination ranges. The principles of the left and right vehicle lamps are basically the same. Taking the left vehicle lamp as an example only, the left vehicle lamp is provided with a first lamp bead area A1 that provides illumination for the first illumination range, a second lamp bead area A3 that provides illumination for the second illumination range, and a third lamp bead area A2 that provides illumination for a relatively short distance.

[0082] As Figure 6 shown, when the vehicle is turning, the non-driving direction in front of the vehicle is the front of the vehicle. The first illumination range and the second illumination range also need to be adjusted. For vehicle lamps that can follow the adjustment when the vehicle is turning, if turning right, the lamp beads 1 in the first lamp bead area A1 also deflect to the right, so that the lamp beads that can provide illumination for the first illumination range will decrease, and the first lamp bead area A1 will correspondingly become smaller. For vehicle lamps that cannot follow the adjustment when the vehicle is turning, if the vehicle turns left, the lamp beads that can provide illumination for the first illumination range will decrease, and the first lamp bead area A1 will correspondingly become smaller, that is, the number of columns will decrease.

[0083] As Figure 7 shown, when the vehicle is tilted downward, the first lamp bead area A1 will correspondingly become smaller, that is, the number of rows will decrease. As Figure 8 shown, when the vehicle is tilted upward, the first lamp bead area A1 will correspondingly become larger, that is, the number of rows will increase. That is, the data obtained and the adjustment range of the vehicle in different driving states are also different.

[0084] In one embodiment, the information data for controlling the vehicle lamp is multi-modal information data, and the multi-modal information data at least includes image data obtained through a camera device and modal data obtained through a radar.

[0085] In one embodiment, the first model is based on the YOLOv5 model. The first model mainly identifies vehicle position information. First, the YOLOv5 model is trained using image data with vehicle annotations in the source domain. Of course, a small amount of image data with pedestrian annotations can also be added for training. Since the main purpose is to identify vehicles, a large number of pictures of different vehicles are required, including sedans, SUVs, sports cars, buses, trucks, trailers, and other functional vehicles. When migrating to the target domain, the corresponding relationship between the position information of other vehicles in the radar data and the image data obtained by the imaging device is used to generate high-confidence pseudo-labels for the image data in the target domain, with the confidence range being 95% - 99%. Further, incremental training is performed on the source domain model to improve the detection performance of the model in the target domain.

[0086] In one embodiment, the image data in the source domain dataset has labels, denoted as , where represents the set of object labels in the source domain image data , represents the bounding box label of the object in the image data, represents the object category label corresponding to the bounding box,

[0087] Based on the YOLOv5 model, supervised training is performed on the source domain dataset , and the loss function used is as follows:

[0088]

[0089] represents the CIOU loss, and its calculation method is shown in the following formula:

[0090]

[0091]

[0092]

[0093] Among them, IOU represents the intersection over union of the predicted target box and the ground truth box, represents the Euclidean distance between the centers of the predicted target box and the ground truth box, represents the diagonal distance of the smallest closed region that can contain both the predicted box and the target box, represents the difference in aspect ratio between the predicted target box and the ground truth box, represents 's weight, and respectively represent the aspect ratios of the ground truth box and the target box, represents the object detection model used,

[0094] It represents an operation for binary cross - entropy loss on classification probability and object prediction probability, and its calculation is shown as follows:

[0095]

[0096]

[0097] where, represents the IOU of the predicted box and the ground - truth box, is the predicted object confidence obtained through the Sigmoid function, where represents the result directly output by the model.

[0098] In one embodiment, the second model can exchange the spatial features of the visible - light image data obtained by the imaging device and the modality data obtained by the radar, realizing the complementary integration of bimodal features, thereby enhancing the visual characteristics and detection performance. The second model mainly identifies the face position information and pedestrian position information. Of course, a small number of rear - view pictures of vehicles can also be input for training, so that it can identify the vehicles driving in the same direction and avoid affecting the driver of the vehicle in front to view the rear - view mirror.

[0099] As Figure 10 shown, in one embodiment, the training process of the second model is as follows: obtain the input of the image data , and the input of the radar data , connect their outputs through the convolutional layer and represent it as , and its calculation formula is as follows:

[0100]

[0101] where, 3×3 represents the size of the convolutional kernel, and Concat includes concatenation and convolution operations;

[0102] Obtain the result feature map divided into two parts: and where, represents the image - data feature converted from the radar data, represents the radar - data feature derived from the image data, convert the radar - data feature into the visible - light image - data feature through linear transformation and convert the visible - light image - data feature into the radar - data feature through linear transformation , as shown in the following formula:

[0103]

[0104]

[0105] Then calculate the values based on the neural network function :

[0106]

[0107] Among them, are respectively the neural network functions specified for calculating values, respectively represent the weight parameters in the neural network, and apply convolutional operations and activation functions to learn the parameters:

[0108]

[0109] Finally, the alignment and modulation features of each modality are obtained as follows:

[0110]

[0111] Among them: ☉ represents the Hadamard product, θ1 is the alignment and modulation parameter of the image data features, θ2 is the alignment and modulation parameter of the radar data features, , respectively represent the image data features and radar data features after feature alignment.

[0112] In one embodiment, multi-modal information data for controlling vehicle lights is obtained, and it is detected, analyzed, and judged whether there is a moving object within the first irradiation range;

[0113] If there is, first execute the first model calculation to determine the position information of other vehicles, that is, whether there is a vehicle and the position of the vehicle, and then execute the second model calculation to determine the face position information or pedestrian position information, that is, whether there is a face and the position of the face;

[0114] If not, first execute the second model calculation to determine the face position information or pedestrian position information, that is, whether there is a face and the position of the face, and then execute the first model calculation to determine the position information of other vehicles, that is, whether there is a vehicle and the position of the vehicle.

[0115] In one embodiment, after detecting the position information of other vehicles, detect and predict their movement trajectories, that is, predict the position of the vehicle after a certain period of time, such as one second later, and control the vehicle lights in advance according to the predicted trajectory, so that the lighting effect of the vehicle lights can be adjusted in the first time to make it more ideal.

[0116] In one embodiment, a third model can be trained. The third model can identify information such as the position information of other vehicles, the position information of faces, and the position information of pedestrians. That is, when there are no moving objects, the first data can be input into the third model for identification without having to execute the algorithm a second time. The third model can directly adopt the identification solutions of existing technologies, mainly to identify more and more comprehensive information. The first model is mainly to improve the reaction speed so that the vehicle lights can be controlled in the first instance and the identification accuracy of vehicles can be improved. The control of the vehicle lights is in a cyclic state with a certain interval period. Frequent turning off and on of the vehicle lights will also affect oncoming vehicles. That is to say, there is enough time within one cycle to identify different objects, but a fast reaction speed is required at the start of the cycle.

[0117] It is possible to directly detect and analyze whether there are moving objects through a radar. If the vehicle is not equipped with a radar, it is also possible to analyze through images. The detection accuracy of using deep learning algorithms is high, but it is time-consuming. For example, YOLOv8 can be used for detection, and OpenCV can also be used for motion detection.

[0118] In one embodiment, the first model adopts a one-stage model algorithm, and the second model adopts a two-stage model algorithm. Two-stage model algorithms include R-CNN, Fast R-CNN, Faster R-CNN, etc., and one-stage model algorithms include YOLO, SSD, etc. One-stage model algorithms are fast, and two-stage model algorithms have high accuracy. For face recognition, a higher accuracy is required. In the ADB vehicle light control logic, the detection of target objects of vehicles and pedestrians needs to be carried out separately, mainly because there are significant differences between vehicles and pedestrians in terms of size, shape, motion characteristics, etc. Separated detection can more accurately identify and locate these targets, thereby achieving more refined light control. For example, vehicles are usually much larger than pedestrians and have more regular motion trajectories, while pedestrians may suddenly appear or move quickly on the road. Through separate detection, the ADB system can more precisely adjust the lights to avoid dazzling vehicle drivers or pedestrians.

[0119] In one embodiment, the LED lights of the matrix light group can be divided into a sensitive area and a non-sensitive area. The LED lights in the sensitive area can illuminate and control the left front area of the vehicle. The multi-modal information data obtained is segmented, and the multi-modal information data obtained by controlling the left front area of the vehicle is separately segmented and then analyzed and calculated using the first model. The control period of the vehicle lights in the sensitive area is 0.5 seconds to 1 second, that is, it is necessary to keep on or off for 0.5 seconds to 1 second before the state can be changed, so as to avoid frequent flashing of the vehicle lights affecting the line of sight and easily causing visual fatigue. The non-sensitive area is mainly in the relatively close area and the right area. The control period of the non-sensitive area can be set between 1 second and 3 seconds, or the off period is longer than the on period, that is, it can only be turned on after 3 seconds of turning off, but it can be turned off after 1 second of turning on.

[0120] In one embodiment, after the first model identifies vehicle information, it further identifies window position information and closes or shields the LED lights at the corresponding positions according to the window position information. This is mainly for vehicles equipped with a multi-row and multi-column matrix of LED lights, where the beads at different heights can be turned off or shielded. The vehicle lights shining on the vehicle body will not affect other people, but if they shine on the window, they may generally interfere with the people inside the vehicle. However, it is very difficult to determine whether there is a face inside the window. Therefore, directly identify the window and close the corresponding vehicle lights when the window is identified.

[0121] In one embodiment, as a supplementary recognition method, in good lighting conditions, face recognition is performed after window recognition. If a face can be recognized, it can be selectively avoided; if a face cannot be recognized, it is completely avoided.

[0122] In one embodiment, in extreme weather conditions, the camera may be contaminated by foreign objects, water droplets, dust, etc. The generated images will all have stains, and the stains need to be removed. On the one hand, for the stains, trained models such as YOLO can be used for recognition. That is, the labeled pictures of multiple pictures with stains are input into the model for training, and then the stains are recognized for all the obtained images. After recognition, the stains are segmented and rendered, or directly input into the training model for feature recognition after removing the stains. On the other hand, the images obtained at a certain time before and after can be compared. If a certain position always has the same feature object, and if the feature object and the connected area nearby still cannot recognize the feature object, it can be removed as a stain.

[0123] Reference Figure 9 , there is provided an ADB vehicle lamp control system, including:

[0124] A main controller module capable of exchanging information with the vehicle computer control module, and the vehicle computer control module can obtain vehicle driving status and environmental information from sensors. The sensors at least include a vehicle speed sensor, a steering sensor, a photosensitive sensor, a rain sensor, an inclination sensor, and a vehicle positioning sensor;

[0125] An information acquisition module at least including a camera device and a radar. The camera device can obtain image data of visible light, and the radar can obtain radar data;

[0126] An LED lamp group, which can be controlled by the main controller module to turn off or shield the lights in part of the irradiation range. The LED lamp group has a first irradiation range and a second irradiation range;

[0127] Separate the data obtained by the information acquisition module into first data and second data. The first irradiation range corresponds to the first data, and the second irradiation range corresponds to the second data. The first data is input into the first model of the main controller module for calculation, and the second data is input into the second model of the main controller module for calculation;

[0128] The main controller module controls the lights in the first irradiation range according to the calculation result of the input first data, and controls the lights in the second irradiation range according to the calculation result of the input second data.

[0129] In some embodiments, a third model or more other models can be set, and different models analyze and identify different data in a targeted manner.

[0130] The vehicle speed sensor can detect the vehicle speed. When the vehicle is traveling fast, the area range that the information acquisition module needs to identify decreases, which can improve the recognition accuracy and speed of key positions in the distance. For the near distance, since the irradiation time is short, it can be ignored, or other algorithms with higher precision can be used. When the vehicle speed is slow, the detection range is large. The near distance can be detected first, and then the focal length can be changed to obtain a clear image of the distance and then detect the distance, or multiple cameras can be used to obtain images of different distances.

[0131] The steering sensor can obtain the wheel steering angle of the vehicle. In a left-hand drive vehicle, the steering angle of the left front wheel is closest to the movement direction of the driver's position, and can be considered as the movement direction of the vehicle. The headlights should theoretically also be adjusted following the rotation direction of the left front wheel. Therefore, the acquisition ranges of the first data and the second data also need to be adjusted. If the visible light image data is obtained by one camera, then the image cutting range can be changed. For example, when the vehicle turns left, there are basically no oncoming vehicles in the front, so the cutting area needs to be moved a certain distance to the left. The data ranges of radar, etc. can also be adjusted. The actual position of the driver is at a certain distance from the left front wheel, so there is also a difference between the movement direction of the driver's position and the left front wheel. The corresponding movement direction angle can be calculated according to the distance and angle between the driver and the front and rear wheels, so that the headlights can always maintain the same direction as the driver's movement direction, and the obtained data can also always maintain a certain relative position.

[0132] According to the information of the photosensitive sensor, the headlights can be automatically turned off in sufficient light and turned on in insufficient light. According to the rain sensor, a special mode can be enabled, especially for the coping methods when the image clarity decreases. For example, when the picture is clear, the third model is used for recognition, and when the picture is not clear, the first model is used for recognition. The first model uses pictures obtained in more different weathers during training, while the third model obtains conventional night pictures. In addition, the angle information obtained by the tilt sensor can be used to adjust the information acquisition range of the vehicle, and the vehicle positioning sensor can be used to obtain the position information, and then the data provided by the navigation can be used for comprehensive analysis to control the headlights.

[0133] The LED lights that illuminate the corresponding positions in the matrix light group can be turned off by cutting off the power supply or by shielding, so that the light intensity at the corresponding positions is reduced. Generally speaking, the frequency of cutting off the power supply cannot be too high, and the frequency of the shielding method can be a little higher, but it should not exceed 0.5 seconds. Otherwise, frequent flashing is also not conducive to driving safety.

[0134] As Figure 11 shown, the average precision (AP) and frames per second (FPS) are used to evaluate the deployment ability of the method of this application, where the average precision (AP) uses the AP with a threshold of 0.5 50 as the average precision index. Other common object detection algorithms are compared as reference models, including YOLOv3, YOLOv5, Mask R-CNN, and YOLOv7, and the results as Figure 10 shown are obtained. The algorithm of this application integrates YOLOv5 and is superior to other models in terms of AP 50 and combines image and radar data, with the FPS reaching 126 frames per second.

[0135] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a definable sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or used in combination with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in combination with an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer diskette case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, a computer-readable medium can even be paper or other suitable media on which a program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or otherwise appropriately processing it if necessary, and then storing it in a computer memory. Among them, the memory can include a mass storage for data or instructions. By way of example and not limitation, the memory can include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disc, a magneto-optical disc, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In appropriate cases, the memory can include removable or non-removable (or fixed) media. In appropriate cases, the memory can be internal or external to the data processing device. In a specific embodiment, the memory is non-volatile memory. In a specific embodiment, the memory includes a read-only memory (ROM) and a random access memory (RAM).Where appropriate, the ROM can be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, simply referred to as PROM), an erasable PROM (Erasable Programmable Read-Only Memory, simply referred to as EPROM), an electrically erasable PROM (Electrically Erasable Programmable Read-Only Memory, simply referred to as EEPROM), an electrically alterable ROM (Electrically Alterable Read-Only Memory, simply referred to as EAROM), or a FLASH memory, or a combination of two or more of these. Where appropriate, the RAM can be a static random access memory (Static Random-Access Memory, simply referred to as SRAM) or a dynamic random access memory (Dynamic Random Access Memory, simply referred to as DRAM), where the DRAM can be a fast page mode dynamic random access memory (Fast Page Mode Dynamic Random Access Memory, simply referred to as FPMDRAM), an extended data output dynamic random access memory (Extended Date Out Dynamic Random Access Memory, simply referred to as EDODRAM), a synchronous dynamic random access memory (Synchronous Dynamic Random-Access Memory, referred to as SDRAM), etc.

[0136] It should be understood that various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits with logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits with appropriate combinational logic gate circuits, programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0137] It should be understood that the various forms of processes shown above can be used, with steps reordered, added or deleted. For example, the steps described in the present disclosure can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and no limitation is imposed herein.

[0138] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present disclosure can easily think of changes or substitutions, which should all be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

1. A method for controlling an ADB headlight, characterized in that, The method at least includes the following steps: Obtain information data for controlling vehicle lights, including image data obtained by a camera device and radar data obtained by radar; Obtain vehicle driving state information, which at least includes vehicle speed information or steering information; Separate the information data according to the vehicle driving state information to obtain at least first data and second data. The first data corresponds to a first illumination range of the vehicle lights, and the second data corresponds to a second illumination range of the vehicle lights. The first illumination range is located in the left front of the controlled vehicle, and the second illumination range is located in the right front of the controlled vehicle. The first illumination range and the second illumination range are adjusted according to vehicle speed information or steering information, and the faster the vehicle speed, the smaller the first illumination range and the second illumination range; Input the first data and the second data into a pre-trained first model and second model respectively. The first model is used to identify first object information within the first illumination range, and the second model is used to identify second object information within the second illumination range. The first object information is the position information of other vehicles, and the second object information is the position information of a human face or pedestrian position information; Turn off or shield the LED lights in the vehicle lights that illuminate the corresponding positions according to the obtained first object information or second object information.

2. The ADB vehicle light control method according to claim 1, wherein The controlled vehicle lights are a matrix headlight including multiple LED lamp beads; or an ADB headlight; or an LED pixel light.

3. The ADB vehicle light control method according to claim 2, wherein The first model is based on the YOLOv5 model. First, the YOLOv5 model is trained using image data with vehicle annotations in the source domain. When migrating to the target domain, the corresponding relationship between the position information of other vehicles in the radar data and the image data obtained by the camera device is used to generate target domain pseudo-labels for the image data in the target domain, thereby improving the detection performance of the first model in the target domain.

4. The ADB vehicle light control method according to claim 3, wherein The image data in the source domain dataset has labels, denoted as , where represents the set of object labels in the source domain image data , represents the bounding box label of the object in the image data, represents the object category label corresponding to the bounding box Based on the YOLOv5 model, for the source domain dataset perform supervised training with the following loss function: Indicates the CIOU loss, and the calculation method is as shown in the following formula: where IOU represents the intersection over union of the predicted target box and the ground truth box, represents the Euclidean distance between the centers of the predicted target box and the ground truth box, represents the diagonal distance of the smallest closed region that can contain both the predicted box and the target box, represents the aspect ratio difference between the predicted target box and the ground truth box, represents the weight of, and represent the aspect ratios of the ground truth box and the target box respectively, represents the object detection model used, Represents an operation for calculating the binary cross-entropy loss of the classification probability and the object prediction probability, The calculation is shown as follows: Among them, represents the IOU between the predicted box and the ground truth box, is the predicted target confidence obtained through the Sigmoid function, where represents the result directly output by the model.

5. The ADB vehicle light control method according to claim 1, wherein The second model can exchange the spatial features of the image data of visible light obtained by the camera device and the radar data obtained by the radar, realize the complementary integration of bimodal features, thereby enhancing the visual characteristics and detection performance.

6. The ADB headlight control method according to claim 5, characterized in that, The training process of the second model is as follows, including: Input for obtaining image data , and input for radar data , connect their outputs through a convolutional layer and represent as , and its calculation formula is as follows: Among them, 3×3 represents the size of the convolution kernel, and Concat includes concatenation and convolution operations; Obtain a result feature map divided into two parts: and , where represents the image data features obtained by converting radar data, represents the radar data features derived from the image data; Convert the radar data features into visible light image data features through linear transformation , convert the visible light image data features into radar data features through linear transformation , as shown in the following formula: Then calculate the value based on the neural network function : Among them, are respectively neural network functions specified for calculating values, respectively represent the weight parameters in the neural network, and apply convolution operations and activation functions to learn the parameters: Finally, the aligned and modulated features of each modality are obtained as follows: Where: ⊙ represents the Hadamard product, θ1 is the image data feature alignment and modulation parameter, and θ2 is the radar data feature alignment and modulation parameter. , respectively represent the image data features and radar data features after feature alignment.

7. The ADB vehicle light control method according to any one of claims 1-6, wherein Obtain the first data for controlling the vehicle lights, and detect whether there are moving objects within the first illumination range; If so, preferentially execute the first model calculation to determine the position information of other vehicles, and then execute the second model calculation to determine the position information of the human face or pedestrian position information; If not, the second model calculation is preferentially executed to determine the face position information or pedestrian position information, and then the first model calculation is executed to determine the position information of other vehicles.

8. The ADB headlight control method according to claim 7, wherein the first model adopts a one-stage model algorithm, and the second model adopts a two-stage model algorithm. After identifying the vehicle information, the first model further identifies the window position information and turns off or shields the LED lights at the corresponding positions according to the window position information.

9. An ADB headlight control system, characterized in that, It includes: a main controller module capable of exchanging information with a vehicle computer control module, and the vehicle computer control module can obtain vehicle driving states and environmental information from sensors; the vehicle driving state information at least includes vehicle speed information or steering information; an information acquisition module at least including a camera device and a radar, the camera device can obtain image data of visible light, and the radar can obtain radar data; an LED lamp group, controlled by the main controller module, can turn off or shield the lights in part of the irradiation range, and determine the first irradiation range and the second irradiation range of the LED lamp group according to the vehicle driving state; separate the data obtained by the information acquisition module into first data and second data, the first irradiation range corresponds to the first data, and the second irradiation range corresponds to the second data; the first irradiation range is located in the front left of the vehicle to be controlled, the second irradiation range is located in the front right of the vehicle to be controlled, the first irradiation range and the second irradiation range are adjusted according to the vehicle speed information or steering information, and the faster the vehicle speed, the smaller the first irradiation range and the second irradiation range; the first data is input into the first model of the main controller module for calculation, and the second data is input into the second model of the main controller module for calculation; the first model is used to identify the first object information within the first irradiation range, and the second model is used to identify the second object information within the second irradiation range, the first object information is the position information of other vehicles, and the second object information is the face position information or pedestrian position information; the main controller module controls the lights in the first irradiation range according to the calculation result of the input first data, and controls the lights in the second irradiation range according to the calculation result of the input second data.

Images