ADB vehicle lamp control method and system

Abstract

The invention relates to the technical field of vehicle lamp control, and provides an ADB vehicle lamp control method which at least comprises the following steps: acquiring information data for controlling a vehicle lamp, and at least comprising image data obtained by camera equipment; the information data is separated to at least obtain first data and second data, the obtained first data is input into a trained first model, the obtained second data is input into a trained second model, and the first model and the second model can identify first object information or second object information after being trained. And according to the obtained first object information or the second object information, turning off or shielding the LED lamps which irradiate the corresponding positions in the LED lamp group. According to the technical scheme, the information is preprocessed, a large amount of interference information can be removed, certain information is finally recognized for a certain area, the detection efficiency and precision are improved, and a better detection effect is achieved under the extreme condition.

Description

Technical Field

[0001] The present disclosure relates to the technical field of vehicle light control, and in particular to an ADB vehicle light control method and system. Background Art

[0002] At present, with the development of automotive electronic technology, the safety and comfort of automobiles are increasingly valued; ADB (Adaptive Driving Beam) technology is used in automobile headlight control. The light distribution control device is used to prevent the vehicle in front of the vehicle, the oncoming vehicle or the pedestrian in the front area of ​​the vehicle from being dazzled. On the other hand, it improves the observability of the front area, which can improve the safety during driving. At present, some adaptive high beam systems often only directly switch the high and low beams of motor vehicles, ignoring the limited field of vision of the car during the period of switching from high beam to low beam, which is easy to cause accidents. Therefore, it is necessary to improve the adaptive high beam system to improve driving safety. The existing target detection and re-identification model has strong detection and processing capabilities for data of a single visual modality, but in a real environment, the data that people can obtain are often multi-modal, and there is often complementary information between the multi-modalities. Fully exploring the relationship between the modalities will help improve the performance of the model. Combining multimodality with domain adaptation and unsupervised technology can more comprehensively utilize multi-source, multimodal information and massive unlabeled data in real environments to achieve more efficient detection and re-identification.

[0003] Although the general control model has achieved good results in the ADB control task of headlights, there are still some problems: on the one hand, since the camera equipment generally collects the camera equipment on the vehicle, it is an optical image and is easily affected by the natural environment. Complex environmental factors such as different lighting conditions and bad weather seriously affect the image quality in the video and interfere with vehicle target detection. Therefore, it also puts forward higher requirements on the practicality and robustness of the target detection algorithm. On the other hand, the target detection model pre-trained on the public data set has the ability to detect objects of multiple categories, but cannot be directly applied to vehicle detection in specific scenes. The vehicle detection model for real scenes often relies on labeled data for training. Although there are some public labeled vehicle detection data sets for vehicle detection model training, the performance will also decrease when migrating to the real environment, which needs to be solved by domain adaptation methods. Radar data has relatively accurate information such as vehicle location and type, while the location of the vehicle in the original camera device is unlabeled, that is, there is an information-rich radar modality and an information-poor monitoring image modality.

[0004] The detection performance of the target detection model is easily affected by factors such as weather and environment. For example, in bad weather conditions, such as heavy rain and fog, the camera equipment cannot extract accurate visual information, resulting in the target detection model being unable to accurately detect the target vehicle. The target detection model based on deep learning requires a large amount of labeled data for training to achieve good detection results. However, the manpower and time cost of labeling data is huge, so how to use limited labeled data to train a target detection model with good detection performance is an important issue.

[0005] The reliability of image data and other multimodal data obtained in extreme environments decreases, especially image data. Image data obtained in weather such as heavy rain and fog has more noise points. When there are more interfering pixels, it is difficult to accurately identify the target object, or the target object is incorrectly identified. Some objects may be misidentified or missed. If the pixels of the image are increased, the calculation time is greatly increased. The illumination range of the high beam headlights is usually a relatively flat range, which can be divided into multiple square areas, and the frequency of occurrence of different objects in different positions is also different. For example, there are usually vehicles traveling in the opposite direction in front of the left. In this case, the risk of turning on the high beam is higher, so it is necessary to quickly identify and turn off the high beam. There are usually pedestrians on the right side of urban roads, and the high beam will also cause discomfort to pedestrians, especially when the face is facing the high beam, they may not be able to open their eyes due to the glare.

[0006] The invention patent with Chinese patent application number "2023101654488" and patent name "A Method and System for ADB Headlight Control" proposes an adaptive headlight control method and system. However, the target vehicle detection based on the camera equipment is easily affected by factors such as weather and environment, and the obtained pictures have a lot of noise points, which leads to the inability of the target detection model to extract accurate visual information, resulting in a decrease in detection performance. In addition, it does not consider the training efficiency problem and cannot perform fast deep learning.

[0007] The invention patent with Chinese patent application number "2021104750488" and patent name "A multimodal small target detection method based on YOLOv5" proposes a multimodal small target detection method, which performs multimodal fusion of the detection results in the visible light mode and infrared mode. Although this can have good robustness in extreme weather and environment, the defect of reduced accuracy of image information recognition due to unclear images in extreme weather has not been solved.

[0008] The Chinese patent application number is "2022115975969", and the invention patent name is "Target detection method based on millimeter-wave radar and visible light image fusion". It proposes an image fusion target detection method, which improves the accuracy at the cost of partial recall. However, it does not solve the problem of how to accurately identify various objects in the picture after the image quality deteriorates in extreme weather conditions such as heavy rain and fog.

[0009] The invention patent with Chinese patent application number "2024111749525" and patent name "A method, device, storage medium and equipment for automobile lighting management" proposes to judge the vehicle's environment through multi-modal perception data, which can effectively respond to changes in road conditions and improve driving safety, and use large models to perform in-depth semantic analysis of perception data, so as to generate more intelligent lighting control strategies. However, due to the large amount of data processed, the reaction time from obtaining data information to controlling automobile lights is long, and oncoming vehicles cannot be detected quickly to quickly turn off the corresponding lighting module. Under the premise of ensuring efficiency, how to accurately identify various objects in the picture when the clarity of extreme weather pictures decreases has not been well solved. Summary of the invention

[0010] The present disclosure provides an ADB vehicle light control method and system to at least solve the problem of reduced recognition accuracy under extreme weather conditions in the prior art.

[0011] According to a first aspect of the present disclosure, there is provided an ADB vehicle light control method, comprising at least the following steps: Acquiring information data for controlling vehicle lights, including image data obtained by camera equipment and radar data obtained by radar; Separating the information data to obtain at least first data and second data, the first data corresponding to a first illumination range of the vehicle light, and the second data corresponding to a second illumination range of the vehicle light; Inputting the first data and the second data into the pre-trained first model and the second model respectively, the first model is used to identify the first object information within the first irradiation range, and the second model is used to identify the second object information within the second irradiation range; According to the obtained first object information or second object information, the LED light illuminating the corresponding position in the vehicle light is turned off or shielded.

[0012] Compared with the prior art, the ADB vehicle light control method disclosed in the present invention has the following beneficial effects: The technical solution of the present application pre-processes image data or multi-modal data, and separates data from different regions, such as dividing high-resolution images by region, or setting different camera equipment for different regions, and optimizing the algorithm model accordingly for the region. Different regions mainly identify different objects. For example, if there are many pedestrians in the region, then the accuracy of pedestrian detection should be improved, that is, more face images should be input during training. If there are many vehicles in the region, the detection capability of vehicle recognition should be improved, and there should be some bias during training, so that different types of vehicles can be accurately identified. The prior art processes multi-modal data uniformly, which not only has more noise points but also a large amount of information. The technical solution of the present application pre-processes the information, which can remove a large amount of interference information, and finally identifies certain information for a certain region, improving detection efficiency and accuracy, that is, having better detection effects under extreme conditions.

[0013] In one possible implementation, the first illumination range is located in the left front of the controlled vehicle, and the second illumination range is located in the right front of the controlled vehicle. After training, the first model can identify the position information of other vehicles in the first illumination range faster and / or more accurately than the second model, and turn off or shield the LED lights in the matrix light group that illuminate the corresponding positions according to the position information of other vehicles. After training, the second model can identify the position information of faces or pedestrians in the second illumination range faster and / or more accurately than the first model, and turn off or shield the LED lights in the matrix light group that illuminate the corresponding positions. The LED lights of the matrix light group are divided into sensitive areas and non-sensitive areas. The LED lights in the sensitive area can illuminate the left front area of ​​the controlled vehicle. Oncoming vehicles, i.e., reverse vehicles, often appear in this area, so there is a greater risk of turning on the high beam. Therefore, the area should quickly detect vehicles and turn off the corresponding high beam. Pedestrians mainly appear in the right front, which can improve the detection accuracy and the reaction speed can be slower.

[0014] In one possible implementation, the information data for controlling the vehicle lights is multimodal information data, which includes at least image data obtained by a camera and modal data obtained by a radar. Equipment such as laser radar, infrared radar, and ultrasonic radar are less affected by weather and environment, while camera equipment has different recognition effects in different environments and may even fail in extreme environments. Therefore, a multimodal approach can complement each other.

[0015] In one possible implementation, the vehicle lights include a plurality of LED lamp bead matrix headlights or ADB headlights or LED pixel lights, so that the illumination range of the vehicle lights can be accurately controlled.

[0016] In one possible implementation, the first model is based on the YOLOv5 model. The YOLOv5 model is first trained on the source domain using image data with vehicle annotations. When migrating to the target domain, the corresponding relationship between other vehicle position information in the radar data and image data obtained by the camera device is used to generate high-confidence target domain pseudo labels for the image data in the target domain, and the source domain data set is further incrementally trained to improve the detection performance of the model in the target domain. The YOLOv5 model has a fast calculation speed and can quickly identify vehicle information.

[0017] In one embodiment, the image data in the source domain dataset has a label, denoted as ,in Represents the source domain image data The set of object labels in Represents the bounding box labels of objects in the image data, Indicates the object category label corresponding to the bounding box, Based on the YOLOv5 model, the source domain dataset For supervised training, the loss function used is as follows:

[0018] Represents the CIOU loss, which is calculated as follows:

[0019]

[0020]

[0021] Among them, IOU represents the intersection-over-union ratio of the predicted target box and the real box. Represents the Euclidean distance between the center point of the predicted target box and the real box, Represents the minimum diagonal distance of the closure area that can contain both the prediction box and the target box. Represents the difference in aspect ratio between the predicted target box and the real box, express The weight of and Represent the aspect ratios of the real box and the target box respectively, represents the target detection model used, represents the operation of binary cross entropy loss on classification probability and object prediction probability, The calculation is as follows:

[0022]

[0023] in, Represents the IOU of the predicted box and the real box, is the predicted target confidence (or category confidence) obtained by the Sigmoid function, where Indicates the result directly output by the model. The algorithm can be trained to improve the detection accuracy.

[0024] In one embodiment, the second model can exchange spatial features of visible light image data obtained by a camera and modal data obtained by a radar, thereby achieving complementary integration of dual-modal features and enhancing visual characteristics and detection performance, thereby improving the stability of detection in different environments.

[0025] In one embodiment, the training process of the second model is as follows: obtaining the input of the image data , and the input of radar data , their outputs are concatenated through the convolutional layer and represented as , and its calculation formula is as follows:

[0026] Among them, 3×3 represents the size of the convolution kernel, and Concat includes concatenation and convolution operations; The resulting feature map is divided into two parts: and ,in, represents the image data features converted from radar data, Represents radar data features derived from image data, and converts radar data features into visible light image data features through linear transformation And visible light image data features are converted into radar data features through linear transformation , as shown below:

[0027]

[0028] Then calculate the value based on the neural network function :

[0029] in, They are specified for calculation The neural network function of the value, Represent the weight parameters in the neural network respectively, and apply convolution operation and activation function to learn the parameters:

[0030] Finally, the alignment and modulation characteristics of each mode are obtained as follows:

[0031] Where: ☉ represents the Hadamard product, , They respectively represent the image data features and radar data features after feature alignment.

[0032] In one possible implementation, first data for controlling the vehicle light is obtained, and a detection and analysis is performed to determine whether there is a moving object within the first illumination range; If so, the first model is used to calculate and determine the position information of other vehicles, and then the second model is used to calculate and determine the position information of the face or the pedestrian; If it does not exist, the second model is executed first to calculate and determine the face position information or pedestrian position information, and then the first model is executed to calculate and determine the position information of other vehicles.

[0033] In one possible implementation, the first model uses a one-stage model algorithm, and the second model uses a two-stage model algorithm. The one-stage model algorithm is fast, and the two-stage model algorithm is highly accurate.

[0034] In one possible implementation, the first model further identifies the window position information after identifying the vehicle information, and turns off or shields the LED lights at the corresponding positions according to the window position information. Preventing high beam irradiation is mainly to avoid the driver's eyes. When the distance is far, the vehicle can be identified. When the distance is close, the detection accuracy needs to be improved, that is, the non-window position will not interfere with the driver, so the lights do not need to be adjusted. In addition, face recognition can be performed after the window is identified, so that the lights can be adjusted more accurately, especially for some vehicles parked on the roadside, where there are basically no drivers, so there is no need to turn off the lights.

[0035] According to a second aspect of the present disclosure, an ADB vehicle light control system is provided, comprising: The main controller module is capable of exchanging information with the driving computer control module, and the driving computer control module is capable of obtaining the vehicle driving state and environmental information from sensors, and the sensors include at least a vehicle speed sensor, a steering sensor, a light sensor, a rain sensor, an inclination sensor and a vehicle positioning sensor; The information acquisition module includes at least a camera and a radar, wherein the camera can obtain visible light image data and the radar can obtain radar data; The LED light group is controlled by the main controller module to turn off or shield part of the light in the illumination range, and the LED light group has a first illumination range and a second illumination range; The first data is input into the first model of the main controller module for calculation, and the second data is input into the second model of the main controller module for calculation; The data obtained by the information acquisition module is separated into the first data and the second data. The first irradiation range corresponds to the first data, and the second irradiation range corresponds to the second data. The main controller module controls the light of the first irradiation range according to the calculation result of the input first data, and controls the light of the second irradiation range according to the calculation result of the input second data. In this way, the detection accuracy of different objects in the corresponding area can be improved in a targeted manner, and the recognition error can be reduced under the premise of ensuring the response speed.

[0036] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which: In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0038] Figure 1 A schematic diagram of the implementation process of the ADB vehicle light control method according to an embodiment of the present disclosure is shown; Figure 2 A schematic diagram showing the main collection range of the first data and the second data of the ADB headlight control method according to an embodiment of the present disclosure when the vehicle speed is lower than 40 km / h is shown; Figure 3 A schematic diagram showing the main collection range of the first data and the second data of the ADB headlight control method according to an embodiment of the present disclosure at a vehicle speed of 60 km / h is shown; Figure 4 A schematic diagram showing the main collection range of the first data and the second data of the ADB headlight control method according to an embodiment of the present disclosure when the vehicle speed is higher than 100 km / h is shown; Figure 5 A schematic diagram showing the division of lamp bead areas during normal driving of the matrix lamp group of the ADB lamp control method according to an embodiment of the present disclosure; Figure 6 A schematic diagram showing the division of lamp bead areas when the matrix lamp group of the ADB lamp control method according to the embodiment of the present disclosure is turned; Figure 7 A schematic diagram showing the division of lamp bead areas when the matrix lamp group of the ADB lamp control method according to the embodiment of the present disclosure is driving downward; Figure 8A schematic diagram showing the division of lamp bead areas when the matrix lamp group of the ADB lamp control method according to the embodiment of the present disclosure is tilted upward; Fig. 9 A schematic diagram of the structure of the ADB vehicle light control system according to an embodiment of the present disclosure is shown; Fig.10 A schematic diagram of modal alignment of visible light image data and radar data according to an embodiment of the present disclosure is shown; Fig.11 A comparative experimental result diagram of the ADB vehicle light control method according to an embodiment of the present disclosure relative to other models is shown. DETAILED DESCRIPTION

[0039] In order to make the purpose, features, and advantages of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0040] refer to Figure 1 , an ADB vehicle light control method, comprising at least the following steps: The information data for controlling the vehicle lights includes at least image data obtained by a camera and radar data obtained by a radar; it can also be data obtained by other sensors. Visible light image data can be obtained by a camera, and invisible light image data requires conversion of data obtained by the radar.

[0041] The information data is separated to obtain at least first data and second data, wherein the first data corresponds to the first illumination range of the headlight, and the second data corresponds to the second illumination range of the headlight; the first illumination range and the second illumination range may have a partial overlap, or the first illumination range is part of the second illumination range, the first data and the second data may also be partially the same, or the second illumination range is part of the first illumination range. Figure 2 , the figure is the driver's driving perspective, which is basically the same as the perspective of the camera. The double-dash line curve range is the LED light group illumination range. A first camera device and a second camera device can be set. The first camera device obtains the first data of the first illumination range or a part of the first data, that is, the first frame K1 position on the left side of the figure is located in the left front of the controlled vehicle. The second camera device obtains the second data of the second illumination range or a part of the second data, that is, the second frame K2 position on the right side of the figure. To ensure the recognition of different objects at different positions, the first frame K1 and the second frame K2 can overlap in some areas to avoid missed detection.

[0042] As another implementation, only one camera device may be provided to perform cutting after obtaining the image data, that is, the first frame K1 position of each image may be cut out as a separate picture, which becomes part of the first data, and the second frame K2 position may also be cut out as a separate picture, which becomes part of the second data, and then calculation and recognition may be performed.

[0043] The first data acquired is input into the trained first model, and the second data acquired is input into the trained second model. Compared with the second model, the first model can identify the first object information within the first irradiation range faster and more accurately after training, and the second model can identify the second object information within the second irradiation range faster and more accurately after training compared with the first model. The second data can also be input into the first model to identify the first object information, and the first data can also be input into the second model to identify the second object information. Reference Figure 2 , the first object information can be other vehicle location information, such as the location information of cars C1 and C2 in the figure, and the second object information can be face location information or pedestrian location information, such as pedestrian R1 or face L1 in the figure; face L1 may not be recognized when pedestrian R1 is far away, and face L1 recognition is easier than pedestrian R1 recognition when pedestrian R1 is close, and the calculation speed is also faster, so features at different distances can be recognized at different vehicle speeds. The range of the image of the camera equipment at different vehicle speeds can also be set differently. When the vehicle speed is below 40km / h, different objects within a larger area can be recognized, that is, reference Figure 2 If the vehicle speed exceeds 40km / h, the acquisition range of the first data and the second data can be reduced. The detection of objects close to the controlled vehicle is not very meaningful because the frequency of light control will not be too high. Frequent flashing of lights will not only affect the life of the lights but also interfere with vision. If the time interval for controlling the lights is 1 second and the car travels more than ten meters in one second, the detection of objects within ten meters of the vehicle is not very meaningful because they may have been out of the illumination range of the lights when the lights are detected and the lights need to be controlled.

[0044] refer to Figure 3 When the vehicle speed is between 40km / h and 80km / h, for example, at 60km / h, the first data acquisition range can be narrowed to the image range beyond 20 meters ahead, that is, the first middle frame K11 in the figure. The general algorithm calculates the square image, so the square first middle frame K11 can be set to multiple to achieve a larger range of detection. The second data can be within the range of the second middle frame K22, so that the focal length of the camera device can be adjusted to achieve clarity of distant objects. Reference Figure 4When the vehicle speed exceeds 100km / h, the first data and the second data can only be acquired in a smaller range beyond 30 meters in front, so that the focal length of the camera device can be further adjusted. Of course, the minimum control range of the adjustable lights is limited, and it is not very meaningful to identify objects at a distance of more than 200 meters. Therefore, it is sufficient to be able to identify objects within about 100 meters, that is, the positions of the first small frame K12 and the second small frame K23 in the figure are beyond 30 meters in front. When driving at high speed, there are generally no pedestrians or very few pedestrians. Therefore, the image data of the first small frame K12 and the second small frame K13 acquired can be considered as the first data and can be first input into the first model for calculation, and then into the second model for calculation. In addition, according to needs, data outside the range of the first small frame K12 and the second small frame K13 can be input into the third model for calculation, and the third model can be used as a supplement to the first model and the second model.

[0045] In one embodiment, the headlight is a matrix headlight including multiple LED lamp beads. The ADB headlight can also be a DLP headlight or an LED pixel lamp. The LED pixel lamp refers to a headlight composed of multiple light-emitting units, which can form an imaging lighting effect similar to an image, improving driving safety and visibility. The matrix headlight generally has only dozens or hundreds of lamp beads, and the LED pixel lamp can reach tens of thousands of light-emitting units. DLP, the full name of Digital Light Processing, is digital light processing. Simply put, it is to digitally process the image signal first and then project it. This technology is mainly realized by the digital micromirror device (DMD) developed by Texas Instruments (TI). Simply put, the core of the DLP headlight is the digital micromirror chip, which is used to realize digital optical processing.

[0046] According to the obtained other vehicle position information or face position information or pedestrian position information, the LED lights in the matrix light group that illuminate the corresponding positions are turned off or shielded. The LED matrix light group 2 is provided with a plurality of lamp beads 1, and different LED lamp beads 1 have different illumination ranges, such as Figure 5 As shown, multiple rows and columns of LED lamp beads 1 can be arranged, and different LED lamp beads 1 have different illumination ranges. The principles of the left and right lights are basically the same. Taking the left light as an example, the left light is provided with a first lamp bead area A1 for providing lighting for the first illumination range, a second lamp bead area A3 for providing lighting for the second illumination range, and a third lamp bead area A2 for providing lighting for a closer distance.

[0047] like Figure 6As shown, when the vehicle is turning, the first illumination range and the second illumination range also need to be adjusted in the direction directly in front of the vehicle that is not the vehicle's driving direction. For headlights that can be adjusted when the vehicle turns, if the lamp bead 1 in the first lamp bead area A1 also deflects to the right when turning right, the number of lamp beads that can provide lighting for the first illumination range will be reduced, and the first lamp bead area A1 will become smaller accordingly. For headlights that cannot be adjusted when the vehicle turns, if the vehicle turns left, the number of lamp beads that can provide lighting for the first illumination range will be reduced, and the first lamp bead area A1 will become smaller accordingly, that is, the number of rows will be reduced.

[0048] like Figure 7 As shown, when the vehicle tilts downward, the first lamp bead area A1 will become smaller, that is, the number of rows will decrease. Figure 8 As shown, when the vehicle tilts upward, the first lamp bead area A1 will become larger, that is, the number of rows will increase. That is, the data obtained and the adjustment range of the vehicle are different under different driving conditions.

[0049] In one embodiment, the information data obtained for controlling the vehicle lights is multimodal information data, and the multimodal information data at least includes image data obtained by a camera device and modal data obtained by a radar.

[0050] In one embodiment, the first model is based on the YOLOv5 model. The first model mainly identifies vehicle position information. First, the YOLOv5 model is trained using vehicle-annotated image data in the source domain. Of course, a small amount of pedestrian-annotated image data can also be added for training. The main purpose is to identify vehicles, so a large number of pictures of different vehicles are required, including sedans, SUVs, sports cars, buses, trucks, trailers and other functional vehicles. When migrating to the target domain, the correspondence between other vehicle position information in the radar data and the image data obtained by the camera equipment is used to generate high-confidence target domain pseudo-labels for the target domain image data, with a confidence range of 95%-99%. The source domain model is further incrementally trained to improve the detection performance of the model in the target domain.

[0051] In one embodiment, the image data in the source domain dataset has a label, denoted as ,in Represents the source domain image data The set of object labels in Represents the bounding box labels of objects in the image data, Indicates the object category label corresponding to the bounding box, Based on the YOLOv5 model, the source domain dataset For supervised training, the loss function used is as follows:

[0052] Represents the CIOU loss, which is calculated as follows:

[0053]

[0054]

[0055] Among them, IOU represents the intersection-over-union ratio of the predicted target box and the real box. Represents the Euclidean distance between the center point of the predicted target box and the real box, Represents the minimum diagonal distance of the closure area that can contain both the prediction box and the target box. Represents the difference in aspect ratio between the predicted target box and the real box, express The weight of and Represent the aspect ratios of the real box and the target box respectively, represents the target detection model used, represents the operation of binary cross entropy loss on classification probability and object prediction probability, The calculation is as follows:

[0056]

[0057] in, Represents the IOU of the predicted box and the real box, is the predicted target confidence (or category confidence) obtained by the Sigmoid function, where Represents the results directly output by the model.

[0058] In one embodiment, the second model can exchange spatial features of visible light image data obtained by a camera and modal data obtained by a radar, achieving complementary integration of dual-modal features to enhance visual characteristics and detection performance. The second model mainly recognizes face position information and pedestrian position information. Of course, a small number of rear-end images of vehicles can also be input for training, so that vehicles traveling in the same direction can be identified to avoid affecting the front vehicle's view of the rearview mirror.

[0059] like Fig.10 As shown, in one embodiment, the training process of the second model is as follows: obtaining the input of the image data , and the input of radar data , their outputs are concatenated through the convolutional layer and represented as , and its calculation formula is as follows:

[0060] Among them, 3×3 represents the size of the convolution kernel, and Concat includes concatenation and convolution operations; The resulting feature map is divided into two parts: and ,in, represents the image data features converted from radar data, Represents radar data features derived from image data, converting radar data features into visible light image data features through linear transformation And visible light image data features are converted into radar data features through linear transformation , as shown below:

[0061]

[0062] Then calculate the value based on the neural network function :

[0063] in, They are specified for calculation The neural network function of the value, Respectively represent the weight parameters in the neural network, and apply convolution operations and activation functions to learn the parameters:

[0064] Finally, the alignment and modulation characteristics of each mode are obtained as follows:

[0065] Where: ☉ represents the Hadamard product, , They respectively represent the image data features and radar data features after feature alignment.

[0066] In one embodiment, multimodal information data for controlling the vehicle lights is obtained, and a detection and analysis is performed to determine whether there is a moving object within the first illumination range; If so, the first model is preferentially executed to calculate and determine other vehicle location information, i.e., whether there is a vehicle and the location of the vehicle, and then the second model is executed to calculate and determine the face location information or pedestrian location information, i.e., whether there is a face and the location of the face; If not, the second model is executed first to calculate the face position information or pedestrian position information, that is, whether there is a face and the position of the face, and then the first model is executed to calculate other vehicle position information, that is, whether there is a vehicle and the position of the vehicle.

[0067] In one embodiment, after detecting the position information of other vehicles, their moving trajectories are detected and predicted, that is, the position of the vehicle after a certain period of time, such as one second, is predicted, and the headlights are controlled in advance according to the predicted trajectory. The headlights can be adjusted in the first place to make the lighting effect of the headlights more ideal.

[0068] In one embodiment, a third model can be trained, and the third model can identify other vehicle location information, face location information, pedestrian location information, and other information, that is, when there is no moving object, the first data can be input into the third model for identification, without the need to execute the algorithm twice. The third model can directly adopt the identification scheme of the existing technology, mainly to identify more and more complete information. The first model is mainly to improve the reaction speed, so that the headlights can be controlled in the first time, and the recognition accuracy of the vehicle can be improved. The headlight control is in a cyclic state, with a certain interval period. Frequent turning on and off of the headlights will also affect oncoming vehicles. That is to say, there is enough time to identify different objects within a cycle, but a faster reaction speed is required at the beginning of the cycle.

[0069] The presence of moving objects can be directly detected and analyzed through radar. If the vehicle is not equipped with radar, it can also be analyzed through images. The detection accuracy of deep learning algorithms is high, but it is time-consuming. For example, YOLOv8 can be used for detection, and OpenCV can also perform motion detection.

[0070] In one embodiment, the first model adopts a one-stage model algorithm, and the second model adopts a two-stage model algorithm. The two-stage model algorithm includes R-CNN, FastR-CNN, Faster R-CNN, etc., and the one-stage model algorithm includes YOLO, SSD, etc. The one-stage model algorithm is fast, and the two-stage model algorithm has high accuracy, and a higher accuracy is required for face recognition. In the ADB headlight control logic, target object detection of vehicles and pedestrians needs to be performed separately, mainly because vehicles and pedestrians have significant differences in size, shape, motion characteristics, etc. Separate detection can more accurately identify and locate these targets, thereby achieving more sophisticated lighting control. For example, vehicles are usually much larger than pedestrians, and their movement trajectories are more regular, while pedestrians may suddenly appear or move quickly on the road. By detecting separately, the ADB system can adjust the lights more accurately to avoid dazzling vehicle drivers or pedestrians.

[0071] In one embodiment, the LED lights of the matrix light group can be divided into sensitive areas and non-sensitive areas. The LED lights in the sensitive area can illuminate and control the left front area of ​​the vehicle, and the acquired multimodal information data is segmented. The multimodal information data obtained by controlling the left front area of ​​the vehicle is segmented separately and analyzed and calculated using the first model. The control cycle of the lights in the sensitive area is 0.5 seconds to 1 second, that is, the state can be changed only after being turned on or off for 0.5 seconds to 1 second, so as to avoid frequent flashing of the lights affecting the line of sight and easily causing visual fatigue. The non-sensitive area is mainly in the closer area and the right area. The control cycle of the non-sensitive area can be set at 1 second to 3 seconds, or the closing cycle can be longer than the opening cycle, that is, it can be turned on after 3 seconds of closing, but it can be turned off after 1 second of opening.

[0072] In one embodiment, the first model further identifies the window position information after identifying the vehicle information, and turns off or shields the LED lights at the corresponding positions according to the window position information. This is mainly for vehicles with multi-row and multi-column matrix LED lights, and the lamp beads at different heights can be turned off or shielded. The headlights shining on the vehicle body will not affect other people, but if they shine on the windows, they may generally cause interference to the people in the car, but it is difficult to judge whether there is a face in the window, so the window is directly identified, and the corresponding headlights can be turned off when the window is identified.

[0073] In one embodiment, as a supplementary recognition method, in the case of good light, the face recognition is performed after the window recognition. If the face can be recognized, it can be selectively avoided, and if the face cannot be recognized, it can be completely avoided.

[0074] In one embodiment, in extreme weather conditions, the camera may be contaminated by foreign objects, water droplets, dust, etc., and the generated images will produce spots, which need to be removed. On the one hand, the spots can be identified using trained models such as YOLO, that is, multiple labeled images with spots are input into the model for training, and then the obtained images are all subjected to spot recognition. After identification, the spots are segmented and rendered, or directly input into the training model for feature recognition after the spots are removed. On the other hand, images obtained before and after a certain time can be compared. If a certain position always has the same feature object, and if the feature object and the nearby connected area are combined and the feature object cannot be identified, it can be used as a spot removal.

[0075] refer to Fig. 9 , provides an ADB vehicle light control system, including: The main controller module is capable of exchanging information with the driving computer control module, and the driving computer control module is capable of obtaining the vehicle driving state and environmental information from sensors, and the sensors include at least a vehicle speed sensor, a steering sensor, a light sensor, a rain sensor, an inclination sensor and a vehicle positioning sensor; The information acquisition module includes at least a camera and a radar, wherein the camera can obtain visible light image data and the radar can obtain radar data; The LED light group is controlled by the main controller module to turn off or shield part of the light in the illumination range, and the LED light group has a first illumination range and a second illumination range; Separate the data obtained by the information acquisition module into first data and second data, the first irradiation range corresponds to the first data, the second irradiation range corresponds to the second data, the first data is input into the first model of the main controller module for calculation, and the second data is input into the second model of the main controller module for calculation; The main controller module controls the light of the first irradiation range according to the calculation result of the input first data, and controls the light of the second irradiation range according to the calculation result of the input second data.

[0076] In some implementations, a third model or more other models may be provided, and different models may be used to perform targeted analysis and identification on different data.

[0077] The vehicle speed sensor can detect the vehicle speed. When the vehicle is traveling faster, the area that the information acquisition module needs to identify is reduced, which can improve the recognition accuracy and speed of key distant locations. The near distance can be ignored due to the short exposure time, or other algorithms with higher accuracy can be used. When the vehicle speed is slow, the detection range is larger, and the near distance can be detected first, and then the focal length can be changed to obtain a clear image in the distance and then detect the distant distance, or multiple cameras can be used to achieve image acquisition near and far.

[0078] The steering sensor can obtain the vehicle's wheel steering angle. On a left-hand drive vehicle, the left front wheel steering angle is closest to the direction of movement of the driver's position, which can be considered as the vehicle's direction of movement. In theory, the headlights should also be adjusted to follow the direction of rotation of the left front wheel, so the range of the first data and the second data acquisition also needs to be adjusted. The visible light image data is acquired by a camera, so the image cutting range can be changed. For example, when the vehicle turns left, there is basically no oncoming vehicle in front, so the cutting area needs to be moved a certain distance to the left. The data range of radar and other data can also be adjusted. The actual driver's position is a certain distance from the left front wheel, so the moving direction of the driver's position is also different from that of the left front wheel. The corresponding moving direction angle can be calculated based on the distance and angle between the driver and the front and rear wheels, so that the headlights can always maintain the same moving direction as the driver, and the acquired data always maintains a certain relative position.

[0079] According to the information from the photosensitive sensor, the lights can be automatically turned off when there is sufficient light, and turned on when there is insufficient light. According to the rain sensor, special modes can be turned on, especially for the treatment of reduced image clarity. For example, when the picture is clear, the third model is used for recognition, and when the picture is not clear, the first model is used for recognition. The first model uses more pictures obtained in different weather conditions during training, and the third model obtains regular night pictures. In addition, the vehicle's information acquisition range can be adjusted according to the angle information obtained by the inclination sensor, and the vehicle positioning sensor can be used to obtain location information, and then the comprehensive analysis of the data provided by the navigation can be used to control the lights.

[0080] To turn off the LED lights in the matrix light group that illuminate the corresponding position, you can turn off the power supply or use shielding to reduce the light intensity at the corresponding position. Generally speaking, the frequency of turning off the power supply should not be too high, and the frequency of shielding can be higher, but not more than 0.5 seconds, otherwise frequent flashing is not conducive to driving safety.

[0081] like Fig.11 As shown in the figure, the average precision (AP) and frames per second (FPS) are used to evaluate the deployment capability of the method of this application, where the average precision (AP) uses an AP threshold of 0.5. 50 As the average precision indicator. We compared other common object detection algorithms as reference models, including YOLOv3, YOLOv5, Mask R-CNN and YOLOv7, and found that Fig.10 As a result, the algorithm of this application integrates YOLOv5 and achieves 50 Outperforming other models, and combining image and radar data, the FPS reaches 126 frames per second.

[0082] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting or processing in other suitable ways as necessary, and then stored in the computer memory. Among them, the memory may include a large capacity memory for data or instructions. By way of example and not limitation, the memory may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory may be inside or outside the data processing device. In a specific embodiment, the memory is a non-volatile memory. In a specific embodiment, the memory includes a read-only memory (ROM) and a random access memory (RAM).Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable Programmable Read-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0083] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0084] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0085] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. An ADB vehicle light control method, characterized in that: The method comprises at least the following steps: Acquiring information data for controlling vehicle lights, including image data obtained by camera equipment and radar data obtained by radar; Separating the information data to obtain at least first data and second data, wherein the first data corresponds to a first illumination range of the headlight, and the second data corresponds to a second illumination range of the headlight; Inputting the first data and the second data into a pre-trained first model and a second model respectively, wherein the first model is used to identify first object information within the first irradiation range, and the second model is used to identify second object information within the second irradiation range; According to the obtained first object information or the second object information, the LED light in the vehicle light that illuminates the corresponding position is turned off or shielded.

2. The ADB vehicle light control method according to claim 1, characterized in that: The first illumination range is located in front of the left side of the controlled vehicle, the second illumination range is located in front of the right side of the controlled vehicle, the first object information is position information of other vehicles, and the second object information is position information of human faces or pedestrians.

3. The ADB vehicle light control method according to claim 2, characterized in that: The controlled vehicle lights include a plurality of LED lamp bead matrix headlights; or ADB headlights; or LED pixel lights.

4. The ADB vehicle light control method according to claim 3, characterized in that: The first model is based on the YOLOv5 model. The YOLOv5 model is first trained on the source domain using image data annotated with vehicles. When migrating to the target domain, the correspondence between other vehicle position information in the radar data and the image data obtained by the camera device is used to generate target domain pseudo labels for the image data in the target domain, thereby improving the detection performance of the first model in the target domain.

5. The ADB vehicle light control method according to claim 4, characterized in that: The image data in the source domain dataset has a label, denoted as ,in Represents the source domain image data The set of object labels in Represents the bounding box labels of objects in the image data, Indicates the object category label corresponding to the bounding box, Based on the YOLOv5 model, the source domain dataset For supervised training, the loss function used is as follows: Represents the CIOU loss, which is calculated as follows: Among them, IOU represents the intersection-over-union ratio of the predicted target box and the real box. Represents the Euclidean distance between the center point of the predicted target box and the real box, Represents the minimum diagonal distance of the closure area that can contain both the prediction box and the target box. Represents the difference in aspect ratio between the predicted target box and the real box, express The weight of and Represent the aspect ratios of the real box and the target box respectively, represents the target detection model used, represents the operation of binary cross entropy loss on classification probability and object prediction probability, The calculation is as follows: in, Represents the IOU of the predicted box and the real box, is the predicted target confidence (or category confidence) obtained by the Sigmoid function, where Represents the results directly output by the model.

6. The ADB vehicle light control method according to claim 1, characterized in that: The second model can exchange spatial features of visible light image data obtained by a camera and radar data obtained by a radar, thereby achieving complementary integration of dual-modal features and enhancing visual characteristics and detection performance.

7. The ADB vehicle light control method according to claim 6, characterized in that: The training process of the second model is as follows, including: Get input for image data , and the input of radar data , their outputs are connected through the convolutional layer and expressed as , and its calculation formula is as follows: Among them, 3×3 represents the size of the convolution kernel, and Concat includes concatenation and convolution operations; The resulting feature map is divided into two parts: and ,in, represents the image data features converted from radar data, Represents radar data features derived from image data; Convert radar data features into visible light image data features through linear transformation , converting the visible light image data features into radar data features through linear transformation , as shown below: Then calculate the value based on the neural network function : in, They are specified for calculation The neural network function of the value, Represent the weight parameters in the neural network respectively, and apply convolution operation and activation function to learn the parameters: Finally, the alignment and modulation characteristics of each mode are obtained as follows: Where: ☉ represents the Hadamard product, , They respectively represent the image data features and radar data features after feature alignment.

8. The ADB vehicle light control method according to any one of claims 2 to 7, characterized in that: Acquire the first data for controlling the vehicle lights, and detect whether there is a moving object within the first illumination range; If so, the first model is used to calculate and determine the position information of other vehicles, and then the second model is used to calculate and determine the position information of the face or the pedestrian; If it does not exist, the second model is executed first to calculate and determine the face position information or pedestrian position information, and then the first model is executed to calculate and determine the position information of other vehicles.

9. The ADB vehicle light control method according to claim 8, characterized in that: The first model uses a one-stage model algorithm, and the second model uses a two-stage model algorithm. After identifying the vehicle information, the first model further identifies the window position information, and turns off or shields the LED lights at the corresponding positions according to the window position information.

10. An ADB vehicle light control system, characterized in that: include: A main controller module capable of exchanging information with a driving computer control module, wherein the driving computer control module is capable of obtaining vehicle driving status and environmental information from sensors; An information acquisition module, comprising at least a camera device and a radar, wherein the camera device can obtain visible light image data, and the radar can obtain radar data; An LED light group, which is controlled by the main controller module to turn off or shield part of the light in the illumination range, and the LED light group has a first illumination range and a second illumination range; Separating the data obtained by the information acquisition module into first data and second data, the first irradiation range corresponds to the first data, and the second irradiation range corresponds to the second data; The first data is input into a first model of the main controller module for calculation, and the second data is input into a second model of the main controller module for calculation; The main controller module controls the light of the first irradiation range according to the calculation result of the first data input, and controls the light of the second irradiation range according to the calculation result of the second data input.

Images