Vehicle anti-dazzling dimming method and device, electronic equipment and computer readable medium

Abstract

The embodiment of the invention discloses a vehicle anti-dazzling dimming method and device, electronic equipment and a computer readable medium. According to the main technical scheme, the method comprises the following steps: acquiring an external illumination image sequence of a vehicle, and acquiring motion sensing information of an opposite vehicle; performing light spot detection processing on the external illumination image sequence to obtain a light spot information sequence; based on the light spot information sequence, the light incident angle of each historical time is determined, and the light incident angle is the included angle between the light emitted by the light source and the normal vector of the windshield of the vehicle; based on the light spot information sequence, the motion sensing information and the light incident angle of each historical time, predicting light spot position information and dazzle light intensity information of N times in the future; and performing regional dimming control on the windshield of the vehicle based on the light spot position information and the dazzle light intensity information at the N times in the future. The vehicle anti-dazzling dimming mode can greatly reduce the dazzling risk of the vehicle and improve the safety and comfort of night driving.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to vehicle anti-glare dimming methods, devices, electronic equipment, and computer-readable media. Background Technology

[0002] When driving at night or in complex lighting conditions, the glare from oncoming headlights can easily dazzle drivers, causing a sudden drop in visibility, delayed reaction time, and significantly increasing the risk of traffic accidents. Traditional anti-glare technologies mainly rely on physical sun visors, polarized glass with fixed transmittance, or manually adjustable chromaticity glass to mitigate glare interference through physical blocking or optical filtering. However, with increasingly complex traffic environments (such as the widespread adoption of LED and xenon headlights and the continuous increase in nighttime traffic density), traditional technologies, lacking intelligence and dynamic response capabilities, struggle to achieve rapid and precise light suppression, thus reducing the safety and comfort of nighttime driving. Summary of the Invention

[0003] This application provides a vehicle anti-glare dimming method, device, electronic device, and computer-readable medium to reduce the risk of glare and improve the safety and comfort of nighttime driving.

[0004] This application provides the following solution: According to a first aspect, a vehicle anti-glare dimming method is provided, the method comprising: acquiring a sequence of external lighting images of the vehicle using a first sensor, and acquiring motion sensing information of an oncoming vehicle using a second sensor; performing spot detection processing on the external lighting image sequence to obtain a spot information sequence, wherein the spot information sequence includes spot information at each historical time; determining the light incidence angle at each historical time based on the spot information sequence, wherein the light incidence angle is the angle between the light emitted from the light source and the normal vector of the vehicle's windshield; predicting spot position information and glare intensity information for the next N time periods based on the spot information sequence, the motion sensing information, and the light incidence angles at each historical time period, wherein N is a preset positive integer; and performing regional dimming control on the vehicle's windshield based on the spot position information and glare intensity information for the next N time periods.

[0005] According to a second aspect, a vehicle anti-glare dimming device is provided, the device comprising: an environmental perception module configured to acquire a sequence of external lighting images of the vehicle using a first sensor, and to acquire motion sensing information of an oncoming vehicle using a second sensor; a data processing module configured to perform spot detection processing on the external lighting image sequence to obtain a spot information sequence, wherein the spot information sequence includes spot information at each historical time; the data processing module is further configured to determine the light incidence angle at each historical time based on the spot information sequence, wherein the light incidence angle... The angle between the light emitted from the light source and the normal vector of the windshield of the vehicle is defined. The data processing module is configured to predict the position information of the light spot at N future times based on the light spot information sequence, where N is a preset positive integer. The glare prediction module is configured to predict the glare intensity information at N future times based on the light spot information sequence, the motion sensing information, and the light incident angle at each historical time. The localized dimming control module is configured to perform localized dimming control on the windshield of the vehicle based on the light spot position information and glare intensity information at the N future times.

[0006] According to a third aspect, an electronic device is provided, comprising: one or more processors; and a memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of any of the methods described in the first aspect.

[0007] According to a fourth aspect, a computer-readable medium is provided having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the method described in any implementation of the first aspect above.

[0008] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides a method for vehicle anti-glare dimming. By acquiring and fusing various environmental perception data, it can quickly identify the position of the light spot and the incident angle of the light from the headlights of oncoming vehicles, and further predict the light spot trajectory and glare intensity. Based on the predicted light spot trajectory and glare intensity, the transmittance of a local area of ​​the windshield is adjusted in real time through regional dimming control. This allows the driver to receive necessary shading only in the affected glass area without affecting the overall visibility range, reducing the risk of glare and thus improving the safety and comfort of nighttime driving. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A flowchart of a vehicle anti-glare dimming method provided in an embodiment of this application.

[0011] Figure 2 This is a flowchart for predicting spot information based on spot candidate maps, provided in an embodiment of this application.

[0012] Figure 3 This is a schematic diagram of the entire spot detection and processing flow provided in the embodiments of this application.

[0013] Figure 4 A flowchart illustrating the steps for determining the incident angle of light, as provided in an embodiment of this application.

[0014] Figure 5 This is a schematic diagram of the entire vehicle anti-glare dimming process provided in the embodiments of this application.

[0015] Figure 6 A schematic block diagram of a vehicle anti-glare dimming device provided in an embodiment of this application.

[0016] Figure 7 A schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0018] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0019] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0020] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0021] When driving at night or in complex lighting conditions, the strong light from the headlights of oncoming vehicles can easily dazzle drivers, and traditional anti-glare technologies struggle to achieve fast and precise light suppression, resulting in lower safety and comfort during nighttime driving.

[0022] In view of this, this application provides a new approach, which will be described in detail below with reference to the accompanying drawings and embodiments.

[0023] Figure 1 This is a flowchart illustrating the vehicle anti-glare dimming process provided in an embodiment of this application. Figure 1 As shown, the method may include the following steps: Step 101: Using the first sensor, acquire the vehicle's external lighting image sequence, and using the second sensor, acquire motion sensing information of the oncoming vehicle.

[0024] Step 102: Perform spot detection processing on the image sequence of external lighting to obtain a spot information sequence, wherein the spot information sequence includes spot information at each historical time.

[0025] Step 103: Based on the light spot information sequence, determine the light incident angle at each historical time, where the light incident angle is the angle between the light emitted from the light source and the normal vector of the vehicle's windshield.

[0026] Step 104: Based on the light spot information sequence, motion sensing information, and the light incident angle at each historical time, predict the light spot position information and glare intensity information for the next N time periods, where N is a preset positive integer.

[0027] Step 105: Based on the light spot position information and glare intensity information for the next N time periods, perform regional dimming control on the vehicle's windshield.

[0028] As can be seen from the above process, this application provides a vehicle anti-glare dimming method. By acquiring and fusing various environmental perception data, it can quickly identify the position of the light spot and the incident angle of the light from the headlights of oncoming vehicles, so as to further predict the light spot trajectory and glare intensity. Based on the predicted light spot trajectory and glare intensity, the transmittance of a local area of ​​the windshield can be adjusted in real time through regional dimming control. This allows the driver to only receive necessary shading in the affected glass area without affecting the overall visibility range, reducing the risk of glare and thus improving the safety and comfort of nighttime driving.

[0029] The following describes in detail each step of the above process and the effects that can be further produced, with reference to the embodiments. It should be noted that the terms "first" and "second" used in this disclosure do not have limitations on size, order, or quantity, but are only used to distinguish them by name. For example, "first sensor" and "second sensor" are used to distinguish two different sensors.

[0030] First, the above step 101, namely "using the first sensor to acquire the vehicle's external lighting image sequence and using the second sensor to acquire the motion sensing information of the oncoming vehicle", will be described in detail with reference to the embodiments.

[0031] In this embodiment, the first sensor can be a forward-facing camera mounted outside the vehicle, and the second sensor can be a millimeter-wave radar. The oncoming vehicle can be a vehicle traveling in the opposite direction to the vehicle being approached. The sequence of external lighting images can be RAW format lighting images continuously captured by the forward-facing camera of the vehicle when the vehicles meet. The motion sensing information can be the relative speed of the oncoming vehicle and the distance between it and the current vehicle.

[0032] The following describes in detail step 102, namely, "performing spot detection processing on the vehicle exterior illumination image sequence to obtain a spot information sequence, wherein the spot information sequence includes spot information from each historical time", with reference to the embodiments.

[0033] In this embodiment, the historical time periods involved in this application can be various time intervals obtained by dividing the process of capturing the above-mentioned sequence of vehicle exterior lighting images in the time dimension. The light spot information can be the information of the light spot formed on the image when the strong light from an oncoming vehicle shines on the current vehicle. The light spot information may include, but is not limited to, features such as the position of the light spot, the area of ​​the light spot, the aspect ratio, and the average brightness.

[0034] For example, for each external lighting image in the sequence of external lighting images, this application can first perform preprocessing such as grayscale conversion and Gaussian filtering on the external lighting image to remove color interference and suppress noise. Then, binarization and mask extraction are performed on the preprocessed external lighting image to separate the spot area from the background and obtain a clear spot binary mask. After that, spot features are extracted through connected component analysis, and the external lighting image with spot features and marked spot areas is used as spot information.

[0035] As one feasible approach, in the aforementioned continuous shooting scenario, real-time ambient brightness data outside the vehicle, collected by a light sensor, is typically used to plan and capture exterior lighting images with different exposure times to avoid losing highlight details due to overexposure in a single frame. For example, in a bright environment, five exterior lighting images with different exposure times can be captured, including one short exposure, three medium exposures, and one long exposure. The short exposure is used to preserve highlight details, the medium exposure to preserve subject details, and the long exposure to preserve shadow details. Therefore, the above exterior lighting image sequence can be processed by spot detection to obtain a spot information sequence through the following steps: The first step involves inter-frame fusion of the exterior lighting image frames within the same time window in the aforementioned sequence, based on a preset time window, to obtain reconstructed images for each historical time period. The time window can be a time range used to divide data into finite intervals along the time dimension. Inter-frame fusion can be a pixel-weighted nonlinear fusion of exterior lighting images with different exposure times. The reconstructed image can be an exterior lighting image where details in locally overexposed areas have been restored. It should be noted that the time intervals obtained by dividing the exterior lighting image sequence according to the time window are the historical times, and each historical time corresponds one-to-one with the collected ambient brightness data.

[0036] For example, the above sequence of exterior lighting images is first segmented according to a time window to obtain various historical times. Then, for each historical time, pixel-level weighted nonlinear fusion processing is performed on each lighting image frame within the same historical time based on the corresponding ambient brightness data to obtain the reconstructed image corresponding to that historical time. Specifically, during the pixel-level weighted nonlinear fusion processing, each lighting image frame can first undergo de-mosaicing and white balance preprocessing. Then, using ambient brightness data as a weighting factor, the contribution of each frame to the highlight region is dynamically adjusted. Next, neighborhood interpolation is performed on saturated pixels to preferentially utilize unsaturated pixels of the same color channel in shorter exposure frames. Finally, a local contrast preservation mechanism is introduced to suppress fusion artifacts. Thus, by fusing multiple frames of raw data under different exposures, rich details in both bright and dark areas can be preserved in a single image, which can improve the accuracy of subsequent spot detection.

[0037] The second step involves performing spot detection processing on the reconstructed images from each historical time period to obtain a spot information sequence. Specifically, a preset spot detection method can be used to perform spot detection processing on the reconstructed images from each historical time period to obtain a spot information sequence. For example, the preset spot detection method may include, but is not limited to, Gaussian fitting, gray-level centroid method, and circle fitting method.

[0038] One feasible approach is to perform spot detection processing on the reconstructed images from each of the aforementioned historical time periods using the following sub-steps to obtain a spot information sequence: The first sub-step involves generating a brightness saliency map and a normalized gradient magnitude map for each reconstructed image. In the brightness saliency map, the brightness value of each pixel represents the probability that the corresponding pixel in the reconstructed image belongs to a strong light source spot. In the normalized gradient magnitude map, the brightness value of each pixel represents the gradient change at the corresponding position in the reconstructed image. The pixel values ​​in both the brightness saliency map and the normalized gradient magnitude map are between 0 and 1. Specifically, for each reconstructed image, the image can be input into a lightweight U-Net (U-shaped Network) segmentation model to output a brightness saliency map. The Sobel operator is used to calculate the gradients of the reconstructed image in the horizontal and vertical directions, and the gradient magnitude map is determined according to a preset gradient magnitude formula. Then, the gradient magnitude map is normalized to obtain a normalized gradient magnitude map whose value range matches that of the brightness saliency map.

[0039] The second sub-step involves performing a weighted linear fusion of the aforementioned brightness saliency map and the aforementioned normalized gradient magnitude map to obtain a candidate spot image. The sum of the weights of the brightness saliency map and the aforementioned normalized gradient magnitude map is 1. It should be noted that the brightness saliency map is directly related to the determination of the spot region and contributes more to spot detection; therefore, the weight of the brightness saliency map is usually higher. For example, the weight of the brightness saliency map can be set to 0.65.

[0040] The third sub-step involves predicting the spot information of the reconstructed image based on the aforementioned spot candidate image. This spot information may include spot location information and spot brightness information. Spot location information may include, but is not limited to, the spot's position coordinates. Spot brightness information may include the average brightness of the spot. It should be noted that predicting spot information based on the spot candidate image corresponding to the reconstructed image can effectively combine the global feature discrimination capability of semantic segmentation with the sensitivity of traditional image processing to local high-frequency information, thereby improving the detection accuracy and efficiency of spot regions.

[0041] As one possible way, such as Figure 2As shown, the light spot information of the reconstructed image can be predicted based on the above light spot candidate image through the following sub-steps S21-S24: S21: Perform threshold-based region segmentation and morphological opening operation on the candidate light spot image to obtain an optimized candidate light spot image that includes at least one connected region.

[0042] In this embodiment, by performing threshold-based region segmentation on the candidate spot image, the spot region can be separated from the background. Then, morphological opening operation is performed to erode and then dilate the segmented candidate spot image, which can effectively remove salt-and-pepper noise and isolated false spots, and finally obtain an optimized candidate spot image including at least one connected region.

[0043] S22: For each connected region in at least one connected region, extract the features of the connected region.

[0044] In this embodiment, the characteristics of the connected region may include the region area, aspect ratio, average brightness, and the rate of change of center point displacement between adjacent frames. Specifically, for each connected region, the region area can be calculated by determining the number of pixels in the connected region, the aspect ratio of the connected region can be calculated using the minimum bounding rectangle method, and the average gray value of the pixels in the connected region can be determined as the average brightness. In addition, the centroid coordinates of the connected region can be calculated by image moments, and the coordinate difference between the centroid coordinates and the center point of the corresponding region in the previous frame can be determined as the displacement. The ratio of the displacement to the region area can be used as the rate of change of center point displacement, which can reflect the speed of light spot movement.

[0045] S23: Based on the features of each extracted connected region, perform spot confidence prediction on each connected region in at least one connected region to obtain the spot region.

[0046] In this embodiment, for each of the at least one connected regions, the features of the connected region are input into a lightweight binary classification network to obtain the confidence level of the connected region. If the confidence level is greater than a preset confidence threshold, the connected region is marked as a candidate spot region corresponding to the real light source. For example, the lightweight binary classification network can be an MLP (Multilayer Perceptron) network, and the preset confidence threshold can be set to 0.7. Since multiple overlapping connected regions may be marked as candidate spot regions, to ensure that each real physical light source corresponds to only one tracking target, a non-maximum suppression algorithm can be used to select the candidate spot region with the highest confidence level among the obtained candidate spot regions as the final spot region.

[0047] S24: Based on the spot region, obtain the spot information of the reconstructed image.

[0048] In this embodiment of the application, the features and location coordinates of the connected region corresponding to the spot region can be determined as the spot information of the reconstructed image.

[0049] By using the above method of first thresholding and then determining the spot information based on confidence, the area to be detected can be narrowed, the efficiency of spot detection can be improved, and the segmented area can be evaluated based on confidence to avoid false detections and improve the accuracy of spot detection results.

[0050] by Figure 3 For example, Figure 3 The above illustrates the entire spot detection process. First, a sequence of images of external lighting is acquired. Then, inter-frame fusion of the images is performed to obtain reconstructed images for each historical time. Next, for each reconstructed image, a candidate spot image corresponding to the reconstructed image is determined. The candidate spot image is then segmented into regions to obtain an optimized candidate spot image including at least one connected region. Spot features are extracted from each connected region, and the spot information is determined by combining the confidence level. Thus, spot detection can be achieved.

[0051] The following describes in detail step 103, namely, "determining the incident angle of light at each historical time based on the light spot information sequence, wherein the incident angle of light is the angle between the light emitted from the light source and the normal vector of the vehicle's windshield," with reference to the embodiments.

[0052] As one possible way, such as Figure 4 As shown, for each of the above historical times, the corresponding incident angle of light can be determined through the following steps S41-S43: S41: Using the intrinsic parameters of the first sensor, determine the light source direction vector in the camera coordinate system based on the light spot information from historical time.

[0053] In this embodiment, the light source direction vector can represent the direction of illumination from the light source of an oncoming vehicle. Specifically, using the calibration matrix and distortion coefficients of the external forward-facing camera, the position coordinates of the light spot information from the aforementioned historical time are mapped to camera coordinates to obtain the light source direction vector.

[0054] S42: Using the pre-acquired vehicle body posture information, the light source direction vector in the camera coordinate system is transformed to the vehicle body coordinate system to obtain the light source direction vector in the vehicle body coordinate system.

[0055] In this embodiment of the application, the vehicle attitude information may be the pitch angle and roll angle information of the vehicle obtained from the inertial sensor.

[0056] S43: Determine the incident angle of light at historical times based on the normal vector of the windshield and the direction vector of the light source in the vehicle coordinate system.

[0057] In this embodiment, the incident angle of light at the aforementioned historical time can be determined using the following formula: in, This indicates the angle of incidence of the light ray. This represents the direction vector of the light source in the vehicle coordinate system. This represents the normal vector of the windshield in the vehicle's coordinate system. If If the angle of incidence is less than a preset threshold, the light entering the driver's field of vision is very likely to cause glare. For example, the angle of incidence threshold could be 25 degrees.

[0058] In this way, by determining the light source direction vector based on the light spot information and transforming the light source direction vector to the vehicle coordinate system, the light source direction vector and the windshield normal vector can be placed in the same coordinate system, which makes it easier to accurately determine the incident angle of the light.

[0059] The following describes in detail step 104, namely, "predicting the position information of the light spot and the glare intensity information of the next N time periods based on the light spot information sequence, motion sensing information and the light incident angle at each historical time, where N is a preset positive integer," with reference to the embodiments.

[0060] In this embodiment, a preset time-series data prediction method can be used to predict the position information of the light spot and the glare intensity information for the next N time periods, based on the aforementioned light spot information sequence, the aforementioned motion sensing information, and the light incidence angle at each historical time. The glare intensity information can be information about the intensity of strong light interfering with the driver's vision. The aforementioned time-series data prediction method may include, but is not limited to, at least one of the following: LSTM (Long Short-Term Memory) network, GRU (Gated Recurrent Unit), and Kalman filter algorithm.

[0061] One feasible approach is to predict the spot position and glare intensity information for the next N time periods based on the aforementioned spot information sequence, motion sensing information, and light incidence angles at each historical time point: The first step is to use a Kalman filter to predict the position information of the light spots at the next N time points based on the light spot information sequence at each historical time point. Specifically: First, define the state vector of the light spot: in, Indicates a point in time. For the first The pixel coordinates of the center of the light spot at that moment. For the first The velocity component of the light spot in the image space at any given time.

[0062] Then, assuming the light spot is in an extremely short time The interior undergoes uniform linear motion, and the state transition equation is constructed as follows: .in, For process noise, the state transition matrix for: in, Indicates the first time.

[0063] The observation equation is constructed as follows .in, This is the observation vector, containing only the coordinates of the center pixel of the light spot. The center pixel coordinates of the light spot can be obtained from the light spot information mentioned above. To measure noise, This is the observation matrix. for: By using the prediction step of the Kalman filter, the position of the light spot in the image space in the next N frames can be inferred, i.e., the position information of the light spot in the next N time periods, providing basic data for real-time glare prediction. Furthermore, by predicting the position information of the light spot in the next N time periods in advance, the glass areas requiring focused dimming can be quickly identified during subsequent windshield localized dimming control, thereby improving the time efficiency of dimming.

[0064] Furthermore, based on the position information of the light spot at N future time points, the incident angle of the light at those N time points can also be predicted, where the position information of the light spot corresponds one-to-one with the incident angle of the light.

[0065] The second step involves using a glare intensity prediction model to predict the glare intensity for the next N time periods, based on the aforementioned spot information sequence, motion sensing information, and light incidence angles at each historical time. This glare intensity prediction model can be a time-series predictor constructed from a gated cyclic unit model. Specifically, for each historical time period, the spot information, motion sensing information, and light incidence angle corresponding to that historical time are determined as the glare features for that historical time. Then, the feature sequence composed of the glare features corresponding to each historical time is input into the glare intensity prediction model, and finally, the glare intensity information for the next N time periods is output.

[0066] It should be noted that the above method of predicting glare intensity based on multi-sensor data features can greatly improve the accuracy of glare intensity prediction results. Furthermore, if the feature sequence corresponds to the most recent 10 time steps, then it can ultimately predict the glare intensity information for one time step in real time, or it can predict the glare intensity information for multiple time steps. In other words, the number of glare intensity information output by the glare intensity prediction model can be set by technicians according to the specific needs of the scenario.

[0067] Furthermore, the above glare intensity prediction model can also output the change in the incident angle over the next N time periods.

[0068] Furthermore, the aforementioned glare intensity prediction model employs a weighted multi-task loss function during training. This weighted multi-task loss function is constructed based on glare intensity accuracy loss and angle smoothing loss. The glare intensity accuracy loss is determined by the difference between the actual glare intensity and the predicted glare intensity, i.e., the L1 loss, used to measure the difference between the actual and predicted glare intensity. The angle smoothing loss is determined by the difference between the actual change in incident angle and the predicted change in incident angle, used to measure the smoothness of the angle trajectory. Specifically, the loss function of the glare intensity prediction model can be: in, This represents the predicted value from the glare intensity prediction model. express Real-time glare intensity measurements (for offline training). Indicates prediction Glare intensity value at any given moment. express Time and The change in the true angle of incidence between time points . Indicates prediction Time and The change in the angle of incidence between moments. Indicates the smoothing coefficient for angle changes, for example The value can be between 0.1 and 0.2. Through this loss function, the glare intensity prediction model can not only accurately predict the glare intensity, but also ensure that the predicted light source trajectory is continuous and reasonable in time, preventing drastic changes in the prediction angle due to small noise in the input.

[0069] The following describes in detail step 105, namely "based on the light spot position information and glare intensity information of the future N time periods, perform regional dimming control on the windshield of the vehicle", with reference to the embodiments.

[0070] In this embodiment, the windshield of the vehicle can be dimmable glass. The windshield can be pre-divided into sub-regions. For example, the entire windshield can be divided into... The system comprises 36 non-overlapping sub-regions. Specifically, based on the future spot position and glare intensity, the glare effect can be mapped onto specific glass areas. The transmittance adjustment range is determined according to the risk level, and the transmittance of specific glass areas is adjusted using a dimming driver. The dimming driver can accept duty cycle PWM (Pulse-Width Modulation) signals and control the transmittance of the glass areas by adjusting the voltage.

[0071] One possible approach is to treat N future times as the current time and perform the following steps: The first step is to map the current spot position information onto a sub-region of the windshield to obtain the target glass area. This target glass area is the area of ​​the glass to be dimmed.

[0072] The second step involves generating a dimming matrix based on the target glass region and the glare intensity information. The elements of the dimming matrix represent the predicted transmittance required for the corresponding glass sub-region. Specifically: For the aforementioned target glass area, the aforementioned target glass area If the light transmittance exceeds the preset glare intensity threshold, the transmittance of the target glass area is expressed by the following formula: The preset glare intensity threshold can be used to determine whether a certain glass area is severely affected by glare and whether a darkening operation needs to be performed. This indicates the target glass area mentioned above. This indicates the required light transmittance for the target glass area. This indicates the minimum light transmittance that the dimmable glass can achieve in this area, and is used to set the maximum blocking effect of the glare suppression area.

[0073] For the neighborhood of the target glass area mentioned above, transitional dimming can be performed using the following formula with decreasing weights: in, Indicates the number of the neighboring region. Indicates the first The required transmittance for each neighborhood. This is the attenuation coefficient. For the region To the area The distance.

[0074] Furthermore, a dimming matrix can be constructed by using the required light transmittance of each sub-region as matrix elements, based on the positional order of the sub-regions of the windshield. This facilitates subsequent strong suppression of glare in glare areas and maintains a smooth transition to surrounding areas, ultimately avoiding abrupt visual changes.

[0075] The third step is to perform regional dimming control on each sub-region of the windshield based on the dimming matrix mentioned above.

[0076] As one possible approach, the vehicle can first collect the actual light transmittance of each sub-region of the windshield to obtain an actual light transmittance matrix, and then optimize the dimming matrix based on the actual light transmittance matrix using the following formula: in, for Actual dimming control command based on time. For prediction Dimming matrix based on time. for The actual transmittance matrix over time. Indicates weight, when When set to 0.7, the actual dimming control command is closer to the predicted value. However, we will make corrections based on actual feedback to reduce errors.

[0077] Finally, based on the optimized dimming matrix, dimming control is applied to each sub-region of the windshield. Specifically, a dimming driver can be used to adjust the light transmittance of each sub-region of the windshield in real time by controlling the voltage according to the optimized dimming matrix.

[0078] In practice, the dimming driver typically performs the aforementioned closed-loop adjustment of transmittance at a refresh rate of 60 Hz. This means that it monitors in real time and adjusts the transmittance of each sub-area of ​​the windshield in a timely manner according to the actual dimming control command. Thus, the dimming target can be kept stable through the closed-loop adjustment of transmittance.

[0079] like Figure 5 As shown, Figure 5 This is a schematic diagram of the entire vehicle anti-glare dimming process described above. In oncoming vehicle scenarios, the system typically first acquires external environmental perception data, which includes at least a sequence of external lighting images and motion sensing information of oncoming vehicles. Then, it performs spot detection to identify the light source, calculates the incident angle of the light to assess the glare risk, and if the glare risk is high, it predicts the spot position trajectory and glare intensity, and performs regional dimming control based on the prediction results. If the glare risk is low, the assessment results can be output to the driver's interface so that the driver can quickly confirm that the current environment is safe.

[0080] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0081] According to another embodiment, a vehicle anti-glare dimming device is provided. Figure 6 A schematic block diagram of the vehicle anti-glare dimming system according to one embodiment is shown. Figure 6 As shown, the vehicle anti-glare dimming device 600 includes an environmental perception module 601, a data processing module 602, a glare prediction module 603, and a localized dimming control module 604. The main functions of each component module are as follows: The environmental perception module 601 is configured to acquire a sequence of external lighting images of the vehicle using a first sensor, and to acquire motion sensing information of oncoming vehicles using a second sensor.

[0082] The data processing module 602 is configured to perform spot detection processing on the above-mentioned vehicle exterior lighting image sequence to obtain a spot information sequence, wherein the above-mentioned spot information sequence includes spot information at each historical time.

[0083] The aforementioned data processing module 602 is further configured to determine the light incident angle at each historical time based on the aforementioned light spot information sequence, wherein the aforementioned light incident angle is the angle between the light emitted from the light source and the normal vector of the windshield of the vehicle.

[0084] The aforementioned data processing module 602 is further configured to predict the position information of the light spot at N future time intervals based on the aforementioned light spot information sequence, wherein N is a preset positive integer.

[0085] Glare prediction module 603 is configured to predict glare intensity information for the next N times based on the above-mentioned light spot information sequence, the above-mentioned motion sensing information, and the light incident angle at each of the above-mentioned historical times.

[0086] The localized dimming control module 604 is configured to perform localized dimming control on the windshield of the vehicle based on the light spot position information and glare intensity information for the N future time periods.

[0087] As one possible implementation method, when the data processing module 602 performs spot detection processing on the above-mentioned vehicle exterior lighting image sequence to obtain a spot information sequence, it is specifically configured to perform inter-frame fusion on the lighting image frames located in the same time window in the above-mentioned vehicle exterior lighting image sequence based on a preset time window to obtain the reconstructed images of each historical time; and to perform spot detection processing on the reconstructed images of each historical time to obtain a spot information sequence.

[0088] As one possible implementation, when the data processing module 602 performs spot detection processing on the reconstructed images at each historical time to obtain a spot information sequence, it is specifically configured to generate a brightness saliency map and a normalized gradient magnitude map corresponding to each reconstructed image. In the brightness saliency map, the brightness value of each pixel represents the probability that the corresponding pixel in the reconstructed image belongs to a strong light source spot, and the brightness value of each pixel in the normalized gradient magnitude map represents the gradient change at the corresponding position in the reconstructed image. The brightness saliency map and the normalized gradient magnitude map are then weighted and linearly fused to obtain a spot candidate map. Based on the spot candidate map, the spot information of the reconstructed image is predicted, whereby the spot information includes spot location information and spot brightness information.

[0089] As one possible implementation, when the data processing module 602 predicts the spot information of the reconstructed image based on the aforementioned spot candidate image, it is specifically configured to perform threshold-based region segmentation and morphological opening operations on the aforementioned spot candidate image to obtain an optimized spot candidate image including at least one connected region; for each of the aforementioned at least one connected region, extract the features of the connected region, wherein the features of the connected region include region area, aspect ratio, average brightness, and the rate of change of center point displacement between adjacent frames; based on the extracted features of each connected region, perform spot confidence prediction on each of the aforementioned at least one connected region to obtain a spot region; and based on the aforementioned spot region, obtain the spot information of the reconstructed image.

[0090] As one possible implementation method, when the data processing module 602 determines the light incident angle at each historical time based on the aforementioned light spot information sequence, it is specifically configured to perform the following steps for each historical time: using the intrinsic parameters of the first sensor, determine the light source direction vector in the camera coordinate system based on the light spot information of the aforementioned historical time; using the pre-acquired vehicle body posture information, transform the light source direction vector in the camera coordinate system to the vehicle coordinate system to obtain the light source direction vector in the vehicle coordinate system; and determine the light incident angle at the aforementioned historical time based on the normal vector of the windshield and the light source direction vector in the vehicle coordinate system.

[0091] As one possible implementation, when the data processing module 602 predicts the spot position information and glare intensity information for the next N times based on the aforementioned spot information sequence, motion sensing information, and light incident angles at each historical time, it is specifically configured to use a Kalman filter to predict the spot position information for the next N times based on the aforementioned spot information sequence at each historical time; and to use a glare intensity prediction model to predict the glare intensity information for the next N times based on the aforementioned spot information sequence, motion sensing information, and light incident angles at each historical time.

[0092] As one possible approach, the aforementioned glare intensity prediction model employs a weighted multi-task loss function during training. This weighted multi-task loss function is constructed based on glare intensity accuracy loss and angle smoothing loss. The glare intensity accuracy loss is determined by the difference between the actual glare intensity and the predicted glare intensity, while the angle smoothing loss is determined by the difference between the actual change in incident angle and the predicted change in incident angle.

[0093] As one possible implementation method, when the regional dimming control module 604 performs regional dimming control on the windshield of the vehicle based on the light spot position information and glare intensity information of the N future times, it is specifically configured to perform the following steps, treating each of the N future times as the current time: mapping the light spot position information corresponding to the current time to a sub-region of the windshield to obtain a target glass region, wherein the windshield is a dimmable glass and the windshield is pre-divided into various sub-regions; generating a dimming matrix based on the target glass region and the glare intensity information; and performing regional dimming control on each sub-region of the windshield based on the dimming matrix.

[0094] As one possible approach, when the regional dimming control module 604 performs regional dimming control on each sub-region of the windshield based on the dimming matrix, it is specifically configured to collect the actual light transmittance of each sub-region of the windshield to obtain an actual light transmittance matrix; optimize the dimming matrix based on the actual light transmittance matrix; and perform dimming control on each sub-region of the windshield based on the optimized dimming matrix.

[0095] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0096] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0097] In addition, embodiments of this application also provide an electronic device, including: One or more processors; and a memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any of the foregoing method embodiments.

[0098] Furthermore, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0099] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0100] in, Figure 7 An exemplary architecture of an electronic device is shown, which may include a processor 710, a video display adapter 711, a disk drive 712, an input / output interface 713, a network interface 714, and a memory 720. The processor 710, video display adapter 711, disk drive 712, input / output interface 713, network interface 714, and memory 720 can communicate with each other via a communication bus 730.

[0101] The processor 710 can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs and implement the technical solution provided in this application.

[0102] The memory 720 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 720 can store the operating system 721 for controlling the operation of the electronic device 700, and the basic input / output system (BIOS) 722 for controlling the low-level operations of the electronic device 700. Additionally, it can store a web browser 723, a data storage management system 724, and a vehicle anti-glare dimming device 600, etc. The aforementioned vehicle anti-glare dimming device 600 can be the application program that specifically implements the aforementioned steps in this embodiment. In summary, when the technical solution provided in this application is implemented through software or firmware, the relevant program code is stored in the memory 720 and is called and executed by the processor 710.

[0103] Input / output interface 713 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0104] Network interface 714 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0105] Bus 730 includes a pathway for transmitting information between various components of the device, such as processor 710, video display adapter 711, disk drive 712, input / output interface 713, network interface 714, and memory 720.

[0106] It should be noted that although the above-described device only shows the processor 710, video display adapter 711, disk drive 712, input / output interface 713, network interface 714, memory 720, bus 730, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the solution of this application, and does not necessarily include all the components shown in the figures.

[0107] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer program product. This computer program product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0108] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for adjusting the brightness of a vehicle to prevent glare, characterized in that, The method includes: The vehicle uses a first sensor to acquire a sequence of images of external lighting, and a second sensor to acquire motion sensing information of oncoming vehicles. The image sequence of external lighting is processed by spot detection to obtain a spot information sequence, wherein the spot information sequence includes spot information at each historical time. Based on the light spot information sequence, the light incident angle at each historical time is determined, wherein the light incident angle is the angle between the light emitted from the light source and the normal vector of the windshield of the vehicle; Based on the light spot information sequence, the motion sensing information, and the light incident angle at each historical time, predict the light spot position information and glare intensity information for the next N time periods, where N is a preset positive integer; Based on the light spot position information and glare intensity information for the next N time periods, the windshield of the vehicle is subjected to regional dimming control.

2. The method according to claim 1, characterized in that, The step of performing spot detection processing on the vehicle exterior lighting image sequence to obtain a spot information sequence includes: Based on a preset time window, inter-frame fusion is performed on the illumination image frames located within the same time window in the vehicle exterior illumination image sequence to obtain the reconstructed images of each historical time. The reconstructed images at each historical time are processed by spot detection to obtain a spot information sequence.

3. The method according to claim 2, characterized in that, The reconstructed images at each historical time are subjected to spot detection processing to obtain a spot information sequence, including: Based on each reconstructed image, a brightness saliency map and a normalized gradient magnitude map are generated for the reconstructed image. The brightness value of each pixel in the brightness saliency map represents the probability that the corresponding pixel in the reconstructed image belongs to a strong light source spot. The brightness value of each pixel in the normalized gradient magnitude map represents the gradient change at the corresponding position in the reconstructed image. The brightness saliency map and the normalized gradient magnitude map are weighted and linearly fused to obtain a candidate spot map; The candidate light spot map is used to predict the light spot information of the reconstructed image, wherein the light spot information includes light spot position information and light spot brightness information.

4. The method according to claim 3, characterized in that, The step of predicting the spot information of the reconstructed image based on the spot candidate map includes: The candidate spot image is subjected to threshold-based region segmentation and morphological opening operation to obtain an optimized candidate spot image including at least one connected region. For each of the at least one connected regions, features of the connected region are extracted, wherein the features of the connected region include region area, aspect ratio, average brightness, and center point displacement change rate between adjacent frames. Based on the features of each extracted connected region, spot confidence prediction is performed on each connected region in the at least one connected region to obtain the spot region. Based on the light spot region, the light spot information of the reconstructed image is obtained.

5. The method according to claim 1, characterized in that, The step of determining the incident angle of light at each historical time based on the light spot information sequence includes: For each of the aforementioned historical times, perform the following steps: Using the intrinsic parameters of the first sensor, the light source direction vector in the camera coordinate system is determined based on the light spot information of the historical time. Using the pre-acquired vehicle body posture information, the light source direction vector in the camera coordinate system is transformed to the vehicle body coordinate system to obtain the light source direction vector in the vehicle body coordinate system. Based on the normal vector of the windshield and the light source direction vector in the vehicle coordinate system, the incident angle of light at the historical time is determined.

6. The method according to claim 1, characterized in that, The method of predicting the position information of the light spot and the glare intensity information of the next N time periods based on the light spot information sequence, the motion sensing information, and the light incident angle at each historical time includes: Using a Kalman filter, the position information of the light spot at the next N time points is predicted based on the light spot information sequence at each historical time. Using a glare intensity prediction model, based on the light spot information sequence, the motion sensing information, and the light incident angle at each historical time, the glare intensity information for the next N time periods is predicted.

7. The method according to claim 6, characterized in that, The glare intensity prediction model employs a weighted multi-task loss function during training. This weighted multi-task loss function is constructed based on glare intensity accuracy loss and angle smoothing loss. The glare intensity accuracy loss is determined by the difference between the actual glare intensity and the predicted glare intensity, while the angle smoothing loss is determined by the difference between the actual change in incident angle and the predicted change in incident angle.

8. The method according to any one of claims 1-7, characterized in that, The method of performing regional dimming control on the vehicle's windshield based on the light spot position information and glare intensity information over the next N time periods includes: Treat each of the N future times as the current time and perform the following steps: The current spot position information is mapped to a sub-region of the windshield to obtain the target glass area. The windshield is a dimmable glass and is pre-divided into sub-regions. Based on the target glass area and the glare intensity information, a dimming matrix is ​​generated; Based on the dimming matrix, regional dimming control is performed on each sub-region of the windshield.

9. The method according to claim 8, characterized in that, The step of performing regional dimming control on each sub-region of the windshield based on the dimming matrix includes: The actual light transmittance of each sub-region of the windshield is collected to obtain an actual light transmittance matrix; Based on the actual light transmittance matrix, the dimming matrix is ​​optimized, and based on the optimized dimming matrix, dimming control is performed on each sub-region of the windshield.

10. A vehicle anti-glare dimming device, characterized in that, The device includes: The environmental perception module is configured to acquire a sequence of external lighting images of the vehicle using a first sensor, and to acquire motion sensing information of oncoming vehicles using a second sensor. The data processing module is configured to perform spot detection processing on the vehicle exterior illumination image sequence to obtain a spot information sequence, wherein the spot information sequence includes spot information at each historical time. The data processing module is further configured to determine the light incident angle at each historical time based on the light spot information sequence, wherein the light incident angle is the angle between the light emitted from the light source and the normal vector of the windshield of the vehicle. The data processing module is further configured to predict the position information of the light spot at N future time points based on the light spot information sequence, where N is a preset positive integer; The glare prediction module is configured to predict glare intensity information for the next N times based on the light spot information sequence, the motion sensing information, and the light incident angle at each historical time. The localized dimming control module is configured to perform localized dimming control on the windshield of the vehicle based on the light spot position information and glare intensity information over the next N time periods.

11. An electronic device, characterized in that, include: One or more processors; A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method according to any one of claims 1 to 9.

12. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 9.

Images