Anti-glare traffic sign generation system, device and generation method thereof

Abstract

The present invention provides an anti-glare traffic sign generation system, device and generation method, wherein the generation method of the present invention pre-processes the environmental image data when a vehicle is running, so that the image data used for detection can be accurately identified even in a weak light environment, and then the image data is respectively subjected to traffic sign recognition and screening, and anti-glare target recognition and screening, and finally a traffic sign icon with a shielding frame is generated; and the traffic sign icon with a shielding frame can effectively prevent the projection light from directly projecting onto the retina of pedestrians or other drivers, thereby reducing the sense of glare, and by optimizing the clarity of the projected sign, more convenient and clear driving instructions and driver driving safety and visual protection of other road users are provided.

Description

Technical Field

[0001] The present invention relates to the field of smart lighting, and in particular to a method and system for generating an anti-glare traffic sign. Background Art

[0002] In the daily driving process, drivers often ignore traffic signs and violate traffic regulations. It is not uncommon for traffic accidents to occur because drivers are distracted by checking traffic signs.

[0003] At present, the method to solve the problem of traffic sign impact is mostly to identify the traffic sign through a camera and then project it onto the ground in front of the car to display the traffic sign, and the projection position is exactly at the driver's attention focus point during normal driving, thereby solving the problem of drivers ignoring traffic signs or being distracted by traffic signs. However, this method still has defects. The projection device used to project traffic signs has high power and high brightness. If it is directly irradiated onto the retina of pedestrians or other drivers, it may cause them to feel strong dazzle and even damage their eyesight.

[0004] Therefore, there is an urgent need for a system or control method to solve the above problems. Summary of the invention

[0005] The present invention proposes an anti-glare traffic sign generation method and system, which can achieve partial shielding of the projection device illumination area while improving the problem of unclear projection signs caused by shielding, so that pedestrians or other drivers are not directly illuminated by the light of the projection device. It not only provides convenient and clear driving instructions to ensure the driving safety of drivers, but also avoids adverse effects such as glare on pedestrians or other drivers.

[0006] The present invention is implemented through the following technical solutions. First, a method for generating an anti-glare traffic sign is provided, which includes an image preprocessing step, a traffic sign detection step, an anti-glare target detection step and a traffic sign generation step.

[0007] The image preprocessing step includes: S10: performing image enhancement processing on the acquired image data to obtain enhanced image data;

[0008] The traffic sign detection step includes: S20: detecting the traffic sign icon of each image in the enhanced image data, and obtaining the type and quantity of the traffic sign icon of each image and the confidence of the detection result;

[0009] S30: sequentially determining the confidence of all traffic sign icons in each image, and when the confidence of the traffic sign icon is greater than a first preset threshold, it is considered that the traffic sign icon exists in the corresponding image; sequentially storing all traffic sign icons that meet the confidence condition in each image to obtain a traffic sign icon set for each image;

[0010] The anti-glare target detection step includes S21: detecting the anti-glare target of each image in the enhanced image data, and obtaining the number, type, and position information of the anti-glare target of each image and the confidence of the detection result;

[0011] S31: sequentially determining the confidence of all anti-glare targets in each image, and when the confidence of the anti-glare target is greater than a second preset threshold, it is considered that the anti-glare target exists in the corresponding image; storing the type and position information of all anti-glare targets that meet the confidence condition in each image, and obtaining an anti-glare target set for each image;

[0012] The traffic sign generating step includes: S40: generating a traffic sign icon with a masking frame according to the traffic sign icon set and the anti-glare target set of each image.

[0013] Furthermore, the step S10 includes:

[0014] S11: downsampling the acquired image data by using the nearest neighbor interpolation method, and extracting the brightness channel of each frame of the video stream as its corresponding initial illumination component;

[0015] S12: performing illumination optimization on the initial illumination component of each frame of the image under a low-light condition to obtain an optimized illumination component of each frame of the image;

[0016] S13: Upsample the optimized illumination component of each frame of the image by the nearest neighbor interpolation method to obtain enhanced image data

[0017] Furthermore, the optimized illumination component in step S12 is obtained by the following steps:

[0018] The horizontal component of the structure weight matrix and the vertical component of the structure weight matrix are solved by the following formula:

[0019]

[0020] G σ (x,y)←exp(-dist(x,y) / 2σ 2 )

[0021] Among them, G σ(x, y) represents the Gaussian kernel function with standard deviation σ, dist(x, y) represents the spatial Euclidean distance between pixels x and y, ε is a constant term, and |·| represents the absolute value operation;

[0022] Then, according to the horizontal component of the structure weight matrix, the vertical component of the structure weight matrix and the initial illumination component, the initial illumination component is optimized by the following formula to obtain the optimized illumination component:

[0023]

[0024] Among them, I0 represents the initial illumination component, I represents the optimized illumination component, W is the structural weight matrix, is a 1st-order derivative filter, ||·||1 is the 1-norm, ||·|| F is the standard norm.

[0025] Furthermore, after obtaining the optimized illumination component, the optimized illumination component is corrected, and the correction process is as follows:

[0026] I g (x) = I(x) γ

[0027] γ is the correction coefficient and I(x) is the optimized illumination component.

[0028] Further, step S40 includes: S41: determining whether the anti-glare target in the image where the anti-glare target exists is in the to-be-projected area, and if so, executing step S42;

[0029] S42: Calculating a coordinate transformation matrix according to the relative position relationship between the acquisition device and the projection device;

[0030] S43: performing coordinate transformation on the position information of the anti-glare target according to the coordinate transformation matrix to obtain the position information of the shielding frame;

[0031] S44: Covering a mask frame at a corresponding position on the traffic sign icon according to the position information of the mask frame, thereby generating a traffic sign icon with a mask frame.

[0032] Furthermore, the calculation formula of the coordinate transformation matrix of step S42 is as follows:

[0033] T=R z ·R y ·R x ·T trans

[0034] Among them, R z is the z-axis rotation transformation matrix, R x is the x-axis rotation transformation matrix, R y is the y-axis rotation transformation matrix, Ttrans is the translation transformation matrix.

[0035] Furthermore, in step S44, before covering a mask frame at a corresponding position on the traffic sign icon according to the position information of the mask frame, the method further includes:

[0036] Adjust the opacity of the mask frame based on lighting conditions, vehicle speed, vehicle turning, and the relative distance between the anti-glare target and the vehicle.

[0037] On the other hand, the present invention also provides an anti-glare traffic sign generating device, comprising:

[0038] Image preprocessing module, traffic sign detection module, anti-glare target detection module and traffic sign generation module;

[0039] The image preprocessing module is used to perform image enhancement processing on the acquired image data to obtain enhanced image data;

[0040] The traffic sign detection module includes a traffic sign recognition submodule and a traffic sign confidence determination submodule;

[0041] Traffic sign recognition submodule: used to detect the traffic sign icon of each image in the enhanced image data, and obtain the type and quantity of the traffic sign icon of each image and the confidence of the detection result;

[0042] Traffic sign confidence determination submodule: used to sequentially determine the confidence of all traffic sign icons in each image. When the confidence of the traffic sign icon is greater than a first preset threshold, it is considered that the traffic sign icon exists in the corresponding image; all traffic sign icons that meet the confidence condition in each image are sequentially stored to obtain a traffic sign icon set for each image;

[0043] The anti-glare target detection module includes an anti-glare target recognition submodule and an anti-glare target confidence determination submodule;

[0044] Anti-glare target recognition submodule: used to detect the anti-glare target of each image in the enhanced image data, and obtain the number, type, location information of the anti-glare target of each image and the confidence of the detection result;

[0045] The anti-glare target confidence determination submodule is used to sequentially determine the confidence of all anti-glare targets in each image. When the confidence of the anti-glare target is greater than a second preset threshold, it is considered that the anti-glare target exists in the corresponding image; the type and position information of all anti-glare targets that meet the confidence condition in each image are stored to obtain the anti-glare target set of each image;

[0046] Traffic sign generation module: used to generate a traffic sign icon with a masking frame according to the traffic sign icon set and its anti-glare target set of each image.

[0047] Furthermore, the image preprocessing module includes:

[0048] Downsampling submodule: downsamples the acquired image data through the nearest neighbor interpolation method, and extracts the brightness channel of each frame in the video stream as its corresponding initial illumination component;

[0049] Lighting optimization submodule: optimizes the initial lighting component of each frame under low light conditions to obtain the optimized lighting component of each frame;

[0050] Upsampling submodule: Upsample the optimized illumination components of each frame through the nearest neighbor interpolation method to obtain enhanced image data

[0051] On the other hand, the present invention also provides an anti-glare traffic sign generation system, characterized in that it includes:

[0052] An image acquisition device, a traffic sign generating device as claimed in any one of claims 7 to 8, and a projection device;

[0053] The image acquisition device collects image data of the vehicle's surrounding environment and transmits it to the traffic sign generation device. The traffic sign generation device performs image preprocessing, traffic sign detection, and anti-glare target detection on the received image data in sequence, and finally outputs the generated traffic sign icon to the ground in front of the vehicle's travel direction through a projection device.

[0054] In summary, the present invention pre-processes the vehicle operating environment so that the image data used for detection can be accurately identified even in a low-light environment, and then the image data is respectively identified and screened for traffic signs and anti-glare targets, which can effectively prevent the projected light from directly hitting the retina of pedestrians or other drivers, reduce the glare, and provide more convenient and clear driving instructions by optimizing the clarity of the projected signs. This unique technical feature comprehensively considers the driving safety of drivers and the visual protection of other road users, and provides an innovative solution for the intelligent development of road traffic systems.

[0055] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a structural block diagram of an exemplary anti-glare traffic sign generation system provided by the present invention;

[0057] Figure 2A structural block diagram of an exemplary traffic sign generating device provided by the present invention;

[0058] Figure 3 for Figure 2 A flow chart of a traffic sign generating method executed by the device shown;

[0059] Figure 4 This is a specific execution flow chart of step S10 of the present invention;

[0060] Figure 5 This is a specific execution flow chart of step S40 of the present invention. DETAILED DESCRIPTION

[0061] Adaptive Driving Beam (ADB) technology: is an automotive lighting technology that can automatically adjust the brightness and direction of the vehicle headlights according to traffic conditions and the surrounding environment to ensure the optimization of the lighting effect and avoid dazzling other vehicles. Therefore, the present invention is based on the concept of adaptive high beam technology and uses the control method of adaptive high beam to realize projection logo. However, in actual application, the existing ADB technology may cause the area corresponding to the light source to be in a dark state when the area is turned off. This method of changing the projection light source with the change of the surrounding environment will affect the driver's recognition of the projection logo. Therefore, the system and anti-glare traffic sign generation method of the present invention can improve the problem of unclear projection signs caused by shielding while partially shielding the area illuminated by the projection device, and can ensure the clear visibility of the projection sign while helping pedestrians or other drivers not to be affected by the glare of the projection sign.

[0062] See also Figure 1 The anti-glare traffic sign generation system of the present invention comprises: an image acquisition device, a traffic sign generation device and a projection device. The image acquisition device acquires image data of the vehicle's surrounding environment and transmits it to the traffic sign generation device, which performs image preprocessing, traffic sign detection, and anti-glare target detection on the received image data in sequence, and finally outputs the traffic sign icon to be output to the ground in front of the vehicle's traveling direction through the projection device.

[0063] See also Figure 2 The traffic sign generation device includes an image preprocessing module, a traffic sign detection module, an anti-glare target detection module and a traffic sign generation module. In order to obtain an anti-glare traffic sign icon, please refer to Figure 3 Each module of the traffic sign generating device processes the input image data by the following method:

[0064] The image preprocessing module is used to execute step S10: performing image enhancement processing on the acquired image data to obtain enhanced image data.

[0065] The image data collected by the image acquisition device is transmitted to the traffic sign generation device in the form of a video stream. The video stream is an image set consisting of several frames of continuous images. The number of frames refers to the number of continuous images that exist per unit time. The higher the number of frames, the smoother the video and the more data collected per unit time.

[0066] Retinex theory is based on human visual perception and color constancy in natural scenes. Its basic idea is to decompose the observed image into the product of two parts: illumination component and reflection component. The specific formula is:

[0067]

[0068] Among them, S(x) represents the input original low-light image, I(x) represents the illumination component, R(x) represents the reflection component, x represents the specific pixel, and the operator is an element-wise multiplication operation.

[0069] However, when the natural scenes in the outside world cannot meet the needs of human visual perception, the image data directly incident on the human eye cannot generate a clear image on the human retina. Therefore, the original low-light image must be enhanced first. The Retinex theory enhances the original image data by optimizing the illumination component and the reflection component.

[0070] See also Figure 4 , in the present invention, step S10 includes: S11: downsampling the acquired image data by nearest neighbor interpolation method, and extracting the brightness channel of each frame of the picture in the video stream as its corresponding initial illumination component;

[0071] After downsampling by the nearest neighbor interpolation method, the image is transferred from the RGB color mode (RGB) color space to the HSV color model (HSV) color space, and its brightness channel V is taken as the initial illumination component. In order to highlight the edge structure information of the image, local blocks are introduced in the process of extracting the brightness channel. The processing formula is as follows:

[0072] I0(x)=S V (x)

[0073]

[0074] i=round(i' / r1)

[0075] j = round(j' / r1)

[0076] K(i',j')=K0(i,j)

[0077] Among them, I0(x) represents the initial illumination component, c contains different color channels, S V (x) represents the V component of the image in the HSV color space, r1 is the downsampling ratio, is the illumination weight matrix, Ω represents the local block centered on x, whose size is 3x3, and L is the illumination adjustment parameter. i and j are the indices of the image pixels, K is the downsampled image, and K0 is the original image before downsampling.

[0078] Through the above processing, adaptive illumination initialization of the original image data is achieved, which solves the problem that the traditional 3-color channel average method ignores the range prior of the illumination component, that is, the brightness of the illumination component should not be less than the original image, so that the later output image can adapt to human eye recognition.

[0079] S12: performing illumination optimization on the initial illumination component of each frame of the image under a low-light condition to obtain an optimized illumination component of each frame of the image;

[0080] The ideal illumination component optimization algorithm is to smooth the spatial texture as much as possible while maintaining the edge structure, and the detailed information of such images should be reflected in the reflection component. According to the Retinex theoretical model, the accurate estimation of the illumination component will directly affect the reflection component. Therefore, the optimization of the illumination component in the present invention is achieved through the following objective function:

[0081]

[0082] Among them, I0 represents the initial illumination component, I represents the optimized illumination component, W is the structural weight matrix, is a 1st-order derivative filter, ||·||1 is the 1-norm, ||·|| F is the standard norm.

[0083] In the above formula, the optimized illumination component I is obtained by minimizing the objective function: is the data fidelity term, used to constrain the difference between and , the regular term is used to limit the scope of the solution space, and α is used to balance the fidelity term and the regularization term.

[0084] In addition, the structural weight matrix W includes two parts in the horizontal direction and the vertical direction:

[0085]

[0086] G σ(x,y)←exp(-dist(x,y) / 2σ 2 )

[0087] Among them, G σ (x, y) represents the Gaussian kernel function with standard deviation σ, dist(x, y) represents the spatial Euclidean distance between pixels x and y, ε is a constant term, and |·| represents the absolute value operation.

[0088] S13: Up-sampling the optimized illumination component of each frame of the image by the nearest neighbor interpolation method to obtain enhanced image data.

[0089] For low-light images, improving image brightness is one of the core issues to be solved. Therefore, before outputting the enhancement result, the present invention also performs gamma correction on the obtained optimized illumination component to achieve nonlinear adjustment of image brightness. To further improve the enhancement result, the optimized illumination component is corrected to obtain the corrected illumination component. The specific formula is:

[0090] I g (x) = I(x) γ

[0091] γ is the correction coefficient, and I(x) is the optimized illumination component. The value of γ can be adjusted according to the actual situation, and is 0.8 in the present invention.

[0092] The original image S(x) is transformed into the corrected illumination I g Make the changes and correct the formula as follows:

[0093]

[0094] Where ε1 is a constant term and r2 is the upsampling ratio.

[0095] Finally, the output image is upsampled by nearest neighbor interpolation to restore the image size and obtain enhanced image data, which is then input into the traffic sign detection module and the anti-glare target detection module for traffic sign recognition and anti-glare target recognition. The traffic sign detection module includes a traffic sign recognition submodule and a traffic sign confidence determination submodule.

[0096] The traffic sign recognition submodule is used to execute step S20: detect the traffic sign icon of each image in the enhanced image data, and obtain the type and quantity of the traffic sign icon of each image and the confidence of the detection result.

[0097] The processed image is analyzed by a traffic sign detection module, wherein the module may adopt a deep learning algorithm such as the YOLO series, or other appropriate algorithms for traffic sign detection, and may screen out whether there are traffic signs in each enhanced image, and analyze the type and quantity of traffic signs on the road ahead of the vehicle displayed in each image in the image data, as well as the confidence level of the detection result.

[0098] The traffic sign confidence determination submodule is used to execute step S30: sequentially determine the confidence of all traffic sign icons in each image, and when the confidence of a traffic sign icon is greater than a first preset threshold, it is considered that the traffic sign icon exists in the corresponding image; all traffic sign icons that meet the confidence condition in each image are sequentially stored to obtain a traffic sign icon set for each image;

[0099] Since a video stream is collected during the driving process of the vehicle, the image data is a group of continuous images. In the process of traffic sign detection, traffic sign recognition is performed on each image data in sequence to complete the real-time detection of traffic signs. For each image, when there is a suspected traffic sign, the confidence that the suspected traffic sign may be a traffic sign is calculated by comparing the suspected traffic sign with a standard traffic sign, and whether it is indeed a traffic sign is determined based on the size of the confidence. In the present invention, the first preset threshold is set to 0.75, that is, if the confidence of the detected traffic sign is higher than 0.75, it is considered that there is a traffic sign in the image, and the icon corresponding to the traffic sign is output to the traffic sign generation module; if there are multiple traffic signs in the image at the same time, and their detection confidences are all higher than 0.75, the icons corresponding to each traffic sign are output in sequence in a certain order and time interval.

[0100] At the same time, the anti-glare target detection module also determines whether there is an anti-glare target in the projection area according to the enhanced image data. The anti-glare target detection module includes an anti-glare target recognition submodule and an anti-glare target confidence determination submodule.

[0101] The anti-glare target recognition submodule is used to execute step S21: detect the anti-glare target of each image in the enhanced image data, and obtain the number, type, position information of the anti-glare target of each image and the confidence of the detection result.

[0102] While performing traffic sign detection on the enhanced image data, anti-glare target detection is also performed on the enhanced image data to obtain the number, type and confidence of the anti-glare targets. However, not all anti-glare targets in all images need to be processed. Only anti-glare targets located in the projection area of ​​the projection device will affect the projection of our traffic sign icons. Therefore, the location information of suspected anti-glare targets also needs to be identified.

[0103] The anti-glare target confidence determination submodule is used to execute step S31: sequentially determine the confidence of all anti-glare targets in each image, and when the confidence of the anti-glare target is greater than a second preset threshold, it is considered that the anti-glare target exists in the corresponding image; the type and position information of all anti-glare targets that meet the confidence conditions in each image are stored to obtain an anti-glare target set for each image.

[0104] If the confidence of the detected anti-glare target is higher than 0.75, its position information is output to the traffic sign generation module; if multiple anti-glare targets are detected at the same time and their detection confidences are all higher than 0.75, their position information is output to the traffic sign generation module at the same time.

[0105] The traffic sign generating module is used to execute step S40: generating a traffic sign icon with a masking frame according to the traffic sign icon set and the anti-glare target set of each image.

[0106] Wherein, S40 specifically includes S41: determining whether the anti-glare target in the image where the anti-glare target exists is in the to-be-projected area, and if so, executing step S42;

[0107] For any image, there may be traffic sign icons and anti-glare targets. The traffic sign images and their quantity around the vehicle are determined based on the traffic sign icon set of each image, and whether the anti-glare target is within the illumination range of the projection device is determined based on the position information of the anti-glare target set. If there is no anti-glare target within the illumination range of the projection device, the projection icon will not be operated. Otherwise, the projection icon of the projection device will be partially shielded, and the position of the shielding frame will be based on the position information of the anti-glare target set.

[0108] S42: Calculating a coordinate transformation matrix according to the relative position relationship between the acquisition device and the projection device;

[0109] Confirm it through the following steps. First, determine the coordinate transformation matrix T. If the relative translation vector between the camera and the projection optical machine is (t x ,t y ,t z ), the relative rotation angle is (θ x ,θy ,θ z ), then the translation transformation matrix is:

[0110]

[0111] The rotation transformation matrix around the x-axis is:

[0112]

[0113] The rotation transformation matrix around the y-axis is:

[0114]

[0115] The rotation transformation matrix around the z-axis is:

[0116]

[0117] Get the sum transformation matrix:

[0118] T=R z ·R y ·R x ·T trans

[0119] S43: performing coordinate transformation on the position information of the anti-glare target according to the coordinate transformation matrix to obtain the position information of the shielding frame;

[0120] According to the obtained transformation matrix T, if the positions of the target frame corner points obtained by the anti-glare target detection module are (x left ,t left ) and (x right ,y right ), then the coordinates of the two corner points of the masking box can be determined by the following formula:

[0121]

[0122] Thus, the positions of two corner points of the masking frame on the traffic sign to be projected are obtained, and the position of the masking frame is determined based on the position information of the existing anti-glare target.

[0123] S44: Covering a mask frame at a corresponding position on the traffic sign icon according to the position information of the mask frame, thereby generating a traffic sign icon with a mask frame.

[0124] By adding the shielding frame, the projected traffic sign icon will not directly hit the anti-glare target, thus completing the avoidance of the projected icon. The shielding frame is initially a shielding frame with an opacity of 100%.

[0125] In a preferred embodiment, before generating the traffic sign icon with the mask frame, the method further includes:

[0126] Adjust the opacity of the mask frame based on lighting conditions, vehicle speed, vehicle turning, and the relative distance between the anti-glare target and the vehicle.

[0127] The settings in a practical application are as follows: (1) If the distance between the vehicle and the anti-glare target is greater than 100m, the opacity of the shielding frame is set to 1.10, otherwise the opacity of the shielding frame is set to 0.30. That is, when the distance is far, the opacity of the shielding frame can be set to a lower level, and the projected traffic icons can be clearer. When the distance is close, the opacity of the shielding frame needs to be set to a higher level to prevent the projected traffic icons from dazzling pedestrians or other vehicles.

[0128] (2) If the vehicle speed is greater than 50 km / h, the opacity of the shielding frame is set to 1.20, otherwise the opacity of the shielding frame is set to 0.50. That is, when the vehicle speed is high, the opacity of the shielding frame needs to be set higher, because the distance between the vehicle and the anti-glare target will be closer at a higher speed, and the shielding area needs to be preset to a lower brightness to prevent glare. When the vehicle speed is low, the opacity of the shielding frame can be set to a lower level.

[0129] (3) The position of the shielding frame is fine-tuned according to the current turning angle signal of the vehicle. If the vehicle is currently in a straight-moving state, there is no need to adjust the position of the shielding frame. If the vehicle is in a left-turning state, the shielding frame needs to be fine-tuned to the right. If the vehicle is in a right-turning state, the shielding frame needs to be fine-tuned to the left to prevent the shielding frame from failing to cover the anti-glare target position when the vehicle is turning.

[0130] (4) If the ambient illuminance is greater than 25 lux, the opacity of the shielding frame is set to 0.70, otherwise it is set to 1.30. That is, when the ambient brightness is brighter, the projected traffic signs can be brighter and clearer, while when the ambient brightness is lower, the projected traffic signs in the shielding frame area must maintain a lower brightness, otherwise it may cause dazzle.

[0131] The anti-glare traffic sign generation system of the present invention is applicable to various types of vehicles, including but not limited to private cars, commercial vehicles and self-driving vehicles. The system can be flexibly applied to various road environments such as urban roads, highways and rural roads. This embodiment takes the application of the system on a certain family SUV as an example, and the specific implementation is as follows:

[0132] 1. Use a low-light CMOS camera and install it at the front of the car to ensure that it has sufficient vision in the direction of the car's travel. It can also obtain image information from the front camera of the car by reading the vehicle's CAN signal. This enables the sensor to collect high-quality images of the surrounding environment in real time at night or in poor lighting conditions.

[0133] 2. For the video acquired from the camera, the Retinex image enhancement algorithm is used to process the low-light image in real time by extracting frames. By analyzing the input image, the corresponding illumination weight matrix is ​​generated to guide the adaptive initialization of the illumination component. Next, under the constraint of structured illumination, the initial illumination component is optimized and smoothed by the introduced equivalent objective function. Subsequently, nonlinear illumination adjustment is performed, and finally the Retinex theory is combined to achieve comprehensive enhancement of the low-light image.

[0134] 3. In the traffic sign detection stage, the YOLOv4 (You Only Look Once version 4) deep learning algorithm is used. YOLOv4 has high detection accuracy and real-time performance, and is suitable for application scenarios of real-time traffic sign detection. First, the YOLOv4 model is pre-trained using the traffic sign dataset TT100k. Then, the pre-trained model is combined with a specific traffic sign dataset and fine-tuned to improve the detection capability of traffic signs. The pre-trained and fine-tuned YOLOv4 model is deployed to the industrial computer system. The trained weight file is loaded into the model, and the model is inferred through the deep learning framework PyTorch.

[0135] 4. In the anti-glare target detection stage, the Faster R-CNN (Region-based Convolutional Neural Network) target detection algorithm is used. First, through the data set of night environment, Faster R-CNN can be trained by focusing on the image samples of vehicles and pedestrians in the training set, which helps the model better identify anti-glare targets in night environment. The trained Faster R-CNN model is deployed to the system, the trained weight file is loaded, and real-time target detection is performed through the deep learning framework PyTorch. The model detects other vehicles and pedestrians in the video stream in real time, and obtains their location information, type and detection confidence. This information is passed to the traffic sign generation module for subsequent anti-glare projection adjustment.

[0136] 5. Analyze the vehicle and pedestrian position information obtained in the anti-glare target detection stage to determine whether their positions are within the projection light machine illumination range. If there is an anti-glare target within the projection light machine illumination range, a mask frame needs to be added to the corresponding position of the projected traffic sign icon. The transparency of the mask frame needs to be determined according to the lighting conditions of the current vehicle environment, the vehicle speed, the vehicle steering situation, and the relative distance between the anti-glare target and the vehicle. For example, if the distance between the target and the vehicle is greater than 100 meters, the opacity of the mask frame is 100%*0.30=30%. If the vehicle speed at this moment is 70km / h>50km / h, the opacity of the mask frame is 30%*1.20=36%. If the vehicle is going straight, there is no need to adjust the position of the mask frame. If the ambient illumination is 10lux, the opacity of the mask frame is 36%*1.30=47%, so the opacity of the mask frame is finally set to 47%, and the position is not adjusted. The projection icon that has been anti-glare processed is transmitted to the projection module.

[0137] 6. The projection light machine projects the projection icon onto the ground in front of the vehicle.

[0138] The anti-glare traffic sign generation method of the present invention pre-processes the vehicle operating environment so that the image data used for detection can be accurately identified even in a low-light environment, and then the image data is respectively identified and screened for traffic signs, and identified and screened for anti-glare targets, which can effectively prevent the projection light from directly hitting the retina of pedestrians or other drivers, reduce the sense of glare, and provide more convenient and clear driving instructions by optimizing the clarity of the projection signs. This unique technical feature comprehensively considers the driving safety of drivers and the visual protection of other road users, and provides an innovative solution for the intelligent development of road traffic systems. The traffic sign detection system of the present invention is expected to reduce urban traffic congestion while reducing traffic accident rates and improving road traffic efficiency, and make positive contributions to urban traffic management and road safety. Through technological innovation, the present invention provides a feasible solution to the problem of visual interference during driving, which has broad application prospects and social significance.

[0139] Finally, it should be noted that the anti-glare traffic sign generation system, device and generation method thereof disclosed in the embodiments of the present invention disclose only the preferred embodiments of the present invention and are only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solution described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solution from the spirit and scope of the technical solution of the embodiments of the present invention.

Claims

1. A method for generating an anti-glare traffic sign, characterized in that: It includes an image preprocessing step, a traffic sign detection step, an anti-glare target detection step, and a traffic sign generation step; The image preprocessing step includes: S10: performing image enhancement processing on the acquired image data to obtain enhanced image data; The traffic sign detection step includes: S20: detecting the traffic sign icon of each image in the enhanced image data, and obtaining the type and quantity of the traffic sign icon of each image and the confidence of the detection result; S30: sequentially determining the confidence of all traffic sign icons in each image, and when the confidence of the traffic sign icon is greater than a first preset threshold, it is considered that the traffic sign icon exists in the corresponding image; sequentially storing all traffic sign icons that meet the confidence condition in each image to obtain a traffic sign icon set for each image; The anti-glare target detection step includes S21: detecting the anti-glare target of each image in the enhanced image data, and obtaining the number, type, and position information of the anti-glare target of each image and the confidence of the detection result; S31: sequentially determining the confidence of all anti-glare targets in each image, and when the confidence of the anti-glare target is greater than a second preset threshold, it is considered that the anti-glare target exists in the corresponding image; storing the type and position information of all anti-glare targets that meet the confidence condition in each image, and obtaining an anti-glare target set for each image; The traffic sign generating step includes: S40: generating a traffic sign icon with a masking frame according to the traffic sign icon set and the anti-glare target set of each image.

2. The anti-glare traffic sign generation method according to claim 1, characterized in that: The step S10 comprises: S11: downsampling the acquired image data by using the nearest neighbor interpolation method, and extracting the brightness channel of each frame of the video stream as its corresponding initial illumination component; S12: performing illumination optimization on the initial illumination component of each frame of the image under a low-light condition to obtain an optimized illumination component of each frame of the image; S13: Up-sampling the optimized illumination component of each frame of the image by the nearest neighbor interpolation method to obtain enhanced image data.

3. The anti-glare traffic sign generation method according to claim 2, characterized in that: The optimized illumination component in step S12 is obtained by the following steps: The horizontal component of the structure weight matrix and the vertical component of the structure weight matrix are solved by the following formula: G σ (x,y)←exp(-dist(x,y) / 2σ 2 ) in, Indicates the pixel x All pixels in the local block Ω(x) centered at y Perform the sum operation, G σ (x,y) represents the Gaussian kernel function with standard deviation σ, Represents the first-order derivative of the initial illumination component I0 in the horizontal direction at pixel y, represents the first-order derivative of the initial illumination component I0 in the vertical direction at pixel y; dist(x,y) represents the spatial Euclidean distance between pixels x and y, ε is a constant term, and |·| represents the absolute value operation; Then, according to the horizontal component of the structure weight matrix, the vertical component of the structure weight matrix and the initial illumination component, the initial illumination component is optimized by the following formula to obtain the optimized illumination component: Among them, I0 represents the initial illumination component, I represents the optimized illumination component, W is the structural weight matrix, is a 1st-order derivative filter, ||·||1 is the 1-norm, ||·|| F is the standard norm, is the data fidelity item, is the regularization term, and α is used to balance the fidelity term and the regularization term.

4. The anti-glare traffic sign generation method according to claim 3, characterized in that: After obtaining the optimized illumination component, the optimized illumination component is corrected, and the correction process is as follows: I g (x)=I(x) γ γ is the correction coefficient and I(x) is the optimized illumination component.

5. The anti-glare traffic sign generation method according to claim 4, characterized in that: Step S40 includes: S41: determining whether the anti-glare target in the image where the anti-glare target exists is in the area to be projected, and if so, executing step S42; S42: Calculating a coordinate transformation matrix according to the relative position relationship between the acquisition device and the projection device; S43: performing coordinate transformation on the position information of the anti-glare target according to the coordinate transformation matrix to obtain the position information of the shielding frame; S44: Covering a mask frame at a corresponding position on the traffic sign icon according to the position information of the mask frame, thereby generating a traffic sign icon with a mask frame.

6. The anti-glare traffic sign generation method according to claim 5, characterized in that: The calculation formula of the coordinate transformation matrix of step S42 is as follows: T=R z ·R y ·R x ·T trans Among them, R z is the z-axis rotation transformation matrix, R x is the x-axis rotation transformation matrix, R y is the y-axis rotation transformation matrix, T trans is the translation transformation matrix.

7. The anti-glare traffic sign generation method according to claim 5, characterized in that: In step S44, before covering a mask frame at a corresponding position on the traffic sign icon according to the position information of the mask frame, the method further includes: Adjust the opacity of the mask frame based on lighting conditions, vehicle speed, vehicle turning, and the relative distance between the anti-glare target and the vehicle.

8. An anti-glare traffic sign generating device, characterized in that: include: Image preprocessing module, traffic sign detection module, anti-glare target detection module and traffic sign generation module; The image preprocessing module is used to perform image enhancement processing on the acquired image data to obtain enhanced image data; The traffic sign detection module includes a traffic sign recognition submodule and a traffic sign confidence determination submodule; Traffic sign recognition submodule: used to detect the traffic sign icon of each image in the enhanced image data, and obtain the type and quantity of the traffic sign icon of each image and the confidence of the detection result; Traffic sign confidence determination submodule: used to sequentially determine the confidence of all traffic sign icons in each image. When the confidence of the traffic sign icon is greater than a first preset threshold, it is considered that the traffic sign icon exists in the corresponding image; all traffic sign icons that meet the confidence condition in each image are sequentially stored to obtain a traffic sign icon set for each image; The anti-glare target detection module includes an anti-glare target recognition submodule and an anti-glare target confidence determination submodule; Anti-glare target recognition submodule: used to detect the anti-glare target of each image in the enhanced image data, and obtain the number, type, location information of the anti-glare target of each image and the confidence of the detection result; The anti-glare target confidence determination submodule is used to sequentially determine the confidence of all anti-glare targets in each image. When the confidence of the anti-glare target is greater than a second preset threshold, it is considered that the anti-glare target exists in the corresponding image; the type and position information of all anti-glare targets that meet the confidence condition in each image are stored to obtain the anti-glare target set of each image; Traffic sign generation module: used to generate a traffic sign icon with a masking frame according to the traffic sign icon set and its anti-glare target set of each image.

9. The anti-glare traffic sign generating device according to claim 8, characterized in that: The image preprocessing module comprises: Downsampling submodule: downsamples the acquired image data through the nearest neighbor interpolation method, and extracts the brightness channel of each frame in the video stream as its corresponding initial illumination component; Lighting optimization submodule: optimizes the initial lighting component of each frame under low light conditions to obtain the optimized lighting component of each frame; Upsampling submodule: Upsample the optimized illumination component of each frame of the image through the nearest neighbor interpolation method to obtain enhanced image data.

10. An anti-glare traffic sign generation system, characterized in that: include: An image acquisition device, a traffic sign generating device as claimed in any one of claims 8 to 9, and a projection device; The image acquisition device collects image data of the vehicle's surrounding environment and transmits it to the traffic sign generation device. The traffic sign generation device performs image preprocessing, traffic sign detection, and anti-glare target detection on the received image data in sequence, and finally outputs the generated traffic sign icon to the ground in front of the vehicle's travel direction through a projection device.

Images