Methods, electronic devices, and storage media for detecting violations related to vehicle lights being retrofitted.

Abstract

This application discloses a method, electronic device, and storage medium for detecting violations based on vehicle lights. The method involves acquiring an initial vehicle image in which the lights are on; inputting the initial vehicle image into a preset ambient light restoration network, and using a maximum reflectance chromaticity attention mechanism within the ambient light restoration network to extract chromaticity features of the initial vehicle image, obtaining road surface reflectance chromaticity features; using a chromaticity feature suppression module within the ambient light restoration network to eliminate road surface reflectance light from the initial vehicle image and the road surface reflectance features, obtaining a chromaticity suppressed image; inputting the chromaticity suppressed image and a depth image of the initial vehicle image into a preset headlight restoration network for headlight halo elimination, obtaining a target vehicle image; and performing violation detection processing on the target vehicle image to obtain the vehicle violation detection result. This improves the accuracy of violation detection.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, electronic device and storage medium for detecting violations based on vehicle lights. Background Technology

[0002] With the rapid development of technology, vehicles have become the primary means of transportation. However, in recent years, traffic accidents caused by vehicles have also been increasing. Especially at night when lighting conditions are insufficient, some vehicles install larger-than-normal headlights, known as bullseye lights, to evade toll payments. These bullseye lights produce a large glare, making license plate numbers unclear in camera images and hindering the accurate identification of offending vehicles. Furthermore, the light from these bullseye lights can reduce driver perception, causing temporary blindness and endangering life and property.

[0003] Current violation detection methods typically involve denoising or calibrating vehicle images, followed by target detection on the processed images to determine if a vehicle is violating regulations. This approach fails to account for the impact of other lighting conditions on the vehicle image during capture, resulting in a less clear image and consequently affecting the accuracy of violation detection. Summary of the Invention

[0004] The main technical problem addressed by this application is to provide a method, electronic device, and storage medium for detecting violations based on the installation of vehicle lights, which can improve the accuracy of violation detection.

[0005] To address the aforementioned technical problems, this application provides a method for detecting violations based on vehicle lights, comprising: acquiring an initial vehicle image in which the lights are on; inputting the initial vehicle image into a preset ambient light restoration network, and performing chromaticity feature extraction processing on the initial vehicle image through a maximum reflectance chromaticity attention mechanism in the ambient light restoration network to obtain road surface reflectance chromaticity features; performing road surface reflectance light elimination processing on the initial vehicle image and the road surface reflectance chromaticity features through a chromaticity feature suppression module in the ambient light restoration network to obtain a reflectance chromaticity suppressed image; inputting the reflectance chromaticity suppressed image and the depth image of the initial vehicle image into a preset headlight restoration network for headlight halo elimination processing to obtain a target vehicle image; and performing violation detection processing on the target vehicle image to obtain a vehicle violation detection result.

[0006] In one embodiment, the step of extracting chromaticity features from the initial vehicle image using the maximum reflectance chromaticity attention mechanism in the ambient light restoration network to obtain road surface reflectance chromaticity features includes: performing frequency division processing on the initial vehicle image to obtain a second frequency image, wherein the image frequency of the second frequency image is less than or equal to a preset cutoff frequency; performing feature extraction processing on the second frequency image to obtain attention features of the second frequency image; performing reflectance chromaticity downsampling processing on the attention features to obtain downsampled image features; and performing normalization processing on the downsampled image features to obtain road surface reflectance chromaticity features.

[0007] In one embodiment, the step of performing road reflection light elimination processing on the initial vehicle image and the road surface reflection chromaticity features through the chromaticity feature suppression module in the ambient light restoration network to obtain a reflection chromaticity suppressed image includes: performing frequency division processing on the initial vehicle image to obtain a first frequency image and a second frequency image, wherein the image frequency of the first frequency image is greater than a preset cutoff frequency, and the image frequency of the second frequency image is less than or equal to the preset cutoff frequency; performing frequency feature extraction processing on the first frequency image to obtain a first frequency feature; performing reflection suppression processing based on the second frequency image and the road surface reflection chromaticity features to obtain a suppressed reflection chromaticity feature; and performing image reconstruction processing based on the first frequency feature and the suppressed reflection chromaticity feature to obtain the reflection chromaticity suppressed image.

[0008] In one embodiment, the step of performing reflection suppression processing based on the second frequency image and the road surface reflection chromaticity features to obtain suppressed reflection chromaticity features includes: performing frequency feature extraction processing on the second frequency image to obtain second frequency features; and performing correction processing based on the second frequency features and the road surface reflection chromaticity features to obtain the suppressed reflection chromaticity features.

[0009] In one embodiment, the step of performing image reconstruction processing based on the first frequency feature and the suppressed reflectance chromaticity feature to obtain the chromaticity suppressed reflectance image includes: merging the first frequency feature and the suppressed reflectance chromaticity feature to obtain merged features; and encoding / decoding the merged features to obtain the chromaticity suppressed reflectance image.

[0010] In one embodiment, the step of inputting the reflection chromaticity suppression image and the depth image of the initial vehicle image into a preset headlight restoration network for headlight halo removal processing to obtain a target vehicle image includes: performing headlight feature extraction processing on the reflection chromaticity suppression image through the headlight restoration network to obtain headlight features; performing fusion processing on the image features of the reflection chromaticity suppression image and the headlight features to obtain fused features; and performing image reconstruction processing based on the fused features and the depth image to obtain the target vehicle image.

[0011] In one embodiment, the step of extracting vehicle headlight features from the reflection chromaticity suppression image using the vehicle headlight restoration network to obtain vehicle headlight features includes: segmenting the feature map of the reflection chromaticity suppression image using the vehicle headlight restoration network to obtain multiple sub-feature maps; performing convolution processing on each sub-feature map to obtain convolutional features of each sub-feature map; stacking and fusing adjacent convolutional features to obtain fused convolutional features; performing downsampling convolution processing on the fused convolutional features to obtain downsampling features; and determining the vehicle headlight features based on the downsampling features and the fused convolutional features.

[0012] In one embodiment, the step of performing violation detection processing on the target vehicle image to obtain a vehicle violation detection result includes: performing target detection processing on the target vehicle image to obtain multiple detection objects; determining whether the vehicle has installed illegal headlights based on a comparison result between the types of the multiple detection objects and a preset illegal headlight type, and obtaining a headlight detection result; in response to the headlight detection result indicating that the vehicle has installed illegal headlights, determining the vehicle's license plate number from the multiple detection objects; and determining the headlight detection result and the license plate number as the vehicle violation detection result.

[0013] To address the aforementioned technical problems, this application provides an electronic device, including a memory and a processor. The memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the aforementioned method for detecting violations based on the installation of vehicle lights.

[0014] To address the aforementioned technical problems, this application provides a computer-readable storage medium, comprising: storing program data, wherein the program data, when executed by a processor, is used to implement the aforementioned method for detecting violations based on the installation of vehicle lights.

[0015] The above scheme involves acquiring an initial vehicle image with the headlights on; inputting this image into a pre-defined ambient light restoration network, where a maximum reflectance chromaticity attention mechanism extracts chromaticity features to obtain road surface reflectance features; then, a chromaticity feature suppression module within the network performs road surface reflectance reduction on both the initial vehicle image and the road surface reflectance features, resulting in a chromaticity-suppressed image; finally, the chromaticity-suppressed image and a depth image of the initial vehicle image are input into a pre-defined headlight restoration network for headlight halo reduction to obtain the target vehicle image; and finally, the target vehicle image is subjected to violation detection processing to obtain the violation detection result. Therefore, by performing road surface reflectance reduction and headlight halo reduction on the initial vehicle image, the resulting target vehicle image eliminates the influence of headlights, improves image clarity, and thus enhances the accuracy of vehicle violation detection. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein: Figure 1 This is a schematic flowchart of an exemplary embodiment of the violation detection method based on vehicle lights installed in this application; Figure 2 This is a schematic diagram of an exemplary embodiment of the initial vehicle image shown in this application; Figure 3 This is a schematic diagram of a physical model of a light source imaging according to an exemplary embodiment shown in this application; Figure 4 This is a schematic diagram of an exemplary embodiment of the target vehicle image shown in this application; Figure 5 This is a schematic diagram of an exemplary embodiment of the maximum reflectance chromaticity attention mechanism shown in this application; Figure 6 This is a schematic diagram of an exemplary embodiment of the ambient light restoration network shown in this application; Figure 7 This is a schematic diagram of an exemplary embodiment of the vehicle headlight restoration network shown in this application; Figure 8 yes Figure 7 A schematic diagram of an exemplary embodiment of the vehicle headlight feature extraction processing in a vehicle headlight restoration network is shown. Figure 9This is a block diagram illustrating a violation detection device based on vehicle lights installed on a vehicle, as shown in an exemplary embodiment of this application; Figure 10 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application; Figure 11 This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It is understood that the specific embodiments described herein are only for explaining this application and not for limiting it. Furthermore, it should be noted that, for ease of description, only the parts related to this application are shown in the accompanying drawings, not all structures. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] First, it's important to note that with rapid technological advancements, vehicles have become the primary mode of transportation. However, in recent years, traffic accidents caused by vehicles have also increased. Especially at night in low-light conditions, some vehicles, in an attempt to evade tolls, install larger-than-normal headlights, known as "bullet-eye" lights. These bullseye lights produce excessive glare, making license plates unclear in camera images and hindering accurate identification of violating vehicles. Furthermore, the glare from these bullseye lights can reduce driver perception, causing temporary blindness and endangering life and property. Current violation detection methods typically involve denoising or calibrating vehicle images before performing target detection on the processed image to determine if a violation has occurred. This approach fails to consider the impact of other lights on the vehicle image during capture, resulting in unclear images and consequently affecting the accuracy of violation detection.

[0019] Based on this, this application provides a method for detecting violations related to vehicle-mounted lights, an electronic device, and a computer-readable storage medium. For details, please refer to [link / reference needed]. Figure 1 , Figure 1 This is a schematic flowchart of an exemplary embodiment of a method for detecting violations based on the installation of vehicle lights, as shown in this application.

[0020] The execution entity of a violation detection method based on vehicle-mounted lights can be a terminal device, a server, or other processing device. The terminal device can be a computer, mobile device, terminal, computing device, in-vehicle device, etc. The execution entity of this violation detection method can also be a violation detection device based on vehicle-mounted lights. In some possible implementations, this violation detection method based on vehicle-mounted lights can be implemented by a processor calling computer-readable instructions stored in memory. The execution entity of this violation detection method based on vehicle-mounted lights can also be a big data cluster. A big data cluster is a computer system architecture formed by multiple computers connected through a network. The big data cluster can be deployed on a private cloud built with K8S (Kubernetes, a container orchestration engine).

[0021] Specifically, one method for detecting violations based on vehicle lights installed in this embodiment includes the following steps: Step S110: Obtain an initial vehicle image. The headlights in the initial vehicle image are in the on state.

[0022] The initial vehicle image refers to an image that has not undergone road surface reflection removal processing and headlight halo removal processing. The initial vehicle image includes the vehicle. The vehicle's headlights are on.

[0023] An initial vehicle image is obtained using a violation detection device based on vehicle lights. As one example, the device photographs a vehicle on the road to obtain an initial vehicle image. As another example, the device photographs a vehicle on the road to obtain a captured image; license plate recognition is performed on the captured image, and if recognition fails, the corresponding captured image is designated as the initial vehicle image. It should be noted that if a vehicle is equipped with illegal overhead lights ("bulge lights"), the light from these lights will obscure the location of the license plate, making it difficult to accurately identify the license plate in the captured image. This embodiment uses such captured images as the initial vehicle image, thereby enabling processing of the initial vehicle image to improve image clarity and achieve license plate recognition.

[0024] As another example, a violation detection device based on vehicle-mounted lights triggers a flash with a preset illumination level upon the arrival of a preset shooting time. At the moment the flash is activated, vehicles within a preset acquisition area are photographed to obtain an initial vehicle image. The flash with the preset illumination level is a low-intensity flash. This example, by activating the low-intensity flash at the shooting time, reduces the intensity of bullseye lights, achieving initial suppression of bullseye lights and improving the clarity of bullseye lights and license plate textures in the initial vehicle image, thereby enhancing overall image clarity.

[0025] Step S120: Input the initial vehicle image into the preset ambient light restoration network, and use the maximum reflectance chromaticity attention mechanism in the ambient light restoration network to extract chromaticity features from the initial vehicle image to obtain road surface reflectance chromaticity features.

[0026] Ambient light restoration network refers to a neural network used to eliminate road surface reflections in images.

[0027] The maximum reflectance chromaticity attention mechanism refers to extracting the maximum reflectance chromaticity from an image based on the physical model characteristics of maximum reflectance chromaticity.

[0028] Road surface reflection chromaticity characteristics refer to the impact of road surface reflected light on vehicle images. Specifically, road surface reflected light refers to the strong reflected light caused by the road surface reflecting off the vehicle's low beam headlights onto the imaging device.

[0029] The vehicle violation detection device based on vehicle-mounted lights inputs an initial vehicle image into a low-pass filter to obtain a second-frequency image. The second-frequency image is then segmented using a classical projection space segmentation method to obtain the maximum reflectance chromaticity feature. This maximum reflectance chromaticity feature is then identified as the road surface reflectance chromaticity feature. For example, the device calculates the maximum chromaticity of pixels in the second-frequency image; it constructs a classical projection space by combining the coefficients of variation of RGB (Red, Green, Blue) values; it analyzes the distribution patterns and projection characteristics of diffuse and specular reflection components within this classical projection space; and based on these patterns and characteristics, it automatically segments the highlight region using region growing to obtain the maximum reflectance chromaticity feature.

[0030] Step S130: The initial vehicle image and road surface reflection chromaticity features are processed by the chromaticity feature suppression module in the ambient light restoration network to eliminate road surface reflection light, thereby obtaining a chromaticity suppressed reflection image.

[0031] The chromaticity feature suppression module refers to the module that suppresses the maximum reflectance chromaticity.

[0032] The violation detection device based on vehicle lights corrects the road surface reflection chromaticity features to obtain suppressed road surface reflection chromaticity features; the first frequency features of the initial vehicle image and the suppressed road surface reflection chromaticity features are merged to obtain merged features; the merged features are encoded and decoded to obtain a reflection chromaticity suppressed image.

[0033] A reflection chromaticity suppressed image is an image after eliminating reflections from the road surface.

[0034] In one embodiment, a violation detection device based on vehicle lights preprocesses an initial vehicle image to obtain a preprocessed initial vehicle image. The preprocessed initial vehicle image is then input into a preset ambient light restoration network to obtain a reflection chromaticity suppressed image output by the ambient light restoration network. Preprocessing may include denoising or filtering.

[0035] Step S140: Input the reflection chromaticity suppression image and the depth image of the initial vehicle image into the preset vehicle light restoration network for vehicle light halo removal processing to obtain the target vehicle image.

[0036] Headlight restoration networks are neural networks used to eliminate headlight halo.

[0037] The target vehicle image refers to an image that has had road surface reflections and headlight halos removed.

[0038] The illegal detection device based on vehicle-mounted lights inputs a reflection chromaticity suppression image and a depth image of the initial vehicle image into a preset vehicle light restoration network for light halo removal processing to obtain the target vehicle image.

[0039] Step S150: Perform violation detection processing on the target vehicle image to obtain the vehicle violation detection result.

[0040] Vehicle violation inspection results refer to the results of inspections conducted to determine whether a vehicle has been fitted with illegal headlights. These results may include the presence of illegal headlights and the vehicle's license plate number.

[0041] The vehicle violation detection device based on the installation of additional headlights performs target detection processing on the target vehicle image to obtain the target detection object. The target detection object is then compared with a preset standard headlight. If they match, the vehicle violation detection result is "no violation"; otherwise, the vehicle violation detection result is "illegally installed headlights." The preset standard headlights refer to the headlights that can be installed on the vehicle.

[0042] It can be seen that by performing road surface reflection light elimination processing and headlight halo elimination processing on the initial vehicle image, the target vehicle image obtained eliminates the influence of headlights on the image, improves the image clarity, and thus improves the accuracy of vehicle violation detection.

[0043] In one embodiment, such as Figure 2 As shown, in low-visibility scenarios such as nighttime or rainy days, images captured by cameras on traffic roads are often illegible, making license plate numbers difficult to discern. Figure 3 As shown, the reasons for the image's lack of clarity include the influence of ambient light or atmospheric light, strong glare caused by added LED lights, and diffuse or strong reflections of the vehicle's low beam headlights from the road surface. All of these factors can make the vehicle's headlights and license plate unclear in the image, thus making it impossible to determine whether the vehicle has been fitted with overhead illuminators.

[0044] In this embodiment, a low-intensity flash is used during shooting to reduce the intensity of the bullseye lights, allowing the license plate and bullseye lights to be clearly seen in the image despite the blurriness. Secondly, an ambient light restoration network is used to process the initial vehicle image, removing large areas of halo caused by strong diffuse reflection from the road surface. Finally, a headlight restoration network is used to process the reflection chromaticity suppression image, removing the influence of the bullseye light halo, resulting in the image shown below. Figure 4 The image shown is a clearer image of the target vehicle. Compared to the initial vehicle image, the bullseye headlights and license plate characters in the target vehicle image are clearer.

[0045] The violation detection device based on vehicle-mounted lights uses the maximum reflectance chromaticity attention mechanism in the ambient light restoration network to extract chromaticity features from the initial vehicle image to obtain road surface reflectance chromaticity features. The steps include: performing frequency division processing on the initial vehicle image to obtain a second frequency image, where the image frequency of the second frequency image is less than or equal to a preset cutoff frequency; performing feature extraction processing on the second frequency image to obtain attention features; performing reflectance chromaticity downsampling processing on the attention features to obtain downsampled image features; and normalizing the downsampled image features to obtain road surface reflectance chromaticity features.

[0046] The violation detection device based on vehicle-mounted lights performs frequency division processing on the initial vehicle image to obtain a second frequency image. Specifically, the violation detection device based on vehicle-mounted lights inputs the initial vehicle image into a low-pass filter to obtain the second frequency image, i.e., a mid-to-low frequency image.

[0047] The steps for obtaining road surface reflection chromaticity features by performing chromaticity feature extraction processing on the second frequency image based on the violation detection device for vehicle-mounted lights include: performing chromaticity extraction processing on the second frequency image using the maximum reflectance chromaticity attention mechanism to obtain target reflection chromaticity features; and determining the target reflection chromaticity features as road surface reflection chromaticity features.

[0048] The target reflectance chromaticity feature refers to the chromaticity feature that meets preset requirements. For example, the target reflectance chromaticity feature is the maximum reflectance chromaticity feature.

[0049] The violation detection device based on vehicle-mounted lights inputs a second-frequency image into a maximum chromaticity feature extraction module to obtain the road surface reflectance chromaticity features. The maximum chromaticity feature extraction module employs a maximum reflectance chromaticity attention mechanism and includes a feature extraction module and a feature pooling layer, which are connected. The maximum chromaticity feature extraction module can be an improved version of HRNet. The violation detection device based on vehicle-mounted lights modifies the last feature layer in the original HRNet into a feature pooling layer. The feature pooling layer includes max pooling, average pooling, and an activation function.

[0050] For example, the image features after downsampling and the road surface reflectance chromaticity features can be expressed by the following formula: F A (x)=1 / (1+e -(R(x)) ) In the above formula, F A (x) represents the chromaticity characteristics of the road surface reflectance, and R(x) represents the image features after downsampling. F A The range of values ​​for (x) is 0-1.

[0051] like Figure 5 As shown, the violation detection device based on vehicle-mounted lights extracts features from the second frequency image through a feature extraction module, obtaining attention features F with dimensions h1·w1·d1. The device then inputs these attention features into a feature pooling layer, which performs R-pooling (RolPooling) on ​​the attention features. The feature pooling layer performs max pooling on the attention features to obtain max pooled features F1, and mean pooling on the attention features to obtain mean pooled features F2. Dividing the max pooled features F1 by the mean pooled features F2 yields the downsampled image features. Finally, the device uses a sigmoid activation function to normalize the downsampled image features, obtaining the road surface reflectance chromaticity feature F. A .

[0052] It can be seen that there is strong diffuse reflection light from the road surface in the second frequency image. By performing chromaticity extraction processing on the second frequency image through the maximum reflectance chromaticity attention mechanism, the strong diffuse reflection light from the road surface can be obtained, and thus the strong diffuse reflection light from the road surface can be suppressed, improving the image clarity.

[0053] The violation detection device based on vehicle-mounted lights uses a chromaticity feature suppression module in the ambient light restoration network to perform road reflection light elimination processing on the initial vehicle image and road surface reflection chromaticity features to obtain a reflection chromaticity suppressed image. The steps include: performing frequency division processing on the initial vehicle image to obtain a first frequency image and a second frequency image, where the image frequency of the first frequency image is greater than a preset cutoff frequency, and the image frequency of the second frequency image is less than or equal to the preset cutoff frequency; performing frequency feature extraction processing on the first frequency image to obtain a first frequency feature; performing reflection suppression processing based on the second frequency image and the road surface reflection chromaticity features to obtain a suppressed reflection chromaticity feature; and performing image reconstruction processing based on the first frequency feature and the suppressed reflection chromaticity feature to obtain the reflection chromaticity suppressed image.

[0054] A violation detection device based on vehicle-mounted lights uses an ambient light restoration network to perform frequency division processing on an initial vehicle image, obtaining a first frequency image and a second frequency image. As an example, the device inputs the initial vehicle image into a high-pass filter to obtain the first frequency image, i.e., the high-frequency image; and inputs the initial vehicle image into a low-pass filter to obtain the second frequency image, i.e., the mid-to-low frequency image. As another example, the device inputs the initial vehicle image into a high-pass filter to obtain the first frequency image; the difference between the initial vehicle image and the first frequency image is determined as the mid-to-low frequency image.

[0055] For example, the second frequency image and the first frequency image satisfy the following formula: P lowmid =P0-P high In the above formula, P lowmid P represents the second frequency image, P0 represents the initial vehicle image, P high Characterize the first frequency image.

[0056] The violation detection device based on vehicle-mounted lights performs frequency feature extraction processing on the first frequency image to obtain the first frequency features. Specifically, the violation detection device based on vehicle-mounted lights uses a feature extraction module to extract features from the first frequency image to obtain the first frequency features, also known as high-frequency features. The feature extraction module can be HRNet (High-Resolution Network).

[0057] The violation detection device based on vehicle-mounted lights performs reflection suppression processing based on a second frequency image and road surface reflection chromaticity features to obtain suppressed reflection chromaticity features. The steps include: extracting frequency features from the second frequency image to obtain second frequency features; and performing correction processing based on the second frequency features and road surface reflection chromaticity features to obtain suppressed reflection chromaticity features.

[0058] The violation detection device based on vehicle-mounted lights performs frequency feature extraction processing on the second frequency image to obtain the second frequency features. Specifically, the violation detection device based on vehicle-mounted lights uses a feature extraction module to extract features from the second frequency image to obtain the second frequency features, which are also known as mid-to-low frequency features.

[0059] The violation detection device based on vehicle-mounted lights performs correction processing based on the second frequency characteristics and road surface reflection chromaticity characteristics to obtain suppressed reflection chromaticity characteristics. Specifically, the violation detection device based on vehicle-mounted lights performs a Γ operation on the second frequency characteristics and road surface reflection chromaticity characteristics to obtain suppressed reflection chromaticity characteristics. The Γ operation refers to a nonlinear power-law transformation operation.

[0060] For example, the Γ operation can be expressed as the following formula: Γ(F lowmid , F A )=F lowmid ·(1-F A ) In the above formula, F lowmid Characterizes low-to-medium frequency features.

[0061] Suppressing reflectance chromaticity characteristics can be expressed by the following formula: N lowmid = F lowmid ·(1-F A ) In the above formula, N lowmid Characterizes the suppressed reflectance chromaticity features.

[0062] The vehicle-mounted light violation detection device performs image reconstruction processing based on a first frequency feature and a suppressed reflection chromaticity feature to obtain a reflection chromaticity suppressed image. Specifically, the device merges the first frequency feature and the suppressed reflection chromaticity feature to obtain a merged feature; the merged feature is then encoded and decoded to obtain the reflection chromaticity suppressed image.

[0063] The violation detection device based on vehicle-mounted lights stacks or stitches together the first frequency feature and the suppressed reflectance chromaticity feature to obtain a merged feature. The merged feature is then input into the encoding / decoding module to obtain the chromaticity suppressed reflectance image. The encoding / decoding module can be an MFTN (Multi-scale Feature Transfer Network) or similar, used for image reconstruction.

[0064] In one embodiment, a violation detection device based on vehicle-mounted lights performs frequency division processing on an initial vehicle image through an ambient light restoration network to obtain a first frequency image and a second frequency image. The image frequency of the first frequency image is greater than a preset cutoff frequency, and the image frequency of the second frequency image is less than or equal to the preset cutoff frequency. Frequency feature extraction processing is performed on the first frequency image to obtain a first frequency feature. Chromaticity feature extraction processing is performed on the second frequency image to obtain road surface reflection chromaticity features. Reflection suppression processing is performed based on the second frequency image and the road surface reflection chromaticity features to obtain suppressed reflection chromaticity features. Image reconstruction processing is performed based on the first frequency features and the suppressed reflection chromaticity features to obtain a reflection chromaticity suppressed image.

[0065] like Figure 6 As shown, the violation detection device based on vehicle-mounted lights inputs an initial vehicle image into a preset ambient light restoration network. The ambient light restoration network performs frequency division processing on the initial vehicle image to obtain a high-frequency image P. high and low-frequency images Plowmid The violation detection device based on vehicle lights retrofitting analyzes high-frequency images (P). high Frequency feature extraction is performed to obtain high-frequency feature F. high The violation detection device based on vehicle lights installed on the vehicle analyzes low- and mid-frequency images (P). lowmid Chromaticity feature extraction is performed to obtain the road surface reflectance chromaticity feature F. A The violation detection device based on vehicle lights installed on the vehicle analyzes low- and mid-frequency images (P). lowmid Feature extraction is performed to obtain the low-to-medium frequency features F. lowmid The violation detection device based on vehicle-mounted lights uses the road surface reflectance color characteristics F. A and mid-to-low frequency characteristics F lowmid Perform the Γ operation to obtain the suppressed reflectance chromaticity feature N. lowmid The violation detection device based on vehicle-mounted lights will suppress the reflected chromaticity feature N. lowmid and high-frequency characteristics F high The input encoding / decoding module performs image reconstruction processing to obtain a reflection chromaticity suppressed image I. The encoding / decoding module includes a decoder and an encoder.

[0066] Before the step of inputting the reflection chromaticity suppression image and the depth image of the initial vehicle image into a preset vehicle light restoration network to perform vehicle light halo elimination processing to obtain the target vehicle image, the violation detection device based on vehicle lights also includes: performing depth estimation processing on the initial vehicle image to obtain an initial estimated image; and performing normalization processing on the initial estimated image to obtain a depth image.

[0067] In one embodiment, the violation detection device based on vehicle-mounted lights inputs an initial vehicle image into a trained depth estimation network to obtain an initial estimated image. The initial estimated image is then normalized using a sigmoid function to obtain a depth image. The depth estimation network can be an algorithm such as DMENet (Defocus Map Estimation Network) or DeepLens (referring to a deep learning-based automated optical lens design system).

[0068] The steps of a vehicle-mounted light violation detection device, which inputs a reflection chromaticity suppression image and a depth image of an initial vehicle image into a preset headlight restoration network for headlight halo removal processing to obtain a target vehicle image, include: extracting headlight features from the reflection chromaticity suppression image through the headlight restoration network to obtain headlight features; fusing the image features of the reflection chromaticity suppression image and the headlight features to obtain fused features; and performing image reconstruction processing based on the fused features and the depth image to obtain the target vehicle image.

[0069] like Figure 7 As shown, the vehicle headlight violation detection device performs headlight feature extraction processing on the reflection chromaticity suppression image I to obtain headlight feature F. S The image feature F is obtained by extracting image features from the reflection chromaticity suppression image based on the violation detection device for vehicle-mounted lights. I The detection device for violations related to vehicle-mounted lights uses the characteristic F of the lights. S and image features F I The fusion process is performed to obtain the fusion feature F. F The violation detection device based on vehicle-mounted lights uses fused features F F The image is multiplied by the depth image D to obtain the multiplied features. These features are then input into the encoding / decoding module for image reconstruction to obtain the target vehicle image O. The encoding / decoding module includes a decoder and an encoder.

[0070] As an example, a violation detection device based on vehicle-mounted lights inputs a reflection chromaticity suppressed image into a backbone network to obtain the light features. The backbone network can be a ResNet (Residual Network), etc.

[0071] As another example, a violation detection device based on vehicle-mounted lights uses a headlight restoration network to extract headlight features from a reflection chromaticity suppression image. The steps include: segmenting the feature map of the reflection chromaticity suppression image using the headlight restoration network to obtain multiple sub-feature maps; performing convolution processing on each sub-feature map to obtain convolutional features for each sub-feature map; stacking and fusing adjacent convolutional features to obtain fused convolutional features; performing downsampling convolution processing on the fused convolutional features to obtain downsampled features; and determining the headlight features based on the downsampled features and the fused convolutional features.

[0072] The headlight restoration network includes a bullseye light extraction module.

[0073] The vehicle-mounted headlight violation detection device uses a headlight restoration network to segment the feature map of the reflection chromaticity suppression image, obtaining multiple sub-feature maps. Specifically, the device extracts features from the reflection chromaticity suppression image to obtain a feature map; then, it segments the feature map proportionally to its width and height to obtain multiple sub-feature maps. For example, if the feature map has dimensions (m, n, c), the device segments it proportionally to its width and height, i.e., 8 equal parts horizontally and 8 equal parts vertically, resulting in 64 sub-feature maps.

[0074] The vehicle-mounted headlight violation detection device performs convolution processing on each sub-feature map to obtain the convolutional features of each sub-feature map. As an example, the device inputs each sub-feature map into a convolutional layer to obtain the convolutional features of each sub-feature map. As another example, the device performs convolution processing on each sub-feature map at multiple scales to obtain multiple convolutional features for each sub-feature map; then, it concatenates these multiple convolutional features to obtain the convolutional features of each sub-feature map. The multiple-scale convolution processing can be two-scale or three-scale. For example, the device inputs the sub-feature map into a 3x3 convolution to obtain the first convolutional feature; inputs it into a 5x5 convolution to obtain the second convolutional feature; and inputs it into a 7x7 convolution to obtain the third convolutional feature. The violation detection device based on vehicle-mounted lights concatenates the first, second, and third convolutional features corresponding to each sub-feature map to obtain the convolutional features of the corresponding sub-feature map.

[0075] The violation detection device based on vehicle-mounted lights stacks and fuses adjacent convolutional features to obtain fused convolutional features. Specifically, the device concatenates the convolutional features of two adjacent sub-feature maps to obtain fused convolutional features.

[0076] The violation detection device based on vehicle-mounted lights performs downsampling convolution processing on the fused convolutional features to obtain downsampling features. Specifically, the device performs channel global max pooling on the fused convolutional features to obtain channel global max pooling features; it then performs convolution processing on the channel global max pooling features to obtain convolutional features; and finally, it performs activation processing on the convolutional features to obtain downsampling features.

[0077] The violation detection device based on vehicle-mounted lights determines the light features using downsampled features and fused convolutional features. Specifically, the device performs a dot product between the downsampled features and the fused convolutional features to obtain the light features. These light features include bullseye headlight features.

[0078] like Figure 8As shown, the feature map of the reflection chromaticity suppression image of the violation detection device based on vehicle-mounted headlights is divided into 64 sub-feature maps bins according to the aspect ratio, such as bin(i), bin(i+1), etc. The violation detection device based on vehicle-mounted headlights performs convolution on bin(i) and bin(i+1) with kernels of scales 3, 5, and 7 respectively, to obtain multiple convolutional features for each sub-feature map. The violation detection device based on vehicle-mounted headlights concatenates the multiple convolutional features of bin(i) to obtain convolutional feature Fb(i). The violation detection device based on vehicle-mounted headlights concatenates the multiple convolutional features of bin(i+1) to obtain convolutional feature Fb(i+1). The violation detection device based on vehicle-mounted headlights concatenates adjacent convolutional features Fb(i) and Fb(i+1) to obtain fused convolutional feature Fbn. The violation detection device based on vehicle-mounted lights performs channel-wise global max pooling (Fbn) to obtain channel-wise global max pooling features with a scale of (1,1,Y). Then, it uses a 1x1 convolution to reduce the dimensionality of these features, resulting in convolutional features with a scale of (1,1,c). Finally, it applies softmax to these convolutional features to obtain downsampled features. The device then performs a dot product between these downsampled features and the fused convolutional features Fbn to output the bullseye light features F. S .

[0079] It should be noted that the splicing or fusion operations described above in this embodiment require alignment of the feature dimensions through convolution.

[0080] As can be seen, this embodiment designs a local spatial attention mechanism for adjacent features based on the spectral local correlation of the bullseye lamp. By using the local spatial attention mechanism for adjacent features to extract the lamp features from the reflection chromaticity suppression image, the features can pay more attention to the bullseye lamp halo features, thereby preserving vehicle details and color.

[0081] The steps of a vehicle violation detection device for adding headlights to a target vehicle image to obtain a vehicle violation detection result include: performing target detection processing on the target vehicle image to obtain multiple detection objects; determining whether the vehicle has added illegal headlights based on the comparison results between the types of the multiple detection objects and preset illegal headlight types, and obtaining a headlight detection result; in response to the headlight detection result indicating that the vehicle has added illegal headlights, determining the vehicle's license plate number from the multiple detection objects; and determining the headlight detection result and the license plate number as the vehicle violation detection result.

[0082] The violation detection device based on vehicle lights inputs the target vehicle image into a trained target recognition network to obtain multiple detection objects and the type of each detection object; the multiple detection objects include vehicle lights and license plate numbers.

[0083] The comparison results include whether the type of the detected object matches the preset type of illegal vehicle light, or whether the type of the detected object does not match the preset type of illegal vehicle light. Specifically, the vehicle-mounted light violation detection device calculates the similarity between the type of each detected object and the preset type of illegal vehicle light. If the similarity is greater than a preset similarity threshold, the type of the corresponding detected object is determined to match the preset type of illegal vehicle light; otherwise, the type of each detected object is determined to not match the preset type of illegal vehicle light. Alternatively, the vehicle-mounted light violation detection device determines whether the size of each detected object matches the size of the preset illegal vehicle light. If so, the type of the detected object matches the preset type of illegal vehicle light; if the sizes are different, the type of the detected object does not match the preset type of illegal vehicle light.

[0084] The vehicle light inspection results include whether the vehicle has illegally installed vehicle lights, or whether the vehicle has not illegally installed vehicle lights.

[0085] The vehicle light violation detection device determines the vehicle has illegally installed lights if the type of the detected object matches the preset illegal light type; otherwise, it determines the vehicle does not have illegally installed lights. The illegal light type is the bullseye light type, which is a type of light that cannot be installed on the vehicle.

[0086] A violation detection device based on vehicle-mounted lights determines the vehicle's license plate number from multiple detection targets. Specifically, the device identifies the vehicle's license plate number from the text information of detection targets whose type is license plate number. After the step of inputting a reflection chromaticity suppression image and a depth image of an initial vehicle image into a preset vehicle light restoration network to perform vehicle light halo elimination processing to obtain a target vehicle image, the method further includes: in response to a first recognition result indicating a failure of license plate number recognition processing on the initial vehicle image, performing license plate number recognition processing on the target vehicle image to obtain a second recognition result, and if the second recognition result indicates a successful recognition, then it is determined that the vehicle to which the license plate number belongs has illegally installed bullseye lights.

[0087] The detection device for illegal installation of vehicle lights also includes: determining whether the number of lights in the detection object is consistent with the preset standard number of lights. If they are consistent, it is determined that the vehicle has not illegally installed lights; if they are inconsistent, it is determined that the vehicle has illegally installed lights.

[0088] The detection device for illegal installation of vehicle lights also includes: determining whether the position of the vehicle lights in the detection object is consistent with the preset position of the vehicle lights. If they are consistent, it is determined that the vehicle has not illegally installed vehicle lights; if they are inconsistent, it is determined that the vehicle has illegally installed vehicle lights.

[0089] The vehicle violation detection device based on the installation of vehicle lights will issue a warning based on the violation detection results. Specifically, in response to the vehicle violation detection result indicating that the vehicle has illegally installed vehicle lights, the device will send the license plate number and preset vehicle light violation warning information to a preset client.

[0090] Figure 9 This is a block diagram illustrating a violation detection device based on vehicle-mounted lights, as shown in an exemplary embodiment of this application. Figure 9 As shown, the exemplary vehicle-mounted light violation detection device 900 includes: an acquisition module 910, a chromaticity feature extraction and processing module 920, a road surface reflection light elimination module 930, a vehicle headlight halo elimination module 940, and a violation detection module 950. Specifically: The acquisition module 910 is used to acquire an initial vehicle image in which the headlights are on.

[0091] The chromaticity feature extraction and processing module 920 is used to input the initial vehicle image into a preset ambient light restoration network, and to perform chromaticity feature extraction processing on the initial vehicle image through the maximum reflectance chromaticity attention mechanism in the ambient light restoration network to obtain the road surface reflectance chromaticity features.

[0092] The road surface reflection light elimination module 930 is used to perform road surface reflection light elimination processing on the initial vehicle image and road surface reflection chromaticity features through the chromaticity feature suppression module in the ambient light restoration network, so as to obtain a reflection chromaticity suppressed image.

[0093] The headlight halo removal module 940 is used to input the reflection chromaticity suppression image and the depth image of the initial vehicle image into a preset headlight restoration network for headlight halo removal processing to obtain the target vehicle image.

[0094] The violation detection module 950 is used to perform violation detection processing on the target vehicle image to obtain the vehicle violation detection result.

[0095] In this exemplary vehicle violation detection device based on vehicle headlights, an initial vehicle image is acquired, in which the headlights are on. The initial vehicle image is then input into a preset ambient light restoration network, where a maximum reflectance chromaticity attention mechanism extracts chromaticity features to obtain road surface reflectance features. A chromaticity feature suppression module in the ambient light restoration network further reduces road surface reflectance light, resulting in a chromaticity-suppressed image. This chromaticity-suppressed image and the depth image of the initial vehicle image are then input into a preset headlight restoration network for headlight halo reduction, yielding a target vehicle image. Finally, violation detection is performed on the target vehicle image to obtain the vehicle violation detection result. Thus, by reducing road surface reflectance light and eliminating headlight halo in the initial vehicle image, the resulting target vehicle image eliminates the influence of headlights on the image, improving image clarity and thereby enhancing the accuracy of vehicle violation detection.

[0096] The functions of each module can be found in the implementation example of the violation detection method based on vehicle-mounted lights, and will not be repeated here.

[0097] To implement the violation detection method based on vehicle-mounted lights described in the above embodiments, this application proposes another electronic device, which can be found in the following details. Figure 10 , Figure 10 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application.

[0098] Electronic device 1000 includes memory 1001 and processor 1002, wherein memory 1001 and processor 1002 are coupled together.

[0099] The memory 1001 is used to store program data, and the processor 1002 is used to execute the program data to implement the violation detection method based on vehicle lights installed in the above embodiment.

[0100] In this embodiment, processor 1002 can also be referred to as CPU (Central Processing Unit). Processor 1002 may be an integrated circuit chip with signal processing capabilities. Processor 1002 can also be a general-purpose processor, digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. The general-purpose processor can be a microprocessor, or processor 1002 can be any conventional processor, etc.

[0101] This application also provides a computer-readable storage medium, such as Figure 11As shown, the computer-readable storage medium 1100 is used to store program data 1101. When the program data 1101 is executed by the processor, it is used to implement the violation detection method based on vehicle lights installed in the method embodiment of this application.

[0102] The methods involved in the embodiments of the violation detection method for vehicle-mounted lights in this application, when implemented as software functional units and sold or used as independent products, can be stored in a device, such as a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0103] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

[0104] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. The term "and / or" is merely a description of the association of related objects, indicating that three relationships can exist; for example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, "many" in this document means two or more. In addition, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of elements, such as including at least one of A, B, and C, and may mean including any one or more elements selected from the set consisting of A, B, and C.

Claims

1. A method for detecting violations based on the installation of vehicle lights, characterized in that, The method includes: Acquire an initial vehicle image, in which the vehicle lights are on; The initial vehicle image is input into a preset ambient light restoration network, and the initial vehicle image is processed by the maximum reflectance chromaticity attention mechanism in the ambient light restoration network to extract chromaticity features and obtain road surface reflectance chromaticity features. The initial vehicle image and the road surface reflection chromaticity features are processed by the chromaticity feature suppression module in the ambient light restoration network to eliminate road surface reflection light, resulting in a chromaticity suppressed reflection image. The reflection chromaticity suppression image and the depth image of the initial vehicle image are input into a preset vehicle light restoration network for vehicle light halo removal processing to obtain the target vehicle image. The target vehicle image is subjected to violation detection processing to obtain the vehicle violation detection result.

2. The method according to claim 1, characterized in that, The step of extracting chromaticity features from the initial vehicle image using the maximum reflectance chromaticity attention mechanism in the ambient light restoration network to obtain road surface reflectance chromaticity features includes: The initial vehicle image is subjected to frequency division processing to obtain a second frequency image, wherein the image frequency of the second frequency image is less than or equal to a preset cutoff frequency; The second frequency image is subjected to feature extraction processing to obtain the attention features of the second frequency image; The attention features are downsampled by reflectance chromaticity to obtain the downsampled image features; The downsampled image features are normalized to obtain the road surface reflectance color features.

3. The method according to claim 1, characterized in that, The step of performing road reflection light elimination processing on the initial vehicle image and the road surface reflection chromaticity features through the chromaticity feature suppression module in the ambient light restoration network to obtain a chromaticity suppressed image includes: The initial vehicle image is subjected to frequency division processing to obtain a first frequency image and a second frequency image. The image frequency of the first frequency image is greater than a preset cutoff frequency, and the image frequency of the second frequency image is less than or equal to the preset cutoff frequency. The first frequency image is subjected to frequency feature extraction processing to obtain the first frequency feature; Reflection suppression processing is performed based on the second frequency image and the road surface reflection chromaticity features to obtain suppressed reflection chromaticity features; Image reconstruction processing is performed based on the first frequency feature and the suppressed reflectance chromaticity feature to obtain the suppressed reflectance chromaticity image.

4. The method according to claim 3, characterized in that, The step of performing reflection suppression processing based on the second frequency image and the road surface reflection chromaticity features to obtain suppressed reflection chromaticity features includes: The second frequency image is subjected to frequency feature extraction processing to obtain the second frequency feature; The suppressed reflection chromaticity feature is obtained by performing correction processing based on the second frequency feature and the road surface reflection chromaticity feature.

5. The method according to claim 3, characterized in that, The step of performing image reconstruction processing based on the first frequency feature and the suppressed reflectance chromaticity feature to obtain the chromaticity suppressed reflectance image includes: The first frequency feature and the suppressed reflection chromaticity feature are merged to obtain the merged feature. The merged features are encoded and decoded to obtain the reflection chromaticity suppressed image.

6. The method according to claim 1, characterized in that, The step of inputting the reflection chromaticity suppression image and the depth image of the initial vehicle image into a preset vehicle headlight restoration network for headlight halo removal processing to obtain the target vehicle image includes: The vehicle headlight features are obtained by performing vehicle headlight feature extraction processing on the reflection chromaticity suppression image through the vehicle headlight restoration network. The image features of the reflected chromaticity suppression image and the headlight features are fused to obtain fused features; The target vehicle image is obtained by performing image reconstruction processing based on the fusion features and the depth image.

7. The method according to claim 6, characterized in that, The step of extracting vehicle headlight features from the reflected chromaticity suppression image using the vehicle headlight restoration network to obtain vehicle headlight features includes: The feature map of the reflection chromaticity suppression image is segmented by the headlight restoration network to obtain multiple sub-feature maps. Convolutional processing is performed on each sub-feature map to obtain the convolutional features of each sub-feature map; The adjacent convolutional features are stacked and fused to obtain fused convolutional features; The fused convolutional features are then subjected to downsampling convolution to obtain downsampling features; The vehicle light features are determined based on the downsampling features and the fused convolutional features.

8. The method according to claim 1, characterized in that, The step of performing violation detection processing on the target vehicle image to obtain the vehicle violation detection result includes: The target vehicle image is subjected to target detection processing to obtain multiple detection objects; Based on the comparison results between the types of the multiple detection objects and the preset types of illegal vehicle lights, it is determined whether the vehicle has been equipped with illegal vehicle lights, and the vehicle light detection result is obtained; In response to the vehicle light detection result indicating that the vehicle has illegally installed vehicle lights, the vehicle's license plate number is determined from the plurality of detection objects; The headlight detection results and the license plate number are determined as the vehicle violation detection results.

9. An electronic device, characterized in that, include: A memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to perform the method as claimed in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, include: The system stores program data, which, when executed by a processor, is used to implement the method as described in any one of claims 1-8.

Images