License plate snapshot dynamic adjustment method, electronic equipment and computer readable storage medium

Abstract

The invention relates to the technical field of image processing, in particular to a license plate snapshot dynamic adjustment method based on cross-modal fusion, electronic equipment and a computer readable storage medium, and the method comprises the steps: continuously collecting multiple frames of images of a vehicle at night through an image sensor, and extracting a highlight area of each frame of image; identifying a hot spot in the highlight area, and obtaining a moving track of the hot spot according to the coordinate change of the center of the hot spot; adjusting the image gain of an image signal processor based on the cross-modal fusion model and the vehicle existence confidence; continuously monitoring until the quality of the image frame acquired after the picture gain adjustment reaches a preset standard; and extracting license plate characters of the vehicle from the image frames with the quality reaching the standard. Visibly, by tracking the position change of the bright spot corresponding to the vehicle headlamp in the image and dynamically adjusting the picture gain, the problem of inaccurate exposure of the license plate area caused by direct strong light of the vehicle headlamp can be effectively solved, and the recognition accuracy of the vehicle license plate is enhanced.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for dynamic adjustment of license plate capture based on cross-modal fusion, an electronic device, and a computer-readable storage medium. Background Technology

[0002] With the continuous development of video surveillance technology, efficient capture of vehicles and license plates has become an important technical means in many fields such as traffic management, violation monitoring, and intelligent parking. License plate capture systems need to accurately capture vehicle and license plate information in various complex environments, especially at night or in low-light conditions. However, in nighttime environments, the strong light from vehicle headlights (the main lights on a vehicle, usually installed at the front to provide road illumination at night or in low-light conditions) often shines directly onto the image sensor (such as a camera), causing image overexposure and loss of detail, severely affecting the capture effect and recognition accuracy. To address this problem, current license plate capture systems mainly employ various methods such as infrared light-assisted capture, automatic exposure (AE) algorithms, static polarization filters, and motion detection and image processing technologies to mitigate the interference of strong headlight light and improve image quality and vehicle license plate recognition performance. Specifically: Infrared-assisted image capture illuminates the license plate with an infrared light source, allowing an infrared-sensitive camera to capture the image and mitigating the effects of visible light to some extent. However, this method is insufficient in dealing with overexposure issues caused by direct headlight glare. When the grayscale value of the headlight area in the image exceeds 200, the image sensor is prone to saturation, resulting in blurred or even unrecognizable vehicle outlines and license plate characters. In strong light conditions, the recognition rate may drop below 50%. Furthermore, infrared systems have limited effectiveness on non-infrared reflective license plates (such as some customized plates), restricting their application scope.

[0003] Automatic exposure (AE) algorithms improve overall image brightness adaptability by adjusting exposure time in real time to adapt to changes in ambient light. However, these algorithms are mostly based on global brightness analysis and are difficult to optimize locally for dynamic light sources (such as fast-moving headlights). When headlights shine directly on the image sensor, the image brightness distribution is uneven. Automatic exposure may result in insufficient grayscale values ​​(below 30) in the license plate area or overexposure (grayscale values ​​exceeding 255) in the headlight area, leading to missed detections or false recognitions. The recognition rate in complex nighttime scenes is approximately 75%.

[0004] Static polarization filters reduce license plate glare interference and improve image contrast by filtering out reflected light from specific directions. However, their fixed polarization direction limits their adaptability to dynamic light source changes, especially under conditions of high-speed vehicle travel (speed > 80 km / h) or multi-angle lighting. Furthermore, this method fails to address image sensor saturation caused by direct headlight glare, resulting in a false negative rate of 15%-20% under strong light interference.

[0005] Motion detection and image processing technologies utilize vehicle motion information to optimize capture timing, such as triggering the shutter by detecting vehicle movement. However, they primarily focus on timing selection and do not adjust imaging parameters based on the dynamic characteristics of the light source (such as the movement trajectory of the bright spot corresponding to the headlight in the image). Existing motion detection technologies typically ignore dynamic changes in the light spot and struggle to effectively resist direct interference from strong headlight glare at night on the image sensor, resulting in a high failure rate for vehicle and license plate detection. Furthermore, changes in ambient light (such as streetlight switching) can also affect the accurate adjustment of exposure parameters.

[0006] It is evident that while traditional license plate capture methods have alleviated the interference of strong headlights to some extent, they still have significant shortcomings in dealing with inaccurate exposure caused by direct headlight glare, such as overexposure and image quality degradation. This makes it difficult to ensure efficient capture and accurate recognition of vehicles and license plates at night and under complex lighting conditions. Summary of the Invention

[0007] In view of this, the present invention proposes a dynamic adjustment method for license plate capture based on cross-modal fusion, an electronic device, and a computer-readable storage medium, aiming to solve the problem that traditional license plate capture methods cannot effectively cope with the inaccurate exposure of the license plate area caused by the direct glare of vehicle headlights at night.

[0008] To solve the above problems, the present invention adopts the following technical solution: In a first aspect, this invention proposes a dynamic adjustment method for license plate capture based on cross-modal fusion, comprising: The image sensor continuously acquires multiple frames of vehicle images during nighttime, and preprocesses each frame to extract the bright areas in the image; wherein, the bright areas are the areas in the image corresponding to the headlights of the vehicle. An adaptive threshold segmentation algorithm is used to identify bright spots in the high-brightness region, and the movement trajectory of the bright spots is obtained based on the coordinate changes of the center of the bright spots in consecutive image frames; wherein, the movement trajectory includes the movement direction and the movement speed. The dynamic features of the movement trajectory and the static features of the image frame are input into a cross-modal fusion model to generate a vehicle presence confidence score. If it is confirmed that the confidence level of the vehicle is greater than the preset confidence threshold, a preliminary gain is calculated based on the moving speed, and the preliminary gain is dynamically fine-tuned in combination with the actual brightness of the license plate area to obtain a final gain; based on the final gain, the image gain of the image signal processor connected to the image sensor is adjusted. Continuously monitor whether the quality of the image frames acquired after image gain adjustment meets the preset standard until it is confirmed that the quality of the acquired image frames meets the preset standard; Extract the vehicle's license plate characters from image frames whose quality meets the preset standard.

[0009] Further, the step of inputting the dynamic features of the movement trajectory and the static features of the image frame into a cross-modal fusion model to generate vehicle presence confidence includes: The cross-modal fusion model includes a dynamic feature extraction branch, a static feature extraction branch, and a feature fusion layer; The dynamic feature extraction branch is used to process the time-series data of the movement trajectory and extract dynamic feature vectors; The static feature extraction branch is used to process the spatial data of the image frame and extract static feature vectors. The feature fusion layer is used to perform weighted fusion of the dynamic feature vector and the static feature vector, and output the vehicle presence confidence.

[0010] Furthermore, the dynamic features include at least one of the moving speed, acceleration, and rate of change of direction of the bright spot; the static features include at least one of the texture, edge, and contour information of the image.

[0011] Furthermore, after the generated vehicle has a confidence level, it also includes: Determine whether the confidence level of the vehicle is greater than a preset confidence threshold; If the value is greater than the specified value, then it is confirmed that a vehicle target exists in the image. If the value is not greater than the specified value, then it is confirmed that there is no vehicle target in the image.

[0012] Furthermore, the step of dynamically fine-tuning the initial gain based on the actual brightness of the license plate area to obtain the final gain includes: Obtain the average grayscale value of the license plate area; Calculate the brightness deviation between the average gray value and the preset target gray range; Based on the brightness deviation, the gain adjustment amount is calculated according to a preset gain adjustment function relationship; The gain fine-tuning amount is superimposed on the initial gain to obtain the final gain.

[0013] Furthermore, after adjusting the image gain of the image signal processor connected to the image sensor based on the final gain, the method further includes: Determine whether the grayscale value of the bright spot in the image exceeds a preset grayscale threshold; If the exposure time is not exceeded, the current exposure time of the image sensor is maintained; If the value exceeds the limit, global gain adjustment is paused and exposure time optimization is performed; after the exposure time optimization is completed, global gain adjustment is reactivated. The exposure time optimization operation includes: Let E be the exposure time of the image sensor, and L be the gray value of the bright spot in the image. Calculate the exposure time using the formula: E = 1 / (kL + b). Where k is a preset first constant and b is a preset second constant; If there is no reflective area for the license plate in the image, the exposure parameters of the image sensor are adjusted according to the calculated exposure time. If there is a reflective area of ​​a license plate in the image, the calculated exposure time is further shortened according to a preset rule, and the exposure parameters of the image sensor are adjusted according to the shortened exposure time. The reflective area of ​​the license plate is the area within the license plate area where the gray value is within a preset reflective gray value range, and the maximum value of the preset reflective gray value range is equal to the preset gray value threshold.

[0014] Furthermore, the preset standard includes: performing noise reduction processing on the image frames acquired after image gain adjustment according to a preset noise reduction algorithm, so that the noise level of the image frames after noise reduction is lower than a preset noise threshold.

[0015] Furthermore, the step of extracting the vehicle's license plate characters from image frames whose quality meets the preset standard includes: using a deep learning model based on a convolutional neural network to extract the overall image information and license plate characters of the vehicle from image frames whose quality meets the preset standard.

[0016] Secondly, the present invention also proposes an electronic device, which is an embedded device, comprising: Memory, used to store program instructions; and A processor is used to execute the program instructions to implement the steps of the license plate capture dynamic adjustment method based on cross-modal fusion as described above.

[0017] Thirdly, the present invention also proposes a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the license plate capture dynamic adjustment method based on cross-modal fusion as described above.

[0018] Compared with existing technologies, the advantages of this invention are as follows: A method for dynamic adjustment of license plate capture based on cross-modal fusion, an electronic device, and a computer-readable storage medium. First, an image sensor continuously acquires multiple frames of vehicle images during nighttime, and preprocesses each frame to extract the bright areas corresponding to the vehicle's headlights, thereby accurately locating the strong light source in the image. Then, an adaptive threshold segmentation algorithm is used to identify bright spots in the bright areas, and their movement trajectory is obtained by tracking the changes in the center coordinates of the bright spots in consecutive image frames, providing real-time basis for subsequent dynamic adjustment of image gain. Next, a vehicle presence confidence score is generated based on the cross-modal fusion model, and a final gain is obtained based on the vehicle presence confidence score. Then, the image gain of the image signal processor connected to the image sensor is adjusted based on the final gain, thereby suppressing local overexposure caused by the strong light of the vehicle's headlights directly hitting the image sensor in advance and effectively improving the overall exposure quality of the image. Finally, the quality of the images acquired after image gain adjustment is continuously monitored to ensure that they meet the preset standard, and then the license plate characters of the vehicle are extracted from the qualified images to ensure the accuracy and stability of vehicle license plate recognition. As can be seen, this cross-modal fusion-based dynamic adjustment method for license plate capture, along with the electronic device and computer-readable storage medium, effectively solves the problem of inaccurate exposure of the license plate area caused by direct glare from vehicle headlights by dynamically adjusting the image gain in combination with the cross-modal fusion model and vehicle presence confidence. This ensures that the image quality meets the preset standard, thereby significantly improving image quality and enhancing the accuracy of vehicle license plate recognition. Attached Figure Description

[0019] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a flowchart illustrating the first embodiment of the license plate capture dynamic adjustment method based on cross-modal fusion of the present invention. Figure 2 This is a flowchart illustrating the second embodiment of the license plate capture dynamic adjustment method based on cross-modal fusion of the present invention. Figure 3 This is a flowchart illustrating the third embodiment of the license plate capture dynamic adjustment method based on cross-modal fusion of the present invention. Figure 4 This is a schematic diagram of the structure of an electronic device in the hardware operating environment involved in the embodiments of the present invention. Detailed Implementation

[0020] The solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] It should be noted that if the embodiments of the present invention involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of the components in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0022] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0023] Please see Figure 1 The diagram shown is a flowchart of the first embodiment of the license plate capture dynamic adjustment method based on cross-modal fusion of the present invention.

[0024] In this embodiment, the license plate capture dynamic adjustment method based on cross-modal fusion includes: Step S100: Use an image sensor to continuously acquire multiple frames of vehicle images during nighttime, and preprocess each frame of image to extract the bright areas in the image.

[0025] The highlighted area is the region in the image that corresponds to the headlights of the vehicle.

[0026] Specifically, in low-light conditions at night, the image quality captured by image sensors (such as cameras and camcorders) is usually low, thus requiring image preprocessing. Various preprocessing methods exist, including image enhancement, filtering, and brightness normalization. Appropriate selection of preprocessing techniques helps to accurately extract bright areas from the image. By continuously acquiring multiple frames of images, dynamic information about the vehicle at different points in time can be obtained, allowing for precise location and prediction of the vehicle's headlights within the image, providing a solid foundation for subsequent bright spot recognition and tracking.

[0027] Step S200: An adaptive threshold segmentation algorithm is used to identify bright spots in the bright area, and the movement trajectory of the bright spots is obtained based on the coordinate changes of the center of the bright spots in consecutive image frames.

[0028] The movement trajectory includes the movement direction and the movement speed.

[0029] Specifically, accurately segmenting and identifying bright spots within highlighted areas is crucial for achieving dynamic adjustment in license plate capture. Adaptive threshold segmentation algorithms can dynamically adjust segmentation criteria based on local image brightness, thereby improving the accuracy of bright spot recognition. The center of a bright spot refers to its point of maximum brightness. By obtaining the precise coordinates of the bright spot center in the image and its changes, the direction and speed of movement of the bright spot center can be calculated, thus reflecting the dynamic positional changes of the bright spot in consecutive image frames and enabling more accurate prediction of the bright spot's trajectory.

[0030] Step S300: Input the dynamic features of the movement trajectory and the static features of the image frame into the cross-modal fusion model to generate vehicle presence confidence.

[0031] Step S400: If it is confirmed that the confidence level of the vehicle is greater than the preset confidence threshold, then calculate the preliminary gain based on the moving speed, and dynamically fine-tune the preliminary gain in combination with the actual brightness of the license plate area to obtain the final gain; based on the final gain, adjust the image gain of the image signal processor connected to the image sensor.

[0032] Step S500: Continuously monitor whether the quality of the image frames acquired after image gain adjustment meets the preset standard until it is confirmed that the quality of the acquired image frames meets the preset standard.

[0033] Specifically, there are various evaluation metrics for image quality, such as brightness balance, contrast, sharpness, and the proportion of overexposed areas. Depending on the specific application requirements, appropriate metrics can be selected to ensure that the image quality meets different recognition requirements. This license plate capture dynamic adjustment method primarily addresses the image overexposure problem in nighttime environments; therefore, the image quality evaluation standard, i.e., the preset standard, is based on the signal-to-noise ratio, and can be comprehensively evaluated in conjunction with exposure conditions when necessary.

[0034] Step S600: Extract the license plate characters of the vehicle from the image frames whose quality meets the preset standard.

[0035] Specifically, license plate character extraction mainly includes key steps such as license plate localization, character segmentation, and character recognition, all of which rely on clear and properly exposed image quality. Therefore, extracting vehicle license plate characters from image frames that meet quality standards can significantly improve the accuracy of license plate character recognition.

[0036] The license plate capture dynamic adjustment method based on cross-modal fusion in this invention aims to solve the technical problems of traditional license plate capture methods, such as easy missed detection in low-speed scenes, easy blurring in high-speed scenes, and inaccurate exposure of the license plate area due to imprecise gain adjustment, so as to achieve robust detection and fine exposure control in the full speed domain.

[0037] Compared with existing technologies, the license plate capture dynamic adjustment method based on cross-modal fusion in this invention includes: continuously acquiring multiple frames of vehicle images using an image sensor during nighttime, and extracting the bright areas of each frame; identifying bright spots in the bright areas, and obtaining the movement trajectory of the bright spots based on the coordinate changes of the center of the bright spots; inputting the dynamic features of the movement trajectory and the static features of the image frames into a cross-modal fusion model to generate a vehicle presence confidence score; calculating and adjusting the image gain of the image signal processor based on the vehicle presence confidence score; wherein the adjustment includes: calculating an initial gain based on the movement speed, and dynamically fine-tuning the initial gain in conjunction with the actual brightness of the license plate area; continuously monitoring until it is confirmed that the quality of the image frames acquired after image gain adjustment meets a preset standard; and extracting the license plate characters of the vehicle from the image frames that meet the quality standard. It is evident that by fusing the dynamic motion information of the bright spot with the static visual information of the image across modes, and by adjusting the gain in a closed-loop feedback manner in conjunction with the license plate brightness, the presence of the vehicle can be accurately determined and the exposure optimized. This effectively solves the problem of missed detection in nighttime scenes and dynamically adjusts the image gain, thereby significantly suppressing the overexposure problem caused by the strong light of the vehicle headlights directly hitting the image sensor, significantly improving image quality, and enhancing the recognition accuracy of vehicle license plates.

[0038] Please see Figure 2 The diagram shown is a flowchart of the second embodiment of the license plate capture dynamic adjustment method based on cross-modal fusion of the present invention.

[0039] This embodiment, compared to other embodiments of the license plate capture dynamic adjustment method based on cross-modal fusion, provides a detailed explanation of the steps for generating vehicle presence confidence based on the cross-modal fusion model.

[0040] In this embodiment, step S300 includes: Step S301: The cross-modal fusion model includes a dynamic feature extraction branch, a static feature extraction branch, and a feature fusion layer.

[0041] Step S302: The dynamic feature extraction branch is used to process the time series data of the movement trajectory and extract dynamic feature vectors.

[0042] Step S303: The static feature extraction branch is used to process the spatial data of the image frame and extract static feature vectors.

[0043] Step S304: The feature fusion layer is used to perform weighted fusion of the dynamic feature vector and the static feature vector, and output the vehicle presence confidence.

[0044] Specifically, a cross-modal fusion model is constructed that includes a dynamic feature extraction branch, a static feature extraction branch, and a feature fusion layer. The time-series data of the bright spot's movement trajectory are processed by the dynamic feature extraction branch (such as a Long Short-Term Memory Network (LSTM) or a Gated Recurrent Unit (GRU)) to extract dynamic feature vectors containing information such as velocity, acceleration, and rate of change of direction. The static feature extraction branch (such as the lightweight convolutional neural network MobileNetV2) processes the current image frame to extract static feature vectors containing information such as texture, edges, and contours. In the feature fusion layer (such as the self-attention mechanism layer), the dynamic feature vector and the static feature vector are adaptively weighted and fused to output a vehicle presence confidence score between 0 and 1.

[0045] For example, in a vehicle idling scenario, the dynamic features of the bright spot's movement trajectory are weak, but the static feature extraction branch can still identify the vehicle through information such as the circular outline of the headlights and the texture of the vehicle grille. The attention mechanism will give higher weight to static features. In a vehicle high-speed driving scenario, the image may be blurred due to motion, causing a decrease in static features. However, the dynamic feature extraction branch can confirm the presence of the vehicle through the high-speed movement trajectory of the bright spot, and the attention mechanism will give higher weight to dynamic features.

[0046] In some embodiments of this application, the dynamic features include at least one of the moving speed, acceleration, and rate of change of direction of the bright spot; the static features include at least one of the texture, edge, and contour information of the image.

[0047] This invention relates to a dynamic adjustment method for license plate capture based on cross-modal fusion, belonging to the field of image processing and vehicle recognition technology. Specifically, it involves a method for dynamically adjusting the gain parameters of an image signal processor (ISP) by tracking the movement trajectory of a vehicle's headlight glare in a nighttime environment. This dynamic adjustment method effectively avoids the problem of vehicles and license plates being unrecognizable due to strong headlight glare directly hitting the image sensor, significantly optimizing the capture effect of vehicles and license plates.

[0048] In some embodiments of this application, after the generated vehicle has a confidence level, the process further includes: Step S701: Determine whether the confidence level of the vehicle is greater than a preset confidence threshold; Step S702a: If it is greater than, then it is confirmed that there is a vehicle target in the image; Step S702b: If it is not greater than, confirm that there is no vehicle target in the image.

[0049] In some embodiments of the present application, dynamically fine-tuning the preliminary gain in combination with the actual brightness of the license plate area to obtain the final gain includes: Step S401: Obtain the average grayscale value of the license plate area; Step S402: Calculate the brightness deviation between the average grayscale value and the preset target grayscale range; Step S403: Calculate the gain fine-tuning amount according to the brightness deviation according to the preset gain fine-tuning function relationship; Step S404: Superimpose the gain fine-tuning amount on the preliminary gain to obtain the final gain.

[0050] Specifically, after confirming the presence of a vehicle, determine the position of the license plate area in the image through the static feature extraction branch in the cross-modal fusion model or an independent license plate localization algorithm (such as YOLOv5); Calculate the average grayscale value L_plate of this license plate area and compare it with the preset target grayscale range [Gray_min, Gray_max] (such as [50, 70]); If L_plate < Gray_min, according to the brightness deviation ΔL = Gray_min - L_plate, calculate a positive gain fine-tuning amount according to the gain fine-tuning formula ΔG = k1 * ΔL (k1 is a positive fine-tuning coefficient, such as 0.4); If L_plate > Gray_max, according to the brightness deviation ΔL = L_plate - Gray_max, calculate a negative gain fine-tuning amount according to the gain fine-tuning formula ΔG = -k2 * ΔL (k2 is a negative fine-tuning coefficient, such as 0.3); Superimpose the calculated gain fine-tuning amount ΔG on the preliminary gain G_preliminary calculated based on the moving speed to obtain the final gain G_final = G_preliminary + ΔG, and adjust the picture gain of the ISP accordingly.

[0051] For example, when the bright spot moving speed v = 30 pixels / second, the initial gain is calculated to be 14dB according to the formula G_preliminary = 0.2v + 8. If the average gray value L_plate of the license plate area is detected to be 45 (below the lower limit of the target range of 50), then ΔL = 5, the gain adjustment ΔG = 0.4 * 5 = 2dB, and the final gain G_final = 14 + 2 = 16dB. If L_plate = 75 (above the upper limit of the target range of 70), then ΔL = 5, the gain adjustment ΔG = -0.3 * 5 = -1.5dB, and the final gain G_final = 14 - 1.5 = 12.5dB.

[0052] Please see Figure 3 The diagram shown is a flowchart of the third embodiment of the license plate capture dynamic adjustment method based on cross-modal fusion of the present invention.

[0053] Compared to other embodiments of the license plate capture dynamic adjustment method based on cross-modal fusion, this embodiment adds a step of optimizing exposure time.

[0054] In this embodiment, after step S400, the method further includes: Step S801: Determine whether the gray value of the bright spot in the image exceeds the preset gray value threshold.

[0055] Specifically, this license plate capture dynamic adjustment method employs grayscale conversion and luminance histogram analysis as image preprocessing techniques. Grayscale conversion simplifies image information, reduces computational complexity, and facilitates subsequent analysis; luminance histogram analysis effectively reflects the brightness distribution characteristics of the image, helping to accurately identify and extract bright areas. It should be noted that after the above preprocessing, the brightness levels in this embodiment are represented by grayscale values.

[0056] Step S802a: If the exposure time has not been exceeded, maintain the current exposure time of the image sensor.

[0057] Step S802b: If the value exceeds the limit, pause global gain adjustment and perform exposure time optimization operation; after the exposure time optimization operation is completed, re-enable global gain adjustment.

[0058] The exposure time optimization operation includes: Let E be the exposure time of the image sensor, and L be the gray value of the bright spot in the image. Calculate the exposure time using the formula: E = 1 / (kL + b). Where k is a preset first constant and b is a preset second constant.

[0059] Specifically, in this embodiment, the current exposure time is set to 1 / 100 second. The preset grayscale threshold is set to 200. When the grayscale value of a detected bright spot exceeds 200, it indicates that the image sensor may be affected by the direct glare of a vehicle's headlights. In this case, the global gain should be paused, and the image brightness should be reduced by adjusting the exposure time to avoid overexposure.

[0060] Specifically, after optimizing the exposure time to reduce image brightness, the previously paused global gain adjustment function is restarted to continue dynamically adjusting the image gain, thereby ensuring image quality and license plate recognition performance.

[0061] Specifically, overexposure of the image sensor becomes a major problem when the headlights of a vehicle shine directly on it. Exposure time (E) is directly proportional to the total luminous flux entering the image sensor, and the total luminous flux is directly related to the grayscale value (L) of the bright spot. To prevent overexposure, the exposure time must be inversely proportional to the grayscale value of the bright spot. Based on this, we established an inverse proportional function model: E = 1 / (kL + b). This inverse proportional function model accurately describes the physical relationship between exposure time and the grayscale value of the bright spot. When the grayscale value L of the bright spot increases, the exposure time E decreases rapidly and non-linearly, effectively suppressing highlight overflow. Here, a first constant k = 0.005 is preset, which serves as a brightness response coefficient to determine the sensitivity of the exposure time to changes in brightness. This value, after repeated adjustments and optimizations, ensures rapid adjustment of exposure time even during drastic brightness changes (such as sudden switching on of high beams). A preset second constant, b=100, serves as an offset or base exposure damping to prevent the exposure time from extending indefinitely in extremely low brightness, while ensuring stable base exposure in low-light environments to avoid overly dark images. This inverse proportional function model, based on real-time brightness feedback, achieves precise non-linear control of exposure time, calculating the optimal exposure time in real-time based on the actual grayscale value of the bright spot.

[0062] Step S803a: If there is no reflective area of ​​the license plate in the image, adjust the exposure parameters of the image sensor according to the calculated exposure time.

[0063] Specifically, when the gray value of the bright spot exceeds 200 and there is no reflective area of ​​the license plate in the image, the exposure parameters of the image sensor are adjusted according to the calculated exposure time in order to dynamically shorten the exposure time (e.g., from 1 / 100 second to 1 / 500 second) and effectively prevent the image sensor from losing vehicle and license plate details due to strong light saturation.

[0064] Step S803b: If there is a license plate reflective area in the image, the calculated exposure time is further shortened according to a preset rule, and the exposure parameters of the image sensor are adjusted according to the shortened exposure time.

[0065] The reflective area of ​​the license plate is the area within the license plate area where the gray value is within a preset reflective gray value range, and the maximum value of the preset reflective gray value range is equal to the preset gray value threshold.

[0066] Specifically, the grayscale value of the reflective area of ​​the license plate is generally between 150 and 200 (preset reflective grayscale range). When a reflective area of ​​the license plate exists in the image, the exposure time is further fine-tuned to between 1 / 800 and 1 / 1000 of a second according to preset rules in order to effectively suppress reflective interference from the license plate area.

[0067] It should be noted that there is no specific order between steps S803a and S803b; either step can be performed depending on the actual situation of the image.

[0068] In some embodiments of this application, the preset standard includes: performing noise reduction processing on the image frames acquired after image gain adjustment according to a preset noise reduction algorithm, so that the noise level of the image frames after noise reduction is lower than a preset noise threshold.

[0069] Specifically, to ensure the accuracy and efficiency of license plate recognition, a preset standard is set to screen image quality. This preset standard includes a preset noise threshold. Only images with noise levels below the preset noise threshold after processing by a preset noise reduction algorithm are considered to be of acceptable quality and can be used for subsequent license plate character recognition, thereby avoiding the waste of computing resources due to poor image quality.

[0070] In some embodiments of this application, extracting the vehicle's license plate characters from image frames whose quality meets the preset standard includes: using a deep learning model based on a convolutional neural network to extract the overall image information and license plate characters of the vehicle from image frames whose quality meets the preset standard.

[0071] Specifically, using a deep learning model based on convolutional neural networks to recognize license plate characters, the target recognition rate can reach over 90%. During the recognition process, the deep learning model also extracts overall image information of the vehicle to achieve multi-dimensional information fusion and analysis, further improving the completeness and reliability of the recognition results.

[0072] The license plate capture dynamic adjustment method based on cross-modal fusion in this invention firstly constructs a cross-modal fusion model that deeply fuses the dynamic features of the bright spot's movement trajectory with the static features of the image. For example, in a vehicle idling scenario, the dynamic features of the bright spot's movement trajectory are weak, but the static feature extraction branch can still identify the vehicle through information such as the circular outline of the headlights and the texture of the vehicle grille. In a high-speed driving scenario, the image may experience a decrease in static features due to motion blur, but the dynamic feature extraction branch can confirm the vehicle's presence through the high-speed movement trajectory of the bright spot. This complementary mechanism effectively solves the limitations of a single technical path, achieving robust vehicle detection across the entire speed range and significantly reducing the low-speed false detection rate and high-speed false recognition rate. Secondly, based on gain pre-adjustment based on movement speed, this invention innovatively introduces a closed-loop feedback fine-tuning mechanism based on the actual brightness of the license plate area. This mechanism can accurately compensate for exposure deviations in the license plate area caused by factors such as changes in ambient light and switching between high and low beams of the vehicle's headlights, ensuring that the grayscale value of the license plate area is always within the ideal range most conducive to recognition. For example, even if two vehicles are traveling at the same speed, this solution can apply different final gains based on license plate brightness feedback due to differences in headlight brightness or ambient glare. This achieves refined exposure control tailored to each vehicle, significantly improving license plate recognition accuracy. Finally, the quality of the image acquired after gain adjustment is continuously monitored to ensure it meets preset standards before extracting the vehicle's license plate characters from the compliant image, guaranteeing the accuracy and stability of license plate recognition. As can be seen, this dynamic adjustment method for license plate capture based on bright spot tracking predicts the movement trajectory of bright spots corresponding to vehicle headlights in real time and dynamically adjusts the image gain. This effectively suppresses overexposure caused by direct headlight glare on the image sensor, ensuring image quality meets preset standards and significantly improving image quality, thereby enhancing the accuracy of vehicle license plate recognition. The license plate capture dynamic adjustment method based on cross-modal fusion in this invention aims to optimize license plate imaging in nighttime environments. Through multi-dimensional dynamic adjustment, it effectively improves the image quality and recognition accuracy of nighttime license plate capture. The method mainly includes the following steps: Bright Spot Detection and Motion Analysis: Detect bright spots corresponding to vehicle headlights and calculate their movement trajectory by comparing the differences in bright spot positions in previous and subsequent image frames; Dynamic gain adjustment: Based on the moving speed and position of the bright spot, the image gain is dynamically adjusted to enhance the brightness of the license plate area, while avoiding the failure of the image sensor due to strong light saturation and preventing unnecessary gain adjustment caused by the street lights being turned on. Exposure time optimization: Dynamically shorten the exposure time to suppress overexposure, targeting the brightness of bright spots and reflective areas of license plates; Noise reduction and image optimization: While increasing gain, noise reduction processing is combined to ensure that the image quality meets the requirements of license plate recognition.

[0073] Accordingly, the license plate capture dynamic adjustment system based on bright spot tracking according to this embodiment of the invention includes the following modules: Image acquisition module: A camera equipped with an ISP for capturing vehicle images in nighttime environments; Bright Spot Detection Module: Identifies bright spots in vehicle headlights and their movement trajectories using an image processing algorithm based on the difference in bright spot positions between consecutive image frames; Parameter adjustment module: Controls the ISP to dynamically adjust gain parameters and exposure time; Image processing module: responsible for image noise reduction and license plate character recognition.

[0074] This license plate capture dynamic adjustment system based on luminance spot tracking achieves dynamic optimization and efficient recognition of nighttime license plate images through the coordinated work of its various modules.

[0075] The license plate capture dynamic adjustment method based on cross-modal fusion in this invention has the following advantages: Strong light interference suppression: By detecting bright spot movement and adjusting dynamic parameters, the saturation problem caused by the direct glare of the vehicle headlights hitting the image sensor is effectively avoided; Strong dynamic adaptability: Based on the difference in bright spot position between consecutive image frames, motion analysis is performed to optimize image gain and exposure time in real time, which can adapt to the vehicle's rapid movement or stationary state. Low-cost implementation: Based on existing ISP hardware and software algorithms, no additional optical components are required, reducing implementation costs; High robustness: Significantly improves license plate character recognition rate under low light and strong light interference conditions, effectively distinguishes between vehicle headlights and streetlights, avoids interference caused by streetlights, and is suitable for a variety of complex traffic scenarios.

[0076] The following are embodiments of the electronic device provided by the present invention. The embodiments of the electronic device and the embodiments of the above-described dynamic adjustment method for license plate capture based on cross-modal fusion belong to the same concept. For details not fully described in the embodiments of the electronic device, please refer to the embodiments of the above-described dynamic adjustment method for license plate capture based on cross-modal fusion.

[0077] Please see Figure 4 As shown, it is a schematic diagram of the structure of an electronic device in the hardware operating environment involved in the embodiment of the present invention.

[0078] In this embodiment, an electronic device, which is an embedded device, includes: Memory 1005 is used to store program instructions; and Processor 1001 is used to execute the program instructions to implement the steps of the license plate capture dynamic adjustment method based on cross-modal fusion as described above.

[0079] The electronic device in this embodiment of the invention can be a desktop computer, laptop, handheld computer, server, or other computing device. Figure 4 As shown, the electronic device may include: a processor 1001 (e.g., a CPU), a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit, such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0080] Those skilled in the art will understand that Figure 4 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0081] like Figure 4 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and computer programs.

[0082] exist Figure 4 In the electronic device shown, the network interface 1004 is mainly used to connect to the backend server and communicate with the backend server; the user interface 1003 is mainly used to connect to the client (user terminal) and communicate with the client; and the processor 1001 can be used to call the computer program stored in the memory 1005. When the computer program is called and executed by the processor 1001, it implements the steps of the above-mentioned dynamic adjustment method for license plate capture based on cross-modal fusion.

[0083] The following are embodiments of the computer-readable storage medium provided by the present invention. The embodiments of the computer-readable storage medium belong to the same concept as the above-described embodiments of the license plate capture dynamic adjustment method and electronic device based on cross-modal fusion. Details not fully described in the embodiments of the computer-readable storage medium can be found in the above-described embodiments of the license plate capture dynamic adjustment method and electronic device based on cross-modal fusion.

[0084] In this embodiment, a computer-readable storage medium stores computer-executable instructions thereon, which, when executed by a processor, implement the steps of the license plate capture dynamic adjustment method based on cross-modal fusion as described above.

[0085] The electronic device and computer-readable storage medium of the present invention are both capable of implementing the steps of the above-described method for dynamic adjustment of license plate capture based on cross-modal fusion. Therefore, they at least have all the beneficial effects brought about by the technical solutions of the above-described embodiments of the method for dynamic adjustment of license plate capture based on cross-modal fusion, which will not be elaborated here.

[0086] The above description is only a part or preferred embodiment of the present invention. Neither the text nor the drawings should limit the scope of protection of the present invention. All equivalent structural transformations made using the content of the present invention specification and drawings under the overall concept of the present invention, or direct / indirect applications in other related technical fields, are included within the scope of protection of the present invention.

[0087] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program goods. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs) containing computer-usable program code. The form of computer program products implemented on ROM, optical memory, etc.

[0088] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program goods according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0089] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The function specified in one or more boxes.

[0090] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A dynamic adjustment method for license plate capture based on cross-modal fusion, characterized in that, include: The image sensor continuously acquires multiple frames of vehicle images during nighttime, and preprocesses each frame to extract the bright areas in the image; wherein, the bright areas are the areas in the image corresponding to the headlights of the vehicle. An adaptive threshold segmentation algorithm is used to identify bright spots in the high-brightness region, and the movement trajectory of the bright spots is obtained based on the coordinate changes of the center of the bright spots in consecutive image frames; wherein, the movement trajectory includes the movement direction and the movement speed. The dynamic features of the movement trajectory and the static features of the image frame are input into a cross-modal fusion model to generate a vehicle presence confidence score. If it is confirmed that the confidence level of the vehicle is greater than the preset confidence threshold, a preliminary gain is calculated based on the moving speed, and the preliminary gain is dynamically fine-tuned in combination with the actual brightness of the license plate area to obtain a final gain; based on the final gain, the image gain of the image signal processor connected to the image sensor is adjusted. Continuously monitor whether the quality of the image frames acquired after image gain adjustment meets the preset standard until it is confirmed that the quality of the acquired image frames meets the preset standard; Extract the vehicle's license plate characters from image frames whose quality meets the preset standard.

2. The method for dynamic adjustment of license plate capture based on cross-modal fusion according to claim 1, characterized in that, The step of inputting the dynamic features of the movement trajectory and the static features of the image frame into a cross-modal fusion model to generate vehicle presence confidence includes: The cross-modal fusion model includes a dynamic feature extraction branch, a static feature extraction branch, and a feature fusion layer; The dynamic feature extraction branch is used to process the time-series data of the movement trajectory and extract dynamic feature vectors; The static feature extraction branch is used to process the spatial data of the image frame and extract static feature vectors. The feature fusion layer is used to perform weighted fusion of the dynamic feature vector and the static feature vector, and output the vehicle presence confidence.

3. The method for dynamic adjustment of license plate capture based on cross-modal fusion according to claim 2, characterized in that, The dynamic features include at least one of the moving speed, acceleration, and rate of change of direction of the bright spot; the static features include at least one of the texture, edge, and contour information of the image.

4. The method for dynamic adjustment of license plate capture based on cross-modal fusion according to claim 1, characterized in that, After the generated vehicle has a confidence level, the following steps are also included: Determine whether the confidence level of the vehicle is greater than a preset confidence threshold; If the value is greater than the specified value, then it is confirmed that a vehicle target exists in the image. If the value is not greater than the specified value, then it is confirmed that there is no vehicle target in the image.

5. The method for dynamic adjustment of license plate capture based on cross-modal fusion according to claim 1, characterized in that, The process of dynamically fine-tuning the initial gain based on the actual brightness of the license plate area to obtain the final gain includes: Obtain the average grayscale value of the license plate area; Calculate the brightness deviation between the average gray value and the preset target gray range; Based on the brightness deviation, the gain adjustment amount is calculated according to a preset gain adjustment function relationship; The gain fine-tuning amount is superimposed on the initial gain to obtain the final gain.

6. The method for dynamic adjustment of license plate capture based on cross-modal fusion according to claim 5, characterized in that, After adjusting the image gain of the image signal processor connected to the image sensor based on the final gain, the method further includes: Determine whether the grayscale value of the bright spot in the image exceeds a preset grayscale threshold; If the exposure time is not exceeded, the current exposure time of the image sensor is maintained; If the value exceeds the limit, global gain adjustment is paused and exposure time optimization is performed; after the exposure time optimization is completed, global gain adjustment is reactivated. The exposure time optimization operation includes: Let E be the exposure time of the image sensor, and L be the gray value of the bright spot in the image. Calculate the exposure time using the formula: E = 1 / (kL + b). Where k is a preset first constant and b is a preset second constant; If there is no reflective area for the license plate in the image, the exposure parameters of the image sensor are adjusted according to the calculated exposure time. If there is a reflective area of ​​a license plate in the image, the calculated exposure time is further shortened according to a preset rule, and the exposure parameters of the image sensor are adjusted according to the shortened exposure time. The reflective area of ​​the license plate is the area within the license plate area where the gray value is within a preset reflective gray value range, and the maximum value of the preset reflective gray value range is equal to the preset gray value threshold.

7. The method for dynamic adjustment of license plate capture based on cross-modal fusion according to claim 1, characterized in that, The preset standard includes: performing noise reduction processing on the image frames acquired after image gain adjustment according to a preset noise reduction algorithm, so that the noise level of the image frames after noise reduction is lower than a preset noise threshold.

8. The method for dynamic adjustment of license plate capture based on cross-modal fusion according to claim 1, characterized in that, Extracting the vehicle's license plate characters from image frames whose quality meets the preset standard includes: using a deep learning model based on a convolutional neural network to extract the overall image information and license plate characters of the vehicle from image frames whose quality meets the preset standard.

9. An electronic device, said electronic device being an embedded device, characterized in that, include: Memory, used to store program instructions; as well as A processor is configured to execute the program instructions to implement the steps of the license plate capture dynamic adjustment method based on cross-modal fusion as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the steps of the license plate capture dynamic adjustment method based on cross-modal fusion as described in any one of claims 1 to 8.

Images