A patrol drone roadside license plate recognition and binding system

Abstract

This invention discloses a roadside license plate recognition and binding system for patrol drones. The system uses a drone data acquisition module to collect license plate images of roadside vehicles; a license plate image enhancement module uses an adaptive image enhancement algorithm to adjust the brightness of the license plate image, generating an initial roadside license plate image; a license plate occlusion compensation module uses an occlusion compensation algorithm to fill in the occluded and missing parts of the initial roadside license plate image, generating a target roadside license plate image; an image verification module verifies the integrity of the target roadside license plate image, generating license plate verification data; and a license plate data binding module binds the target roadside license plate image with the corresponding vehicle location data when the license plate verification data passes, generating license plate binding data. By employing image acquisition, adaptive enhancement, occlusion compensation, integrity verification, and data binding techniques, the system effectively improves image quality, restores occluded license plate information, ensures recognition accuracy, and achieves precise binding of license plates to locations.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and more specifically to a roadside license plate recognition and binding system for patrol UAVs. Background Technology

[0002] With their advantages of flexibility, maneuverability, and ability to quickly cover large areas, drones are increasingly being used in traffic violation patrols. Patrol drones in this field are used to automatically identify license plates of roadside vehicles, while simultaneously collecting vehicle license plate information, location information, and vehicle images, which are then transmitted to a back-end management system in real time. This allows traffic management departments to monitor the dynamic information of roadside vehicles in real time.

[0003] Due to the complex and ever-changing roadside environment, existing patrol drones are highly susceptible to environmental interference, significantly reducing the accuracy of license plate recognition results. In poor lighting conditions, such as at night, in backlight, or under direct sunlight, license plate images captured by drone cameras are prone to being too dark, overexposed, or reflective, resulting in blurred license plate characters and severely impacting the accuracy of the license plate recognition algorithm. Furthermore, obstacles such as trees, buildings, and billboards around urban roads can easily obscure license plates, causing missed shots or incomplete images, further affecting the accuracy of license plate recognition and reducing the reliability of the license plate-location information binding, thus failing to provide accurate and effective data support for traffic management. Summary of the Invention

[0004] The invention provides a roadside license plate recognition and binding system for patrol drones, which solves the technical problem that existing patrol drones are easily affected by environmental factors, resulting in a significant reduction in the accuracy of license plate recognition results.

[0005] The present invention provides a roadside license plate recognition and binding system for patrol drones, comprising: a drone data acquisition module, a license plate image enhancement module, a license plate occlusion compensation module, an image verification module, and a license plate data binding module;

[0006] The drone data acquisition module is used to collect license plate images and environmental data of vehicles parked on the roadside.

[0007] The license plate image enhancement module is used to adjust the image brightness of the license plate image using an adaptive image enhancement algorithm to generate an initial roadside license plate image;

[0008] The license plate occlusion compensation module is used to use an occlusion compensation algorithm to complete the occluded and missing parts in the initial roadside license plate image and generate a target roadside license plate image.

[0009] The image verification module is used to verify whether the target roadside license plate image is complete and generate license plate verification data;

[0010] The license plate data binding module is used to bind the target roadside license plate image with the corresponding vehicle location data in the environmental data when the license plate verification data passes the verification, thereby generating license plate binding data.

[0011] Optionally, the UAV data acquisition module includes: a UAV and a sensor module and an image acquisition module mounted on the UAV, wherein the image acquisition module includes a variable-focus infrared night vision lens and a polarizing filter lens;

[0012] The image acquisition module is used to acquire images of license plates of vehicles parked on the roadside;

[0013] The sensor module is used to collect light intensity data, temperature data, humidity data, vehicle speed data, and vehicle position data of the roadside vehicles to construct environmental data.

[0014] Optionally, the execution steps of the license plate image enhancement module include:

[0015] Step S11: Divide the license plate image into multiple scales to generate multiple image regions;

[0016] Step S12: Calculate the mean brightness and variance brightness of the image region respectively;

[0017] Step S13: Compare the mean brightness and the variance brightness with the darkest threshold, the brightest threshold, and the variance threshold respectively to determine the initial darkest region, the initial brightest region, and the initial uneven brightness region.

[0018] Step S14: Use an adaptive histogram equalization algorithm to enhance the contrast of the initial dark areas, the initial bright areas, and the initial uneven brightness areas to construct an initial enhanced image;

[0019] Step S15: Based on the environmental data, a dynamic threshold filtering algorithm is used to dynamically suppress noise in the initial enhanced image to generate a target enhanced image;

[0020] Step S16: Perform image color correction on the target enhanced image to generate an initial roadside license plate image.

[0021] Optionally, the specific steps of step S14 are as follows:

[0022] Step S141: Perform histogram equalization and difference fusion on the initial dark region to generate the target dark region;

[0023] Step S142: Perform reverse equalization and difference fusion on the initial overbright area to generate the target overbright area;

[0024] Step S143: Divide the initial uneven brightness region into sub-blocks to produce multiple initial brightness sub-blocks;

[0025] Step S144: Perform adaptive histogram equalization on the initial brightness sub-blocks respectively to generate target brightness sub-blocks;

[0026] Step S145: Use a weighted average method to perform cross-block information fusion on the target brightness sub-block to generate a target brightness non-uniform region;

[0027] Step S146: Integrate the dark areas, bright areas, and uneven brightness areas of the target to generate an initial enhanced image.

[0028] Optionally, the specific steps of step S15 are as follows:

[0029] Step S151: Input the environmental data into a preset environmental noise model to predict the noise data of the initial enhanced image;

[0030] Step S152: Divide the initial enhanced image into sub-regions to generate multiple sub-regions;

[0031] Step S153: Calculate the regional features of the sub-regions respectively;

[0032] Step S154: Adjust the global threshold corresponding to the noise data according to the regional characteristics and the noise data to generate a dynamic threshold;

[0033] Step S155: Apply an adaptive filtering method to the initial enhanced image according to the dynamic threshold to suppress noise and generate the target enhanced image.

[0034] Optionally, the specific steps of step S16 are as follows:

[0035] Step S161: Perform semantic segmentation on the target enhanced image to generate an initial license plate character area, an initial background color area, and an initial background area;

[0036] Step S162: Use a high-fidelity color mapping algorithm to perform color correction on the initial license plate character area to generate the target license plate character area;

[0037] Step S163: Set the initial background color area adjustment value to a preset color reference to generate the target background color area;

[0038] Step S164: Perform appropriate color balancing on the initial background area to generate the target background area;

[0039] Step S165: Construct an initial roadside license plate image using the target license plate character area, the target background color area, and the target background area.

[0040] Optionally, the execution steps of the license plate occlusion compensation module include:

[0041] Step S21: Use a deep learning model to perform semantic segmentation and occlusion recognition on the initial roadside license plate image to generate occlusion areas;

[0042] Step S22: Extract the contour features and texture data of the occluded area;

[0043] Step S23: Use the contour features and texture data to supplement the occluded areas of the initial roadside license plate image to generate the target roadside license plate image.

[0044] Optionally, the specific steps of step S23 are as follows:

[0045] Step S231: Use a generative adversarial network to predict the character structure corresponding to the occluded region and generate an initial character outline;

[0046] Step S232: Use the contour features and the texture data to smoothly migrate the initial character contour to generate the target character contour;

[0047] Step S233: Update the initial roadside license plate image using the target character contour to generate the target roadside license plate image.

[0048] Optionally, the execution steps of the image verification module include:

[0049] Step S31: Extract the license plate region from the target roadside license plate image using a preset semantic segmentation model;

[0050] Step S32: Perform character recognition on the license plate area to generate character count and character position data;

[0051] Step S33: When both the number of characters and the character position data meet the preset character threshold, the license plate verification data is set to verification passed;

[0052] Step S34: When either the number of characters or the character position data does not meet the preset character threshold, the license plate verification data is set to verification failure.

[0053] Optionally, the execution steps of the license plate data binding module include:

[0054] Step S41: When the license plate verification data fails, the flight path of the UAV is adjusted using the corresponding vehicle location data in the environmental data, and the flight path is sent to the UAV data acquisition module to re-acquire data.

[0055] Step S42: When the license plate verification data is verified, the target roadside license plate image is bound to the corresponding vehicle location data in the current environmental data to generate license plate binding data.

[0056] As can be seen from the above technical solutions, the present invention has the following advantages:

[0057] The roadside license plate recognition and binding system for patrol drones provided by this invention includes: a drone data acquisition module, a license plate image enhancement module, a license plate occlusion compensation module, an image verification module, and a license plate data binding module. The drone data acquisition module collects license plate images and environmental data, providing multi-dimensional information support for subsequent processing. The license plate image enhancement module adaptively optimizes image brightness, improves license plate image quality, and enhances clarity. The license plate occlusion compensation module fills in the occluded parts of the license plate, restoring complete information and reducing recognition errors caused by occlusion. The image verification module verifies the integrity of the license plate image, ensuring the accuracy and reliability of the data entering the binding stage. After successful verification, the license plate data binding module binds the license plate and location data, providing accurate data for traffic management and other purposes. Attached Figure Description

[0058] Figure 1 This is a schematic diagram of a roadside license plate recognition and binding system for patrol drones according to the present invention;

[0059] Figure 2 This is a schematic diagram illustrating the execution steps of the license plate image enhancement module in a roadside license plate recognition and binding system for patrol drones according to the present invention.

[0060] Figure 3 This is a schematic diagram illustrating the execution steps of a license plate occlusion compensation module in a roadside license plate recognition and binding system for patrol drones according to the present invention.

[0061] Figure 4 This is a schematic diagram illustrating the execution steps of the image verification module in a roadside license plate recognition and binding system for patrol drones according to the present invention.

[0062] Figure 5 This is a schematic diagram illustrating the execution steps of the license plate data binding module in a roadside license plate recognition and binding system for patrol drones according to the present invention. Detailed Implementation

[0063] To better understand the technical content of this invention, specific embodiments are provided below, and the invention will be further described in conjunction with the accompanying drawings.

[0064] See Figures 1 to 5 A roadside license plate recognition and binding system for patrol drones includes: a drone data acquisition module, a license plate image enhancement module, a license plate occlusion compensation module, an image verification module, and a license plate data binding module;

[0065] The drone data acquisition module is used to collect license plate images and environmental data of vehicles parked on the roadside.

[0066] The license plate image enhancement module is used to adjust the image brightness of the license plate image using an adaptive image enhancement algorithm to generate an initial roadside license plate image.

[0067] The license plate occlusion compensation module is used to complete the occluded and missing parts in the initial roadside license plate image using an occlusion compensation algorithm, and generate the target roadside license plate image.

[0068] The image verification module is used to verify whether the target roadside license plate image is complete and generate license plate verification data;

[0069] The license plate data binding module is used to bind the target roadside license plate image with the corresponding vehicle location data in the environmental data when the license plate verification data passes the verification, thereby generating license plate binding data.

[0070] In embodiments of the present invention, such as Figure 1 As shown, the drone data acquisition module, license plate image enhancement module, license plate occlusion compensation module, and image verification module are connected in sequence. One end of the license plate data binding module is connected to the drone data acquisition module, and the other end is connected to the image verification module.

[0071] The drone's data acquisition module collects real-time images of vehicle license plates and environmental data from the roadside, including but not limited to information such as light intensity at the time of capture, weather conditions (e.g., sunny, cloudy, rainy), shooting angle, and surrounding environmental features. This comprehensive environmental data provides a foundation for subsequent processing. Given the complex and variable roadside environment, different light intensities, weather conditions, and shooting angles can all affect license plate image quality. Collecting this data allows subsequent modules to process license plate images specifically based on the actual environmental conditions, avoiding the neglect of environmental factors that can interfere with the recognition results when considering only the license plate image.

[0072] The license plate image acquired by the UAV data acquisition module is input into the license plate image enhancement module. This module uses an adaptive image enhancement algorithm to analyze and adjust the brightness of the license plate image. Based on the actual brightness of the image, it dynamically enhances or weakens the image brightness to highlight the key information of the license plate, generating an initial roadside license plate image. Roadside lighting conditions are often unstable, potentially including excessive brightness, excessive darkness, or backlighting, all of which can lead to blurred license plate images and unclear details, affecting license plate recognition. The adaptive image enhancement algorithm can automatically optimize the brightness of the license plate image according to different lighting conditions, making the license plate characters more clearly visible, effectively improving image quality, reducing recognition errors caused by lighting issues, and improving the accuracy of license plate recognition.

[0073] The initial roadside license plate image generated by the license plate image enhancement module is transmitted to the license plate occlusion compensation module. This module uses an occlusion compensation algorithm to analyze and complete the occluded and missing parts of the image. By learning and inferring from the shape, character distribution, and other features of the license plate, it predicts the content of the occluded part, thereby generating the target roadside license plate image. Roadside vehicles may have partially obstructed information due to surrounding objects (such as trees, other vehicles, etc.) or damage to the license plate itself, affecting recognition. The license plate occlusion compensation module can complete these missing parts, restoring the complete information of the license plate and avoiding situations where the license plate number cannot be accurately identified due to partial occlusion, further improving the applicability and accuracy of the license plate recognition system in complex roadside environments.

[0074] The target roadside license plate image generated by the license plate occlusion compensation module is input into the image verification module, which performs integrity verification on the image. By checking whether the characters and borders of the license plate are complete, it determines whether the license plate image has been incorrectly filled in or has other integrity issues, generating license plate verification data. Although the license plate occlusion compensation module fills in missing parts of the license plate, the filling results may be inaccurate. The image verification module can promptly detect these problems, ensuring that the license plate image input to the license plate data binding module is complete and accurate. Only license plate images that pass verification will proceed to the subsequent binding steps, avoiding incorrect binding of license plate and vehicle location data due to erroneous license plate images, thus ensuring the reliability of the license plate recognition and binding system.

[0075] When the license plate verification data generated by the image verification module passes verification, the license plate data binding module binds the target roadside license plate image with the corresponding vehicle location data from the environmental data acquired by the drone data acquisition module. This records the license plate number and the vehicle's specific location on the roadside, generating license plate binding data. Accurate binding of license plate images with vehicle location data provides reliable data support for subsequent traffic management, vehicle tracking, and other applications. In complex roadside environments, accurate license plate recognition and binding allow for a clear understanding of each vehicle's location, improving traffic management efficiency, enabling precise vehicle monitoring and management, and fully leveraging the role of patrol drones in roadside license plate recognition scenarios.

[0076] The various modules of the patrol drone roadside license plate recognition and binding system provided by this invention work together to comprehensively ensure the accuracy of license plate recognition and the reliability of the system, taking into account the complex and ever-changing environmental factors on the roadside, from data acquisition, image enhancement, occlusion compensation, image verification to data binding. It effectively solves the technical problem that the accuracy of license plate recognition results of existing patrol drones is greatly reduced due to environmental interference.

[0077] Furthermore, the UAV data acquisition module includes: the UAV and a sensor module and an image acquisition module mounted on the UAV. The image acquisition module includes a variable-focus infrared night vision lens and a polarizing filter lens.

[0078] Image acquisition module, used to acquire images of license plates of vehicles parked on the roadside;

[0079] The sensor module is used to collect data on light intensity, temperature, humidity, vehicle speed, and vehicle location of vehicles on the roadside to construct environmental data.

[0080] In this embodiment of the invention, a drone serves as the carrier of the entire data acquisition module, responsible for patrolling the roadside environment according to a predetermined route or instructions, reaching suitable locations to collect data from roadside vehicles. The sensor module includes a light sensor, a temperature sensor, a humidity sensor, a speed sensor (such as Doppler radar), and a Global Positioning System (GPS). The light sensor detects the lighting conditions of the roadside environment in real time. The temperature sensor obtains the roadside environmental temperature. Temperature changes affect the performance of the image acquisition equipment and the physical state of the license plate (e.g., high temperatures may cause slight deformation of the license plate, while low temperatures may cause frost or snow covering the license plate). Collecting temperature data helps the system analyze the potential impact of environmental factors on license plate recognition. The humidity sensor measures the ambient humidity. High humidity environments may cause fog or water vapor to condense on the license plate surface, resulting in blurred license plates; low humidity environments may cause static electricity to attract dust, contaminating the license plate. Accurately obtaining humidity data provides a basis for subsequent judgment of the causes of license plate image quality changes. A speed sensor (such as Doppler radar) measures the vehicle's speed. The movement of a vehicle affects the clarity of the captured image. If a vehicle is moving at high speed, license plate images captured by a standard lens are prone to blurring and motion blur. Knowing the vehicle's speed allows for adjustments to shooting parameters, such as shutter speed, in conjunction with the image acquisition module to obtain a clear license plate image. Vehicle location data is acquired by determining the vehicle's exact location on the roadside using the Global Positioning System (GPS). Comprehensive environmental data is constructed by using roadside vehicle illumination intensity data, temperature data, humidity data, vehicle speed data, and vehicle location data.

[0081] Furthermore, the execution steps of the license plate image enhancement module include:

[0082] Step S11: Divide the license plate image into multiple scales to generate multiple image regions;

[0083] Step S12: Calculate the mean and variance of brightness for each image region.

[0084] Step S13: Compare the mean brightness and the variance brightness with the darkest threshold, the brightest threshold, and the variance threshold respectively to determine the initial darkest region, the initial brightest region, and the initial uneven brightness region.

[0085] Step S14: Use an adaptive histogram equalization algorithm to enhance the contrast of the initial dark areas, the initial bright areas, and the initial uneven brightness areas to construct an initial enhanced image;

[0086] Step S15: Based on environmental data, a dynamic threshold filtering algorithm is used to dynamically suppress noise in the initial enhanced image to generate the target enhanced image;

[0087] Step S16: Perform image color correction on the target enhancement image to generate an initial roadside license plate image.

[0088] In embodiments of the present invention, such as Figure 2 As shown, the acquired license plate image is divided into multiple image regions at multiple scales. This step is based on the fact that different regions of the license plate image may have different lighting and detail conditions. For example, the upper half of the license plate may be too bright due to direct sunlight, while the lower half may be too dark due to shadows. By dividing the image into multiple local regions, it is possible to further refine the processing based on the characteristics of each region, avoiding the problem of over- or under-processing some areas due to applying a uniform processing method to the entire image.

[0089] The mean and variance of brightness are calculated for each image region. The mean brightness reflects the average brightness level of the region, while the variance reflects the dispersion of pixel brightness within the region, i.e., the uniformity of brightness. For example, a region with a low mean brightness and a small variance indicates that the region is generally dark and has relatively uniform brightness; a large variance indicates that there are significant brightness differences within the region. These two parameters allow for a quantitative assessment of the brightness characteristics of each image region, providing data support for subsequent region status determination.

[0090] The calculated mean and variance of brightness are compared with overly dark, overly bright, and variance thresholds, respectively, to determine the initial overly dark, overly bright, and uneven brightness regions. The overly dark, overly bright, and variance thresholds are pre-set based on a large amount of actual license plate image data and experience to define different brightness states. For example, when the mean brightness of an image region is lower than the overly dark threshold, it is determined to be an initial overly dark region; if the mean brightness is higher than the overly bright threshold, it is an initial overly bright region; and regions with a brightness variance exceeding the variance threshold are identified as initial uneven brightness regions. This method allows for precise location of areas in the image requiring enhancement, improving the targeting of the processing.

[0091] An adaptive histogram equalization algorithm is used to enhance the contrast of initially overly dark, overly bright, and unevenly lit areas, constructing an initial enhanced image. Compared to traditional histogram equalization algorithms, the adaptive histogram equalization algorithm dynamically adjusts the histogram distribution according to the characteristics of each local region, avoiding image distortion caused by global processing. For overly dark areas, the algorithm stretches their grayscale range to make dark details clearer; for overly bright areas, it appropriately compresses the grayscale range to prevent overexposure; for unevenly lit areas, it adjusts the grayscale distribution to make the brightness within the area more uniform, thereby enhancing the contrast of the entire image, making the license plate characters stand out more, and facilitating subsequent recognition.

[0092] A dynamic threshold filtering algorithm is used to dynamically suppress noise in the initial enhanced image based on environmental data, generating the target enhanced image. Factors such as changes in lighting and electromagnetic interference in the roadside environment can introduce noise into the acquired license plate images, affecting recognition performance. The dynamic threshold filtering algorithm dynamically adjusts the filtering threshold based on environmental data (such as light intensity and temperature). For example, in environments with strong light, sensor saturation may cause salt-and-pepper noise. In this case, the algorithm will increase the threshold based on the light intensity data to more effectively remove noise. When temperature changes cause fluctuations in image sensor performance, resulting in noise, the algorithm can also adaptively adjust based on temperature data, thereby maximizing noise suppression and improving image quality without sacrificing image details.

[0093] Image color correction is performed on the target enhanced image to generate an initial roadside license plate image. In complex environments, image acquisition equipment may experience color deviations due to factors such as lighting conditions and equipment characteristics, affecting the distinction between license plate characters and the background. Color correction adjusts the image's color channel parameters (such as the brightness and saturation of the red, green, and blue channels) to make the license plate image colors more realistic and vivid, enhance the contrast between license plate characters and the background, and further improve the clarity and readability of the license plate image, providing a high-quality image foundation for subsequent license plate recognition.

[0094] Furthermore, the specific steps of step S14 are as follows:

[0095] Step S141: Perform histogram equalization and difference fusion on the initial dark areas to generate the target dark areas;

[0096] Step S142: Perform reverse equalization and difference fusion on the initial overbright area to generate the target overbright area;

[0097] Step S143: Divide the initial uneven brightness region into sub-blocks to produce multiple initial brightness sub-blocks;

[0098] Step S144: Perform adaptive histogram equalization on the initial brightness sub-blocks to generate target brightness sub-blocks;

[0099] Step S145: Use a weighted average method to perform cross-block information fusion on the target brightness sub-blocks to generate a target brightness non-uniform region;

[0100] Step S146: Integrate the dark areas, bright areas, and uneven brightness areas of the target to generate an initial enhanced image.

[0101] In this embodiment of the invention, initially dark areas often obscure the details of license plate characters due to shadows, insufficient nighttime lighting, or other reasons. Histogram equalization is applied to these areas. This algorithm redistributes image grayscale values, stretching the image's grayscale range and making the pixel distribution, originally concentrated in low grayscale values, more uniform, thereby improving the overall brightness of the image. For example, on a license plate obscured by tree shadows, characters in the originally dark areas are difficult to discern; after histogram equalization, the details in the dark areas are enhanced. However, histogram equalization may introduce noise or over-enhance the background. Therefore, a difference fusion technique is used to calculate the difference between the histogram equalized image and the original image, preserving the true license plate character information while removing unnecessary noise and background interference, generating the target dark areas, and making the license plate characters in these dark areas clearly visible.

[0102] Initially overexposed areas are often caused by direct sunlight or strong light source reflections. Excessive brightness leads to overexposure of license plate characters, resulting in loss of detail. Inverse equalization adjusts the grayscale values ​​of these overexposed areas, compressing their grayscale range and reducing their brightness. For example, in areas of the license plate with severe reflections under direct sunlight, inverse equalization can effectively prevent the characters from becoming blurry due to overexposure. Similarly, the subsequent difference fusion step compares the inversely equalized image with the original image to remove any abnormal changes that may have been caused by inverse equalization, preserving the true information of the license plate and generating the target overexposed areas, thus restoring the details of the license plate characters in these overexposed areas.

[0103] Uneven initial brightness areas typically appear when parts of the license plate are illuminated while others are in shadow, or when the surface reflection of the license plate is inconsistent. Dividing this area into sub-blocks to generate multiple initial brightness sub-blocks allows for more refined processing of the brightness differences in each small area. Because the brightness variations of different sub-blocks differ, uniform processing is unlikely to achieve ideal results. After division, targeted enhancement can be performed based on the specific characteristics of each sub-block. Adaptive histogram equalization is applied to each initial brightness sub-block. The adaptive histogram equalization algorithm can dynamically adjust the histogram equalization parameters according to the grayscale distribution characteristics of the sub-block itself, avoiding the over-enhancement or under-enhancement problems that may occur when processing local areas by traditional global histogram equalization. For example, for a sub-block that contains both bright and dark areas, adaptive histogram equalization can make reasonable grayscale adjustments to the bright and dark parts separately, making the brightness distribution within the sub-block more uniform, enhancing the contrast between the license plate characters and the background, generating the target brightness sub-block, and effectively improving the image quality of areas with uneven brightness.

[0104] A weighted averaging method is used to fuse cross-block information of target brightness sub-blocks. Since each sub-block may have boundary differences during processing, direct stitching can lead to image discontinuities and stitching artifacts. The weighted averaging method assigns different weights to sub-blocks based on factors such as position and brightness, fusing information from adjacent sub-blocks, smoothing the transition between sub-blocks, and making the entire target brightness uneven area appear more natural and coherent. This eliminates the adverse effects of sub-block division and individual processing, generating a target brightness uneven area and improving the overall image quality. The processed target overly dark areas, target overly bright areas, and target brightness uneven areas are then integrated to generate an initial enhanced image. Through the previous refined processing of different brightness areas, the clarity and contrast of license plate characters in each area are significantly improved. After integrating these areas, an image with enhanced overall contrast and prominent license plate characters is formed, providing a high-quality image foundation for subsequent image verification and license plate recognition. This effectively solves the problem of difficulty in license plate recognition due to brightness issues in complex roadside environments.

[0105] Furthermore, the specific steps of step S15 are as follows:

[0106] Step S151: Input the environmental data into the preset environmental noise model to predict the noise data of the initial enhanced image;

[0107] Step S152: Divide the initial enhanced image into sub-regions to generate multiple sub-regions;

[0108] Step S153: Calculate the regional features of each sub-region;

[0109] Step S154: Adjust the global threshold corresponding to the noise data according to the regional characteristics and noise data to generate a dynamic threshold;

[0110] Step S155: Apply an adaptive filtering method to the initial enhanced image according to the dynamic threshold to suppress noise and generate the target enhanced image.

[0111] In this embodiment of the invention, environmental data acquired by the UAV data acquisition module, such as light intensity, temperature, humidity, and electromagnetic environment around the vehicle, is input into a preset environmental noise model. This model is trained based on a large amount of real-world environmental data and image noise features, and can predict the type, intensity, and distribution of noise that may exist in the initial enhanced image according to different environmental conditions. For example, in high-temperature environments, the probability of thermal noise generated by the image sensor increases, and the model can predict the approximate intensity of thermal noise; in areas with complex electromagnetic environments, it can estimate the distribution range of salt-and-pepper noise. In this way, noise data of the image is obtained in advance, providing a basis for subsequent targeted noise suppression.

[0112] The initial enhanced image is divided into sub-regions, generating multiple sub-regions. This is because the noise distribution and image content characteristics of different regions within an image often differ; for example, the noise sensitivity of the license plate character region and the background region differs. Sub-region division allows for refined processing tailored to the specific circumstances of each sub-region, avoiding the problem of losing image details or incomplete noise suppression when applying a uniform noise suppression strategy to the entire image. For instance, the license plate character region needs to preserve character edge details as much as possible while suppressing noise, while the background region can have its noise suppression level appropriately enhanced.

[0113] The regional features of each sub-region are calculated separately. These features include, but are not limited to, the mean gray level, gray level variance, and texture features of the sub-region. The mean gray level reflects the average brightness level of the sub-region, the gray level variance reflects the dispersion of pixel gray levels within the sub-region, and the texture features describe the arrangement and distribution patterns of pixels within the sub-region. For example, the texture features of the license plate character sub-region usually have obvious regularity (the stroke structure of the characters); while the texture of the background sub-region is relatively messy. By calculating these regional features, we can gain a deeper understanding of the image content and characteristics of each sub-region, providing a reference for subsequent adjustment of noise suppression parameters.

[0114] Based on the regional characteristics of the sub-regions and the noise data predicted in step S151, the global threshold corresponding to the noise data is adjusted to generate a dynamic threshold. The global threshold is a preset initial threshold used to determine whether a pixel is noise. However, since the image content and noise conditions of different sub-regions are different, a uniform global threshold cannot meet the noise suppression requirements of all sub-regions. Therefore, the global threshold is dynamically adjusted based on the gray-scale mean, variance, and texture characteristics of the sub-regions, combined with the predicted noise data. For example, for sub-regions with high noise intensity and few image details, the threshold is appropriately lowered to more strictly remove noise; for sub-regions containing important license plate character information, the threshold is increased to avoid accidentally deleting character details, thus obtaining a dynamic threshold suitable for each sub-region.

[0115] Based on the generated dynamic threshold, an adaptive filtering method is used to suppress noise in the initial enhanced image, generating the target enhanced image. The adaptive filtering method automatically adjusts the filtering parameters according to the dynamic threshold of each sub-region, performing targeted noise removal for different sub-regions. For example, the filtering intensity is increased for sub-regions with more noise, and decreased for sub-regions with rich image details, effectively suppressing noise while preserving the maximum amount of detail in the license plate image. After this step, noise in the image is effectively removed, the license plate characters are clearer, and the image quality is further improved.

[0116] Furthermore, the specific steps of step S16 are as follows:

[0117] Step S161: Perform semantic segmentation on the target enhanced image to generate an initial license plate character area, an initial background color area, and an initial background area;

[0118] Step S162: Use a high-fidelity color mapping algorithm to perform color correction on the initial license plate character area to generate the target license plate character area;

[0119] Step S163: Set the initial background color area adjustment value to a preset color reference to generate the target background color area;

[0120] Step S164: Perform appropriate color balancing on the initial background area to generate the target background area;

[0121] Step S165: Construct an initial roadside license plate image using the target license plate character area, target background color area, and target background area.

[0122] In this embodiment of the invention, the target enhanced image is semantically segmented to generate an initial license plate character area, an initial background color area, and an initial background area. In a license plate image, the license plate characters, background color, and background have different features and semantic information. Semantic segmentation allows for the accurate division of these different regions. For example, license plate characters typically have specific shapes and colors, background colors have fixed color specifications, while backgrounds are relatively complex and diverse. Semantic segmentation provides a foundation for subsequent targeted color correction of different regions.

[0123] A high-fidelity color mapping algorithm is used to correct the color of the initial license plate character area to generate the target license plate character area. Under complex lighting conditions, the color of the license plate characters may deviate, making them difficult to identify. The high-fidelity color mapping algorithm can establish a mapping relationship between the standard color of the license plate characters and the actual captured colors, and precisely adjust the character colors. This algorithm can preserve the details and texture information of the characters as much as possible while ensuring color accuracy, making the license plate characters clearer and more distinguishable.

[0124] The initial background color area is adjusted to a preset color reference to generate the target background color area. License plate background colors typically have clear color standards, such as blue or yellow. Due to environmental factors, the background color may deviate. By adjusting the color of the initial background color area to a preset color reference, the background color can be restored to the standard color, enhancing the overall standardization and recognizability of the license plate image.

[0125] The initial background area undergoes appropriate color balancing to generate the target background area. The background color may be affected by the surrounding environment, resulting in uneven or overly vibrant colors, which could interfere with license plate recognition. Appropriate color balancing adjusts the background color to make it more natural and harmonious, reducing interference with the license plate itself. Simultaneously, it avoids over-processing that could lead to loss of background information or the introduction of new visual distractions.

[0126] An initial roadside license plate image is constructed using the target license plate character area, target background color area, and target background area. After color correction and adjustment of different areas in the previous steps, these processed areas are recombined to form a license plate image with accurate and clear colors.

[0127] Furthermore, the execution steps of the license plate occlusion compensation module include:

[0128] Step S21: Use a deep learning model to perform semantic segmentation and occlusion recognition on the initial roadside license plate image to generate occlusion regions;

[0129] Step S22: Extract the contour features and texture data of the occluded area;

[0130] Step S23: Use contour features and texture data to fill in the occluded areas of the initial roadside license plate image to generate the target roadside license plate image.

[0131] In embodiments of the present invention, such as Figure 3 As shown, a deep learning model is used to perform semantic segmentation and occlusion identification on the initial roadside license plate image, generating occlusion regions. Semantic segmentation is a technique that classifies different semantic categories (such as license plate characters, background color, occlusions, etc.) in an image at the pixel level. Deep learning models (such as convolutional neural networks, CNNs) are trained on a large amount of labeled data and can learn the features and patterns of different objects. In this step, the model performs pixel-by-pixel analysis on the initial roadside license plate image, identifies which areas belong to occlusions, and marks these areas to generate information about the occlusion regions. For example, if the license plate is partially obscured by tree branches, the model can accurately identify the area where the branches are located and use it as the occlusion region.

[0132] Extract the contour features and texture data of the occluded area. Contour features describe the shape and boundary information of the occluded area, such as whether the occluder is a regular rectangle or an irregular shape. Texture data reflects the details and texture patterns of the occluded area's surface, such as whether the surface of the occluder is smooth or rough. By extracting these features and data, we can gain a deeper understanding of the characteristics of the occluded area, providing a basis for subsequent occlusion area supplementation. For example, for a license plate area obscured by leaves, the extracted texture data can reflect the texture features of the leaves, which helps to simulate a more realistic effect when supplementing the occluded area.

[0133] The system uses contour features and texture data to fill in occluded areas in the initial roadside license plate image, generating the target roadside license plate image. In this step, the system uses extracted contour features and texture data, combined with the overall features of the license plate and common license plate characters and background color patterns, to fill in the occluded areas. For example, if the occluded area covers part of the license plate characters, the system can infer the possible content of the occluded characters based on known license plate character patterns and information about surrounding unoccluded characters, and then fill in the missing parts. Simultaneously, texture data is used to ensure the supplemented area matches the surrounding environment in texture, making the generated target roadside license plate image look more natural and realistic.

[0134] Furthermore, the specific steps of step S23 are as follows:

[0135] Step S231: Use a generative adversarial network to predict the character structure corresponding to the occluded region and generate the initial character outline;

[0136] Step S232: Use contour features and texture data to smoothly migrate the initial character contour to generate the target character contour;

[0137] Step S233: Update the initial roadside license plate image using the target character contour to generate the target roadside license plate image.

[0138] In this embodiment of the invention, a Generative Adversarial Network (GAN) is used to predict the character structure corresponding to the occluded region and generate an initial character contour. The GAN consists of a generator and a discriminator. In the license plate occlusion compensation scenario, the generator's task is to attempt to generate possible character structures based on the input license plate image information (including characters in the unoccluded portion, the license plate background color, etc.) and the location of the occluded region; the discriminator is responsible for judging whether the character structure generated by the generator is realistic and reasonable. During training, the generator continuously adjusts its parameters to generate increasingly realistic character structures to deceive the discriminator; the discriminator also continuously optimizes to improve its ability to distinguish between real and fake characters. After training with a large amount of data, the GAN can learn the common structures and patterns of license plate characters. When faced with an actual occluded region, the generator can predict the most likely character structure of that region and convert it into an initial character contour.

[0139] The initial character outline is smoothly migrated using contour features and texture data to generate the target character outline. The contour features and texture data of the occlusion area extracted in step S22 contain the shape and surface details of the occlusion. This step applies these features and data to the initial character outline to better integrate the supplemented character outline with the surrounding environment. The smooth migration process adjusts the edges of the initial character outline according to the shape of the occlusion area, ensuring a natural connection with the boundary of the occlusion and avoiding abrupt edges. Simultaneously, based on the texture data, the surface texture of the character outline is simulated to match the texture of the supplemented character with the background color of the surrounding license plate, occlusion, etc. For example, if the occlusion is a rough billboard, the generated character outline surface will also simulate a similar rough texture effect.

[0140] The initial roadside license plate image is updated using the target character contours to generate the target roadside license plate image. The smoothly migrated target character contours are then accurately embedded into the occluded area of ​​the initial roadside license plate image, replacing the original occluded portion. During the embedding process, the system adjusts the image pixels according to the shape, color, and other information of the character contours to ensure that the supplemented characters are consistent with the original license plate image in terms of color, brightness, etc. Through this step, the occluded license plate area is effectively filled in, the previously missing character information is restored, and a complete target roadside license plate image is generated.

[0141] Furthermore, the execution steps of the image verification module include:

[0142] Step S31: Extract the license plate region from the target roadside license plate image using a preset semantic segmentation model;

[0143] Step S32: Perform character recognition on the license plate area to generate character count and character position data;

[0144] Step S33: When both the number of characters and the character position data meet the preset character threshold, set the license plate verification data to "verification passed".

[0145] Step S34: When either the number of characters or the character position data does not meet the preset character threshold, the license plate verification data is set to verification failure.

[0146] In embodiments of the present invention, such as Figure 4 As shown, a pre-defined semantic segmentation model is used to extract the license plate region from a target roadside license plate image. This model, trained on a large dataset of license plate images, accurately identifies the license plate region from the background. In roadside environments, license plate images may encounter complex background interference, such as buildings, trees, and billboards. This model utilizes deep learning algorithms to analyze the features of various parts of the image (such as color, texture, and shape) to separate the license plate region from the background. For example, by recognizing the unique color combinations, rectangular shape, and character arrangement patterns of the license plate, the model accurately delineates the area containing the license plate.

[0147] The extracted license plate area undergoes character recognition to generate character count and location data. The character recognition process utilizes Optical Character Recognition (OCR) technology or deep learning character recognition algorithms to identify each character within the license plate area. During recognition, the system records the specific content of each character and determines its precise location within the license plate area, thereby generating character count and location data. In complex roadside environments, license plate characters may appear blurred or distorted due to factors such as lighting, wear, and occlusion compensation. The character recognition algorithm uses deep analysis and matching of character features to identify characters as accurately as possible. For example, for partially blurred characters, the algorithm combines the character's contextual information and the shape features of common license plate characters to infer their accuracy, ensuring the accuracy of the recognition results and generating corresponding character count and location data.

[0148] When both the number of characters and their positions meet the preset character thresholds, the license plate verification data is set to pass verification. The preset character thresholds are set according to the standard format and requirements of license plate numbers, including the range of character counts (e.g., ordinary civilian vehicle license plates typically have 7 characters) and the reasonable position range of characters within the license plate area. If the number of characters recognized in the license plate area matches the standard number of characters for a license plate number, and the position of each character is within a reasonable range, it indicates that the license plate information in the image is complete and accurate, and the license plate verification data is set to pass verification. This means that the license plate image processed by the previous modules meets the requirements for license plate recognition and data binding, and can proceed to the subsequent license plate data binding process.

[0149] If either the number of characters or the character position data fails to meet the preset character threshold, the license plate verification data is set to verification failure. Any discrepancy in the number of characters or the presence of characters outside a reasonable range indicates a potential problem with the license plate image, such as missing characters, misaligned characters, or incorrect occlusion compensation. For example, if the number of recognized characters is less than the standard number of characters in a license plate number, it may mean that some characters are obscured and not correctly compensated; if the character positions deviate significantly from their normal positions, it may indicate a deviation during image enhancement or occlusion compensation. In such cases, setting the license plate verification data to verification failure prevents erroneous license plate images from entering the license plate data binding process, avoiding errors in subsequent traffic management, vehicle tracking, and other applications due to inaccurate license plate information.

[0150] Through the above execution steps of the image verification module, the target roadside license plate image is comprehensively and rigorously verified, effectively filtering out problematic license plate images and ensuring that only complete and accurate license plate images can enter the license plate data binding module, thereby improving the reliability and accuracy of the entire patrol drone roadside license plate recognition and binding system.

[0151] Furthermore, the execution steps of the license plate data binding module include:

[0152] Step S41: If the license plate verification data fails, adjust the drone's flight path using the corresponding vehicle location data in the environmental data, and send the flight path to the drone data acquisition module to re-acquire data.

[0153] Step S42: When the license plate verification data passes the verification, bind the target roadside license plate image with the corresponding vehicle location data in the current environmental data to generate license plate binding data.

[0154] In embodiments of the present invention, such as Figure 5As shown, when license plate verification fails, it means the image verification module determines that the target roadside license plate image has problems, such as missing characters or incorrect positions, and cannot meet the requirements for license plate recognition and data binding. In this case, the module adjusts the drone's flight path using the corresponding vehicle location data from the environmental data. The environmental data records the vehicle's specific location, including latitude and longitude, and the road segment it's on. By analyzing this location data, the module calculates the optimal flight path for the drone to re-collect data, such as adjusting the drone's altitude and angle to approach the vehicle from a more suitable position to obtain a clearer and more complete license plate image. Then, the planned flight path is sent to the drone's data acquisition module, triggering it to re-collect data from the vehicle. This process effectively avoids recognition errors caused by poor license plate image quality and increases the probability of obtaining valid license plate images by proactively adjusting the acquisition strategy. For example, in scenarios where license plate verification fails because the vehicle is partially obscured by a building, the drone can use the new flight path to circle to the other side of the building and re-capture the license plate image, thereby obtaining license plate data that meets the requirements.

[0155] When the license plate verification data passes, it indicates that the target roadside license plate image is complete and accurate, meeting the conditions for data binding. At this point, the license plate data binding module binds the target roadside license plate image with the corresponding vehicle location data in the current environmental data. In addition to vehicle location information, environmental data also includes light intensity, weather conditions, vehicle speed, and other information. Combining the license plate image with this rich environmental data generates detailed license plate binding data. For example, the binding data not only includes the license plate number and vehicle location but also records the lighting conditions at the time of capture, providing data support for subsequent analysis of the license plate recognition performance under different lighting conditions. Through accurate data binding, reliable vehicle information association data is provided for applications such as traffic management and security monitoring, enabling relevant departments to perform operations such as vehicle tracking and violation determination based on this data.

[0156] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A roadside license plate recognition and binding system for patrol drones, characterized in that, include: The system includes a drone data acquisition module, a license plate image enhancement module, a license plate occlusion compensation module, an image verification module, and a license plate data binding module. The drone data acquisition module is used to collect license plate images and environmental data of vehicles parked on the roadside. The license plate image enhancement module is used to adjust the image brightness of the license plate image using an adaptive image enhancement algorithm to generate an initial roadside license plate image; The license plate occlusion compensation module is used to use an occlusion compensation algorithm to complete the occluded and missing parts in the initial roadside license plate image and generate a target roadside license plate image. The image verification module is used to verify whether the target roadside license plate image is complete and generate license plate verification data; The license plate data binding module is used to bind the target roadside license plate image with the corresponding vehicle location data in the environmental data when the license plate verification data passes the verification, thereby generating license plate binding data. The execution steps of the license plate image enhancement module include: Step S11: Divide the license plate image into multiple scales to generate multiple image regions; Step S12: Calculate the mean brightness and variance brightness of the image region respectively; Step S13: Compare the mean brightness and the variance brightness with the darkest threshold, the brightest threshold, and the variance threshold respectively to determine the initial darkest region, the initial brightest region, and the initial uneven brightness region. Step S14: Use an adaptive histogram equalization algorithm to enhance the contrast of the initial dark areas, the initial bright areas, and the initial uneven brightness areas to construct an initial enhanced image; Step S15: Based on the environmental data, a dynamic threshold filtering algorithm is used to dynamically suppress noise in the initial enhanced image to generate a target enhanced image; Step S16: Perform image color correction on the target enhanced image to generate an initial roadside license plate image; The specific steps of step S14 are as follows: Step S141: Perform histogram equalization and difference fusion on the initial dark region to generate the target dark region; Step S142: Perform reverse equalization and difference fusion on the initial overbright area to generate the target overbright area; Step S143: Divide the initial uneven brightness region into sub-blocks to produce multiple initial brightness sub-blocks; Step S144: Perform adaptive histogram equalization on the initial brightness sub-blocks respectively to generate target brightness sub-blocks; Step S145: Use a weighted average method to perform cross-block information fusion on the target brightness sub-block to generate a target brightness non-uniform region; Step S146: Integrate the dark areas, bright areas, and uneven brightness areas of the target to generate an initial enhanced image.

2. The patrol drone roadside license plate recognition and binding system according to claim 1, characterized in that, The UAV data acquisition module includes: a UAV and a sensor module and an image acquisition module mounted on the UAV. The image acquisition module includes a variable-focus infrared night vision lens and a polarizing filter lens. The image acquisition module is used to acquire images of license plates of vehicles parked on the roadside; The sensor module is used to collect light intensity data, temperature data, humidity data, vehicle speed data, and vehicle position data of the roadside vehicles to construct environmental data.

3. The patrol drone roadside license plate recognition and binding system according to claim 1, characterized in that, The specific steps of step S15 are as follows: Step S151: Input the environmental data into a preset environmental noise model to predict the noise data of the initial enhanced image; Step S152: Divide the initial enhanced image into sub-regions to generate multiple sub-regions; Step S153: Calculate the regional features of the sub-regions respectively; Step S154: Adjust the global threshold corresponding to the noise data according to the regional characteristics and the noise data to generate a dynamic threshold; Step S155: Apply an adaptive filtering method to the initial enhanced image according to the dynamic threshold to suppress noise and generate the target enhanced image.

4. The patrol drone roadside license plate recognition and binding system according to claim 1, characterized in that, The specific steps of step S16 are as follows: Step S161: Perform semantic segmentation on the target enhanced image to generate an initial license plate character area, an initial background color area, and an initial background area; Step S162: Use a high-fidelity color mapping algorithm to perform color correction on the initial license plate character area to generate the target license plate character area; Step S163: Set the initial background color area adjustment value to a preset color reference to generate the target background color area; Step S164: Perform appropriate color balancing on the initial background area to generate the target background area; Step S165: Construct an initial roadside license plate image using the target license plate character area, the target background color area, and the target background area.

5. The patrol drone roadside license plate recognition and binding system according to claim 1, characterized in that, The execution steps of the license plate occlusion compensation module include: Step S21: Use a deep learning model to perform semantic segmentation and occlusion recognition on the initial roadside license plate image to generate occlusion areas; Step S22: Extract the contour features and texture data of the occluded area; Step S23: Use the contour features and texture data to supplement the occluded areas of the initial roadside license plate image to generate the target roadside license plate image.

6. The patrol drone roadside license plate recognition and binding system according to claim 5, characterized in that, The specific steps of step S23 are as follows: Step S231: Use a generative adversarial network to predict the character structure corresponding to the occluded region and generate an initial character outline; Step S232: Use the contour features and the texture data to smoothly migrate the initial character contour to generate the target character contour; Step S233: Update the initial roadside license plate image using the target character contour to generate the target roadside license plate image.

7. The patrol drone roadside license plate recognition and binding system according to claim 1, characterized in that, The execution steps of the image verification module include: Step S31: Extract the license plate region from the target roadside license plate image using a preset semantic segmentation model; Step S32: Perform character recognition on the license plate area to generate character count and character position data; Step S33: When both the number of characters and the character position data meet the preset character threshold, the license plate verification data is set to verification passed; Step S34: When either the number of characters or the character position data does not meet the preset character threshold, the license plate verification data is set to verification failure.

8. The patrol drone roadside license plate recognition and binding system according to claim 1, characterized in that, The execution steps of the license plate data binding module include: Step S41: When the license plate verification data fails, the flight path of the UAV is adjusted using the corresponding vehicle location data in the environmental data, and the flight path is sent to the UAV data acquisition module to re-acquire data. Step S42: When the license plate verification data is verified, the target roadside license plate image is bound to the corresponding vehicle location data in the current environmental data to generate license plate binding data.

Images