Vehicle precise positioning and recognition system fusing multi-scale features

Abstract

The application belongs to the technical field of intelligent transportation, and discloses a vehicle precise positioning and identification system fusing multi-scale features; comprising a data acquisition module, a feature extraction module, a feature fusion module, a region detection module, a correlation binding module, a character recognition module, a database retrieval module and a verification correction module. Multi-scale feature maps are extracted through a lightweight target detection model, a shielding perception suppression mask is generated based on the response difference distribution of adjacent scale feature maps, and effective fusion of multi-scale features is realized. The matching affinity is calculated in the form of geometric offset vector combined with feature similarity, and the precise correlation binding of vehicles and license plates is completed through the maximum weight matching of two-part graphs. High-precision character recognition is realized by using perspective transformation correction and stroke direction features, and intelligent inference and verification of the load state are realized through visual morphological attribute feature analysis. The application improves the vehicle detection precision in the shielding scene and realizes consistent identification of vehicles of different sizes.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, and more specifically, to a vehicle precise positioning and identification system that integrates multi-scale features. Background Technology

[0002] Existing vehicle recognition technologies face several technical bottlenecks in bridge monitoring applications, failing to meet the demands of bridge management for accurate vehicle identification. In bridge entrance monitoring scenarios, existing vehicle detection methods lack effective occlusion handling mechanisms. When large vehicles pass parallel to or closely following small vehicles, traditional detection algorithms often fail to accurately identify occluded vehicles. Existing technologies exhibit inconsistent recognition accuracy for vehicles of different distances and sizes; the detection performance differs significantly between small vehicles approaching the bridge from a distance and large vehicles approaching from close range, failing to achieve comprehensive and accurate identification. Regarding vehicle identity verification, existing systems lack a reliable mechanism for associating vehicle and license plate areas. When multiple vehicles pass the bridge simultaneously, mismatches between license plates and vehicles can easily occur, affecting the accuracy of vehicle identification. Furthermore, existing methods lack robustness under complex lighting conditions, inclement weather, and damaged license plates on bridges, resulting in a significant decrease in character recognition accuracy. In addition, existing technologies lack the ability to determine load status based on vehicle visual features, failing to identify the actual load condition of vehicles through visual analysis and unable to verify the accuracy of vehicle registration information, thus limiting the ability of bridge management departments to comprehensively monitor passing vehicles. These technical deficiencies affect the practicality and reliability of bridge vehicle monitoring systems.

[0003] In view of this, the present invention proposes a vehicle precise positioning and identification system that integrates multi-scale features to solve the above problems. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of the prior art and to achieve the above objectives, the present invention provides the following technical solution: a vehicle precise positioning and identification system integrating multi-scale features, comprising: The data acquisition module is used to acquire vehicle video frames collected by traffic monitoring equipment; The feature extraction module is used to input vehicle video frames into a pre-built lightweight target detection model and extract multi-scale feature maps of vehicle video frames at different scales through the backbone network of the lightweight target detection model. The feature fusion module is used to generate an occlusion-aware suppression mask based on the response difference distribution between adjacent scale feature maps in the multi-scale feature map, and to use the occlusion-aware suppression mask to perform feature fusion on the multi-scale feature map to obtain a fused localization feature map. The region detection module is used to generate candidate vehicle regions and candidate license plate regions based on the decoding of fused localization feature maps, and to extract the local feature slices corresponding to the candidate license plate regions. The association binding module is used to construct a geometric offset vector based on the spatial mapping relationship between the candidate vehicle region and the candidate license plate region on the fused localization feature map, and calculate the matching affinity by combining local feature slices, so as to complete the association binding between the candidate vehicle region and the candidate license plate region based on the matching affinity, and determine the target vehicle region and the corresponding target license plate region. The character recognition module is used to perform character recognition on the target license plate area to obtain the license plate text information; The database retrieval module is used to retrieve the vehicle load database based on the license plate text information and obtain the standard load data corresponding to the license plate text information. The verification and correction module is used to extract the visual morphological attribute features of the target vehicle area, generate the visual inferred load range based on the visual morphological attribute features, and perform consistency verification and correction annotation on the standard load data based on the visual inferred load range.

[0005] The technical effects and advantages of the vehicle precise positioning and identification system integrating multi-scale features of this invention are as follows: This invention improves the overall performance of vehicle detection and recognition systems in complex traffic environments. In densely populated and occluded scenarios, it accurately identifies partially obscured vehicles, effectively solving the problems of missed detections and false detections inherent in traditional methods. This invention achieves consistent high-precision recognition of vehicles of different sizes, obtaining balanced detection results for both small targets at a distance and large targets at close range. Regarding vehicle-license plate association, this invention significantly improves matching accuracy, avoiding erroneous associations in multi-vehicle scenarios. This invention possesses intelligent load status judgment capabilities based on visual analysis, enabling it to identify the actual load conditions of vehicles and detect data anomalies, providing crucial support for traffic safety supervision. In character recognition, this invention exhibits excellent robustness under harsh conditions such as complex lighting, changing angles, and damaged license plates. This invention possesses adaptive performance adjustment capabilities, dynamically balancing detection accuracy and processing speed according to actual needs, meeting the real-time requirements of different scenarios. Attached Figure Description

[0006] Figure 1 This is a schematic diagram of the vehicle precise positioning and identification system that integrates multi-scale features according to the present invention. Detailed Implementation

[0007] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0008] like Figure 1 As shown, embodiments of the present invention propose a vehicle precise positioning and identification system that integrates multi-scale features. The system includes the following modules: The data acquisition module is used to acquire vehicle video frames collected by traffic monitoring equipment. This module acquires vehicle video streams in real time from different scenarios such as intersection monitoring, highway monitoring, and parking lot monitoring through a multi-channel data acquisition interface, ensuring data continuity and integrity. Vehicle video frames contain key data such as vehicle appearance information, license plate information, and driving status, providing raw input for subsequent feature extraction and recognition analysis. During data acquisition, metadata information such as timestamps, camera positions, and shooting angles is also recorded, providing fundamental support for the traceability and verification of payload data.

[0009] The feature extraction module takes vehicle video frames as input to a pre-built lightweight object detection model and extracts multi-scale feature maps of the vehicle video frames at different scales through the backbone network of the lightweight object detection model. The lightweight object detection model adopts an improved MobileNet architecture as its backbone network, and significantly reduces computational complexity through depthwise separable convolutions and channel shuffling techniques. The multi-scale feature maps include high-resolution fine-grained feature maps, medium-resolution semantic feature maps, and low-resolution global feature maps, which are used to capture vehicle detail information, mid-level semantic information, and overall structural information, respectively. The feature extraction process effectively integrates features at different scales through top-down paths and lateral connections, providing multi-level feature representations for subsequent feature fusion.

[0010] The feature fusion module generates an occlusion-aware suppression mask based on the response difference distribution between adjacent scale feature maps in a multi-scale feature map. This mask is then used to fuse features across the multi-scale feature maps, resulting in a fused localization feature map. By analyzing the response pattern differences between feature maps at different scales, this module adaptively identifies vehicle occlusion regions, generating accurate suppression masks that effectively reduce the negative impact of occlusion on detection accuracy. The occlusion-aware suppression mask dynamically adjusts feature weights at different spatial locations, highlighting the feature responses of clearly visible areas and suppressing noise features in occluded areas. Finally, a high-quality fused localization feature map is generated through weighted fusion.

[0011] The region detection module is used to generate candidate vehicle regions and candidate license plate regions based on the fused localization feature map, and to extract the corresponding local feature slices of the candidate license plate regions. This module employs a dual-branch detection head design, with separate optimizations for the overall vehicle and the local license plate, respectively. A non-maximum suppression algorithm is used to filter high-quality candidate regions. The vehicle region detection branch focuses on the overall vehicle outline and shape features, while the license plate region detection branch focuses on rectangular textures and character patterns. Local feature slices are accurately extracted from the fused localization feature map through Region of Interest (RoI) pooling operations, preserving key feature information of the license plate region and providing a feature foundation for subsequent association and binding.

[0012] The association and binding module is used to construct a geometric offset vector based on the spatial mapping relationship between candidate vehicle regions and candidate license plate regions on the fused localization feature map. It then calculates the matching affinity by combining local feature slices to complete the association and binding between candidate vehicle regions and candidate license plate regions based on the matching affinity, thereby determining the target vehicle region and its corresponding target license plate region. This module solves the problem of one-to-many or many-to-one association between vehicles and license plates in complex scenarios. Through spatial geometric constraints and feature similarity measurements, it establishes an accurate vehicle-license plate correspondence, ensuring the reliability and completeness of the recognition results.

[0013] The character recognition module is used to perform character recognition on the target license plate area to obtain the license plate text information. This module adopts a deep learning sequence recognition network, which can accurately recognize various types of license plate characters, including combinations of Chinese characters, letters and numbers. It supports license plate format standards in different regions and improves recognition accuracy under complex angles and lighting conditions through perspective transformation correction and character segmentation technology.

[0014] The database retrieval module is used to search the vehicle load database based on license plate text information and obtain standard load data corresponding to the license plate text information. This module uses an efficient database query interface to quickly match license plate numbers with vehicle registration information, obtaining standard load parameters including vehicle model, rated load, and vehicle type, providing authoritative reference data for load verification.

[0015] The verification and correction module extracts visual morphological attribute features of the target vehicle area, generates a visually inferred load range based on these features, and performs consistency verification and correction annotation on the standard load data based on this range. This module analyzes the vehicle's appearance using computer vision technology, infers the actual load status, cross-validates with standard load data, identifies potential data anomalies or illegal loading behaviors, and provides intelligent load monitoring capabilities.

[0016] The modules are connected via wired and / or wireless means to enable data transmission between them.

[0017] In a preferred embodiment of the present invention, the above-mentioned generation of an occlusion perception suppression mask based on the response difference distribution between adjacent scale feature maps in the multi-scale feature map, and the use of the occlusion perception suppression mask to perform feature fusion on the multi-scale feature map to obtain a fused localization feature map, includes: Step 2.1: Calculate the channel response difference between the first-scale feature map and the corresponding second-scale feature map in the multi-scale feature map after upsampling and alignment, and normalize the channel response difference to obtain the difference response map. Specifically, this includes: First, clarifying the composition of the multi-scale feature map. This feature map is generated by the backbone network of the lightweight object detection model through different levels of convolution and pooling operations when processing vehicle video frames. It contains multiple scale levels from high-resolution detail features to low-resolution semantic features. Each scale level corresponds to a different spatial resolution and receptive field size, capable of capturing multi-level information from local texture to global structure. Next, the feature fusion module selects two adjacent scale levels from the multi-scale feature map as processing objects. The higher-resolution feature map is defined as the first-scale feature map, and the adjacent feature map with lower resolution is defined as the second-scale feature map. For example, a feature map with a resolution of 512×512 is selected as the first-scale feature map, and a feature map with a resolution of 256×256 is selected as the second-scale feature map. Subsequently, to ensure effective numerical computation between the two feature maps at different scales, the feature fusion module performs bilinear interpolation upsampling on the first-scale feature map, adjusting its spatial resolution to be completely consistent with the second-scale feature map. This ensures that the two feature maps have the same height and width parameters in the spatial dimension, while maintaining their respective channel dimension feature representations unchanged. After spatial alignment, the feature fusion module calculates the response difference between the first-scale and second-scale feature maps channel by channel. By subtracting element-wise, it obtains the numerical difference between the two feature maps at each spatial location and in each channel direction, forming an initial channel response difference matrix. To eliminate the influence of differences in numerical ranges between different channels on subsequent processing, the feature fusion module performs normalization on the channel response differences. It uses a max-min normalization method to scale the difference data to a standard range between 0 and 1, ultimately obtaining a difference response map that effectively characterizes the distribution of response differences between the two scale feature maps.

[0018] Step 2.2 involves inputting the differential response map into a learnable mapping layer to generate an occlusion-aware suppression mask. The values ​​of each element in the occlusion-aware suppression mask represent the probability that the corresponding spatial location is occluded. Specifically, the feature fusion module inputs the normalized differential response map into a pre-designed learnable mapping layer for further processing. This mapping layer consists of multiple convolutional layers, batch normalization layers, and activation function layers, capable of learning a non-linear mapping relationship from the differential response map to the occlusion probability distribution. The network structure of the mapping layer is specially designed. The first convolutional layer uses a 3×3 convolutional kernel to extract spatial features from the differential response map, capturing response change patterns within the neighborhood. The second convolutional layer uses a 1×1 convolutional kernel for inter-channel feature fusion. Finally, a sigmoid activation function maps the feature values ​​to between 0 and 1, ensuring that the output effectively represents the probability distribution. During training, the mapping layer learns how to transform differences in feature responses into criteria for determining occlusion. For example, when the feature responses of a spatial location show significant differences across different scales, it usually indicates that the location may be subject to occlusion interference. The mapping layer sets the output value for such locations to a higher probability value. The occlusion-aware suppression mask generated after processing by the mapping layer has each element ranging from 0 to 1. The closer the value is to 1, the higher the probability that the corresponding spatial location is in an occluded state, and the features at that location need to be suppressed. The closer the value is to 0, the more reliable the features at that location are, and no suppression operation is required. In this way, the system can adaptively identify and mark regions that may be affected by occlusion.

[0019] Step 2.3 involves element-wise multiplying the occlusion-aware suppression mask with the first-scale feature map to obtain the suppressed first-scale feature map. This suppressed first-scale feature map is then concatenated and convolved with the second-scale feature map to reduce dimensionality, generating a fused localization feature map. Specifically, the feature fusion module performs element-wise multiplication of the generated occlusion-aware suppression mask with the first-scale feature map. The core principle of this operation is to use the probability values ​​in the mask to weight and adjust the feature response at each spatial location in the first-scale feature map. Specifically, when the value at a certain location in the mask is close to 0, it indicates that the feature at that location is reliable, and the multiplied feature value remains essentially unchanged. When the value at a certain location in the mask is close to 1, it indicates that the location may be affected by occlusion, and the multiplied feature value will be significantly suppressed, thereby reducing the negative impact of the occluded region on the overall feature expression. Finally, the first-scale feature map after occlusion-aware suppression processing is obtained. Next, the feature fusion module concatenates the suppressed first-scale feature map with the second-scale feature map, merging the two feature maps along the channel dimension to form a combined feature representation containing richer multi-scale information. This concatenation operation effectively integrates complementary information from different scale levels, enhancing the integrity and robustness of the feature representation. Since the concatenated feature map increases exponentially in the channel dimension, to control computational complexity and extract the most critical fusion features, the feature fusion module processes the concatenated features using a specially designed convolutional dimensionality reduction module. This module uses a 1×1 convolution operation to reduce the number of channels to the original scale, and simultaneously selects and strengthens the feature combinations most beneficial to vehicle localization and recognition through learned weight parameters, ultimately generating a fused localization feature map that integrates multi-scale information and possesses occlusion perception capabilities.

[0020] In this embodiment of the invention, the method employs the following techniques: calculating the channel response difference between adjacent scale feature maps after upsampling and alignment in a multi-scale feature map and normalizing it to obtain a difference response map; inputting the difference response map into a learnable mapping layer to generate an occlusion perception suppression mask to characterize the occlusion probability of each spatial location; then multiplying the occlusion perception suppression mask element-wise with the first scale feature map to obtain a suppressed feature map; and concatenating it with the second scale feature map and performing convolutional dimensionality reduction to generate a fused localization feature map. This overcomes the technical problems of existing multi-scale feature fusion methods lacking a perception and suppression mechanism for occlusion interference, leading to noise features in occluded areas interfering with the overall feature expression quality, and the inability of simple feature concatenation or addition fusion to effectively handle feature conflicts and inconsistencies between different scales. This achieves adaptive identification and suppression of the negative impact of occluded areas on feature fusion, ensuring that the fused localization feature map maintains high-quality feature expression in complex traffic scenarios, and improving the accuracy and robustness of subsequent vehicle and license plate detection.

[0021] In a preferred embodiment of the present invention, the above-mentioned calculation of the channel response difference between the first-scale feature map and the corresponding second-scale feature map in the multi-scale feature map after upsampling alignment, and the normalization of the channel response difference to obtain the difference response spectrum, includes: Step 2.1.1 involves performing bilinear interpolation upsampling on the first-scale feature map to obtain an upsampled feature map with the same spatial dimensions as the second-scale feature map. Specifically, the feature fusion module first determines the spatial resolution parameters of the first and second-scale feature maps. The first-scale feature map typically has a higher spatial resolution, such as 512×512 pixels, containing richer details and local texture features. The second-scale feature map has a lower spatial resolution, such as 256×256 pixels, but contains stronger semantic information and global context features. To ensure effective numerical computation between the two feature maps, the feature fusion module uses a bilinear interpolation algorithm to perform upsampling on the first-scale feature map. The bilinear interpolation algorithm calculates the value of newly inserted pixels by weighted averaging the values ​​of four adjacent pixels, effectively expanding the spatial resolution while maintaining the smoothness of the feature map. Specifically, the upsampling process adjusts the spatial resolution of the first-scale feature map from the original 512×512 to 256×256, which is exactly the same as the second-scale feature map. This ensures that the two feature maps have the same parameter configuration in the height and width dimensions, while keeping their feature representation in the channel dimension unchanged. The result is an upsampled feature map with the same spatial size as the second-scale feature map, laying the foundation for subsequent difference calculations.

[0022] Step 2.1.2 calculates the sum of the absolute differences between the upsampled feature map and the second-scale feature map in each channel direction to obtain the initial difference matrix. Specifically, after completing the upsampling alignment of the first-scale feature map, the feature fusion module performs a channel-by-channel difference calculation between the upsampled feature map and the second-scale feature map. Since the two feature maps typically have the same configuration in the channel dimension, such as 256 channels each, the feature fusion module iterates through each channel, subtracting the corresponding pixel values ​​of the upsampled feature map and the second-scale feature map at the same channel position to obtain the difference results for each spatial position in that channel direction. To avoid the positive and negative differences canceling each other out and affecting the judgment of the degree of difference, the feature fusion module takes the absolute value of the difference results for each channel to ensure that all difference information is effectively preserved and accumulated. Next, the feature fusion module sums the absolute differences in all channel directions. For each spatial location in the feature map, the absolute differences in all 256 channels at that location are accumulated to obtain a scalar value to characterize the overall degree of difference of that spatial location in all channel directions. Finally, an initial difference matrix with the same spatial resolution as the original feature map but with only one channel is formed. The value of each element in this matrix directly reflects the feature difference intensity of the corresponding spatial location.

[0023] Step 2.1.3 involves performing a maximum-minimum normalization operation on the initial difference matrix. The normalized initial difference matrix is ​​then used as the difference response map. Specifically, since the numerical range of the initial difference matrix may vary significantly, with differences across different spatial locations potentially spanning multiple orders of magnitude, the feature fusion module performs a maximum-minimum normalization operation to ensure the stability and consistency of subsequent processing. Specifically, the feature fusion module first calculates the maximum and minimum values ​​of all elements in the initial difference matrix, then uses a normalization formula to perform a linear transformation on each element, mapping the original numerical range to a standard interval between 0 and 1. After normalization, elements in the initial difference matrix with values ​​close to 0 indicate smaller feature differences at corresponding spatial locations, suggesting good consistency across different scales. Elements with values ​​close to 1 indicate larger feature differences at corresponding spatial locations, potentially indicating occlusion, illumination variations, or other interference factors. The feature fusion module will determine this normalized single-channel matrix as a differential response map. This map can intuitively show the consistency distribution of multi-scale features at various spatial locations in the vehicle video frame, providing an important basis for subsequent occlusion perception processing.

[0024] In this embodiment of the invention, by employing bilinear interpolation upsampling of the first-scale feature map to obtain an upsampled feature map with the same spatial size as the second-scale feature map, calculating the sum of the absolute differences between the upsampled feature map and the second-scale feature map in each channel direction to obtain an initial difference matrix, and then performing a maximum-minimum value normalization operation on the initial difference matrix to obtain a difference response map, the technical problems of existing feature difference calculation methods, such as the inability to perform effective numerical calculations due to spatial resolution mismatch, the easy loss of difference information due to positive and negative cancellation effects of simple difference calculations, and the lack of standardization processing leading to inconsistent numerical ranges affecting the stability of subsequent processing, are overcome. This achieves the technical effect of ensuring accurate difference calculations for feature maps of different scales, effectively preserving and accumulating the difference information of all channels through absolute value summation, and mapping the degree of difference to a standard range through normalization processing, generating a difference response map that accurately reflects the consistent distribution of multi-scale features, and providing a reliable data foundation for occlusion perception and feature fusion.

[0025] In a preferred embodiment of the present invention, the above-mentioned construction of a geometric offset vector based on the spatial mapping relationship between candidate vehicle regions and candidate license plate regions on the fused localization feature map, and the calculation of matching affinity in combination with local feature slices, includes: Step 4.1: Determine the coordinates of the first center point of the candidate vehicle region and the second center point of the candidate license plate region. Calculate the coordinate difference between the first and second center point coordinates, and define this difference as a geometric offset vector. Specifically, the association and binding module first extracts spatial location information from the candidate vehicle region and candidate license plate region output by the region detection module. Each candidate region is represented by a rectangular bounding box, containing parameters such as the coordinates of the top-left corner, bottom-right corner or center point, width, and height. For each candidate vehicle region, the association and binding module calculates the first center point coordinates based on the geometric parameters of its rectangular bounding box. Specifically, the calculation method is to add the x-coordinate of the top-left corner and the x-coordinate of the bottom-right corner of the bounding box and divide by 2 to obtain the center x-coordinate, and add the y-coordinate of the top-left corner and the y-coordinate of the bottom-right corner and divide by 2 to obtain the center y-coordinate, thereby determining the precise center position of the candidate vehicle region in the fused localization feature map coordinate system. Similarly, the association and binding module performs the same calculation operation for each candidate license plate region to obtain the second center point coordinates, ensuring that the center point coordinates of all regions are calculated and represented based on a unified coordinate system. Next, the association and binding module calculates the coordinate difference between the first and second center point coordinates for each pair of candidate vehicle and candidate license plate regions. This difference calculation includes the coordinate difference in the x-direction and y-direction, forming a two-dimensional vector. This two-dimensional vector intuitively reflects the spatial offset direction and distance of the candidate license plate region relative to the candidate vehicle region. The association and binding module then determines the calculated coordinate difference vector as a geometric offset vector. This vector contains basic spatial geometric relationship information between the two candidate regions, providing important geometric constraints for subsequent association and matching analysis.

[0026] Step 4.2 maps the local feature slices of the candidate license plate region to the feature space of the candidate vehicle region to obtain the projected feature response map. Specifically, the association binding module processes the local feature slices corresponding to the candidate license plate region extracted by the region detection module. These local feature slices are local feature representations extracted from the fused localization feature map at the candidate license plate region location, containing detailed feature information within that region. To achieve feature association analysis between different candidate regions, the association binding module needs to map the local feature slices of the candidate license plate region to the feature space of the candidate vehicle region for comparison. Specifically, the association binding module uses a feature mapping network to perform spatial transformation and feature encoding on the local feature slices of the candidate license plate region. This network consists of multiple convolutional and pooling layers and can learn the mapping relationship from the license plate feature space to the vehicle feature space. During the mapping process, the network converts the local feature slices of the candidate license plate region into feature vectors with the same dimension and semantic meaning as the feature representation of the candidate vehicle region, ensuring that the features of the two different types of regions can be effectively compared in a unified feature space. The parameters of the mapping network are optimized through an end-to-end training process, enabling it to learn the inherent correlation patterns between license plate regions and vehicle regions. For example, license plates are typically located at the front or rear of a vehicle, and the texture features of the license plate are correlated with the overall appearance of the vehicle. After mapping, the association and binding module obtains a projected feature response map that reflects the distribution of candidate license plate region features in the candidate vehicle region feature space. This response map provides a crucial feature representation foundation for subsequent feature similarity calculations and matching decisions.

[0027] Step 4.3: Calculate the spatial distance penalty factor based on the geometric offset vector and the feature similarity score based on the projected feature response map. Then, weightedly fuse the spatial distance penalty factor and the feature similarity score to generate a matching affinity. Specifically, the association binding module calculates the spatial distance penalty factor based on the calculated geometric offset vector. This penalty factor is used to quantify the spatial geometric rationality between the candidate vehicle region and the candidate license plate region. In the specific calculation process, the association binding module first calculates the magnitude of the geometric offset vector, i.e. ,in and They represent direction and The coordinate difference in direction and the magnitude of the distance reflect the straight-line distance between two candidate regions. Next, the association and binding module sets a reasonable distance threshold range, based on statistical analysis of the relative positions of vehicles and license plates in real-world scenarios. For example, license plates are typically located in the front or rear region within a certain range from the center of the vehicle. When the magnitude of the geometric offset vector falls within a reasonable range, the spatial distance penalty factor is close to 1, indicating that the spatial relationship between the two regions conforms to the geometric constraints of the vehicle-license plate. When the magnitude exceeds the reasonable range, the penalty factor decreases with increasing distance, penalizing unreasonable spatial relationships. Simultaneously, the association and binding module calculates a feature similarity score based on the projected feature response map. By calculating the cosine similarity or other similarity metrics between the feature representations of the candidate vehicle region and the projected feature representation of the candidate license plate region, a numerical score reflecting the degree of feature matching between the two regions is obtained. A higher similarity score indicates a stronger correlation between the two regions at the feature level. Finally, the association binding module will weight and fuse the spatial distance penalty factor and the feature similarity score, and use preset weight parameters to linearly combine the two components to generate a matching affinity that comprehensively considers spatial geometric relationship and feature matching degree. This affinity value can fully reflect the association strength between candidate vehicle area and candidate license plate area.

[0028] In this embodiment of the invention, by determining the center point coordinates of the candidate vehicle region and the candidate license plate region and calculating the coordinate difference as a geometric offset vector, mapping the local feature slices of the candidate license plate region to the feature space of the candidate vehicle region to obtain a projected feature response map, calculating the spatial distance penalty factor based on the geometric offset vector, calculating the feature similarity score based on the projected feature response map, and weightedly fusing the two to generate a matching affinity, this invention overcomes the technical problems of existing vehicle license plate association methods that rely solely on spatial distance or feature similarity for matching, are prone to association errors due to the limitations of a single constraint condition, and lack an effective multimodal information fusion mechanism that cannot comprehensively utilize geometric and feature information. This invention achieves the technical effect of effectively constraining the spatial positional relationship of vehicles and license plates through geometric offset vectors, realizing cross-regional feature association analysis through projected feature response maps, and comprehensively considering spatial geometric rationality and feature matching degree through a weighted fusion mechanism, thereby improving the accuracy and robustness of vehicle license plate association binding.

[0029] In a preferred embodiment of the present invention, the above-mentioned association binding between candidate vehicle regions and candidate license plate regions based on matching affinity, and the determination of target vehicle regions and corresponding target license plate regions, may include: Step 5.1 involves constructing a bipartite graph with candidate vehicle regions as the first type of nodes and candidate license plate regions as the second type of nodes. Matching affinity is used as the weight of the corresponding edges in the bipartite graph. Specifically, the association binding module constructs a bipartite graph data structure based on all candidate vehicle regions and candidate license plate regions output by the region detection module. This bipartite graph is a special graph structure where nodes are divided into two disjoint sets, and each edge connects nodes from only different sets. Specifically, the association binding module treats all candidate vehicle regions as the first type of node set, with each candidate vehicle region corresponding to a first-type node in the graph. Each node stores the region's location information, size parameters, and feature representation attributes. Similarly, all candidate license plate regions are treated as the second type of node set, with each candidate license plate region corresponding to a second-type node in the graph, also storing corresponding region attribute information. Next, the association binding module establishes connecting edges between each first-type node and each second-type node, forming a complete bipartite graph structure. Each edge represents a possible vehicle-license plate association hypothesis. For each connection edge, the association binding module uses the previously calculated matching affinity as the weight value of that edge. The weight value reflects the reasonableness of the association binding between the corresponding candidate vehicle region and the candidate license plate region. The higher the weight value, the more likely the association hypothesis is a true vehicle-license plate pairing. In this way, the originally complex many-to-many association binding problem is transformed into a bipartite graph matching problem with a clear mathematical representation, providing a clear problem modeling framework for subsequent optimization solutions.

[0030] Step 5.2: Perform maximum weighted bipartite graph matching on the bipartite graph based on weights, filtering out matching edges that meet the preset affinity threshold. Specifically, the association binding module uses the classic maximum weighted bipartite graph matching algorithm to solve the constructed bipartite graph. The goal of this algorithm is to find a matching set in the bipartite graph such that the sum of the weights of all edges in the matching set reaches the maximum value, while satisfying the constraint that each node can be matched by at most one edge. Commonly used solution algorithms include the Hungarian algorithm or the Kuhn-Munkres algorithm. These algorithms can find the optimal solution in polynomial time, ensuring the global optimality of the matching result. During the execution of maximum weighted matching, the algorithm comprehensively considers all possible vehicle-license plate combinations, and gradually constructs the optimal matching scheme through dynamic programming or augmented path search to ensure the maximization of the overall matching effect. After the matching algorithm is completed, a set of matching edges is obtained. Each matching edge connects a candidate vehicle region and a candidate license plate region, indicating that the algorithm considers this pairing to be the optimal association relationship under the current conditions. To further ensure matching quality, the association and binding module sets a preset affinity threshold to filter the matching results. Only matching edges with weight values ​​exceeding the threshold are retained as the final association and binding result, while matching edges with weight values ​​below the threshold are filtered out to prevent low-quality erroneous associations from affecting the overall recognition performance. The preset affinity threshold is usually determined based on statistical analysis of the training data, aiming to ensure high matching accuracy while avoiding overly strict thresholds that could lead to missed detections.

[0031] Step 5.3 involves identifying the first and second type nodes connected by the matching edges as the target vehicle region and target license plate region, respectively, thus completing the association binding. Specifically, after completing the maximum weight bipartite graph matching and filtering out high-quality matching edges that meet the preset affinity threshold, the association binding module performs final target determination processing on the filtering results. For each retained matching edge, the candidate vehicle region represented by the first type node connected by the edge is determined as the target vehicle region, indicating that the region is recognized by the system as a real vehicle target. Correspondingly, the candidate license plate region represented by the second type node connected by the matching edge is determined as the target license plate region corresponding to the target vehicle region, indicating that the license plate region belongs to the vehicle target. Through this determination process, the system achieves the function of accurately filtering out real targets from multiple candidate regions, while establishing a one-to-one correspondence between target vehicle regions and target license plate regions. The association binding module records the pairing information of the determined target vehicle regions and target license plate regions in the system data structure, including their respective position coordinates, size parameters, feature representations, and association relationship identifiers. This information will serve as input data for subsequent character recognition, database retrieval, and load verification modules. At this point, the association and binding module has successfully completed the association and binding task between the candidate vehicle area and the candidate license plate area, laying a reliable foundation for the subsequent processing stages of the entire vehicle precise positioning and recognition system.

[0032] In this embodiment of the invention, by constructing a bipartite graph with candidate vehicle regions as the first type of nodes and candidate license plate regions as the second type of nodes, and using matching affinity as edge weights, performing maximum weight matching on the bipartite graph based on the weights, and filtering matching edges that meet a preset affinity threshold, and determining the nodes connected by the matching edges as the target vehicle region and target license plate region to complete the association binding, the technical means overcome the technical problems of existing association binding methods that are prone to falling into suboptimal solutions due to greedy strategies or local optimization, lack of global matching constraints leading to erroneous one-to-many or many-to-one associations, and lack of a quality threshold filtering mechanism that easily retains low-quality matching results. Thus, it achieves the technical effect of ensuring the mathematical rigor of the association binding problem through bipartite graph modeling, ensuring the globally optimal association result through the maximum weight matching algorithm, and ensuring the high quality of the final matching through affinity threshold filtering, thereby realizing accurate and reliable one-to-one association binding between candidate vehicle regions and candidate license plate regions.

[0033] In a preferred embodiment of the present invention, the above-mentioned character recognition of the target license plate area to obtain license plate text information may include: Step 6.1 involves performing perspective transformation correction on the target license plate area to obtain a corrected license plate image. Specifically, the character recognition module first preprocesses the target license plate area determined by the association and binding module. Due to changes in the shooting angle of traffic monitoring equipment and the vehicle's driving posture on the road, the target license plate area may suffer from geometric distortions such as perspective distortion, tilting, or trapezoidal distortion. These distortions directly affect the accuracy of subsequent character recognition. To eliminate the influence of geometric distortion, the character recognition module uses a perspective transformation correction algorithm to perform geometric correction on the target license plate area. Specifically, the character recognition module first detects the four boundary corners of the target license plate area. Through computer vision algorithms such as edge detection, line detection, and corner detection, it automatically locates the coordinates of the four vertices of the rectangular license plate area. Then, based on the detected corner coordinates and the corresponding corner coordinates of a standard rectangular license plate, it calculates the perspective transformation matrix. This matrix describes the geometric transformation relationship from a distorted license plate to a standard license plate. Finally, it applies the calculated perspective transformation matrix to perform pixel remapping on the original target license plate area, correcting the distorted license plate image into a corrected license plate image with a standard rectangular shape and neatly arranged characters. During the correction process, image interpolation algorithms such as bilinear interpolation or bicubic interpolation are used to ensure the smoothness and clarity of the corrected image, avoiding the impact of image quality loss caused by the transformation process on subsequent character recognition results.

[0034] Step 6.2: Extract the local stroke direction features of each character in the corrected license plate image and construct a character feature sequence based on these features. Specifically, after perspective transformation correction, the character recognition module performs character-level feature extraction on the corrected license plate image. First, based on the standard layout rules of license plate images, the module segments the corrected image into multiple character regions. For example, according to the standard format of Chinese license plates, the image is segmented into regions corresponding to the province abbreviation, letter codes, and numeric codes. For each segmented character region, the module uses the Histogram of Oriented Gradients (HOG) feature extraction algorithm or a similar stroke direction feature extraction method to analyze the gradient and directional distribution of pixel grayscale values ​​within that region. Specifically, the algorithm divides the character region into multiple small grid cells, calculates the magnitude and direction of the pixel gradient within each cell, and statistically analyzes the histograms of gradient distributions in different directions, forming a feature vector that characterizes the local stroke direction features of the character. Local stroke direction features can effectively capture key structural information of characters, such as the direction, turns, and intersections of strokes. These features are crucial for distinguishing different characters. The character recognition module arranges and combines the local stroke direction features extracted from all character positions in the license plate image from left to right to construct a complete character feature sequence. This sequence contains detailed feature descriptions of each character in the license plate, providing rich input information for subsequent sequence recognition.

[0035] Step 6.3: Input the character feature sequence into the sequence recognition model, determine the probability distribution corresponding to each character based on the output of the sequence recognition model, and select the character combination with the highest probability based on the probability distribution as the license plate text information. Specifically, the character recognition module will input the constructed character feature sequence into the pre-trained sequence recognition model for character recognition processing. This sequence recognition model usually adopts deep learning architectures such as Recurrent Neural Network (RNN), Long Short-Term Memory network (LSTM) or Transformer, which can process variable-length sequence inputs and learn the temporal dependency relationship between elements in the sequence. During the training process, the sequence recognition model has learned the feature patterns of a large number of license plate character samples and the context association rules between characters, and has the ability to infer character categories from stroke direction features. When the character feature sequence is input into the sequence recognition model, the model will process each character feature in the sequence one by one, combine the local features and context information of the character, and output the probability distribution of all possible character categories corresponding to the character position. For example, for a certain character position, the model may output that the probability of the character at this position being "A" is 0.8, the probability of being "B" is 0.1, and the probabilities of being other characters are different values respectively. The character recognition module will collect the probability distribution information output by the sequence recognition model for each character position in the license plate, then make decisions based on these probability distributions, and select the character with the highest probability at each character position as the recognition result of that position. To further improve the recognition accuracy, the character recognition module will also consider the constraint rules of the license plate format, for example, the first character must be a provincial abbreviation, and some positions can only be letters or numbers, etc., and filter unreasonable character combinations through rule constraints. Finally, the character recognition module will combine the recognition results of each character position in order to form complete license plate text information, such as "Jing A12345", which accurately reflects the character content of the target license plate.

[0036] In the embodiment of the present invention, since the technical means of performing perspective transformation correction on the target license plate area to obtain a corrected license plate image in a standard rectangle, extracting the local stroke direction features of each character in the corrected license plate image and constructing a character feature sequence, inputting the character feature sequence into the sequence recognition model and selecting the character combination with the highest probability based on the output probability distribution as the license plate text information are adopted, the technical problems in the existing character recognition methods that the recognition accuracy decreases due to their sensitivity to geometric distortion, the effective description of character structure information is lacking merely based on pixel values or simple texture features, and recognition optimization cannot be performed by utilizing the association relationship between characters due to the lack of sequence context modeling are overcome. Therefore, the technical effects of eliminating the influence of geometric distortion through perspective transformation correction to ensure the standardization of character images, effectively capturing the key structure information of characters through local stroke direction features to enhance recognition robustness, and comprehensively utilizing character features and context information through the sequence recognition model to realize high-precision license plate character recognition are achieved.

[0037] In a preferred embodiment of the present invention, the above-mentioned extraction of visual morphological attribute features of the target vehicle region and generation of visual inference load interval based on the visual morphological attribute features may include: Step 7.1 involves cropping the overall vehicle feature region corresponding to the target vehicle region from the fused localization feature map, and performing a global average pooling operation on the overall vehicle feature region to obtain the vehicle morphology description vector. Specifically, the verification and correction module first performs a precise region cropping operation on the fused localization feature map based on the target vehicle region location information determined by the association and binding module. Specifically, the verification and correction module performs feature cropping at the corresponding position in the fused localization feature map according to the bounding box coordinate parameters of the target vehicle region, including the coordinates of the upper left and lower right corners, extracting the overall vehicle feature region that completely covers the target vehicle region. This overall vehicle feature region contains all feature information of the target vehicle in the fused localization feature map, including both local detail features of the vehicle, such as the texture and shape features of components like doors, windows, and wheels, and the overall contour features and spatial structure information of the vehicle. To convert the two-dimensional overall vehicle feature region into a one-dimensional compact feature representation, the verification and correction module performs a global average pooling operation on the cropped feature region. This operation calculates the average value of each channel in the feature region at all spatial locations, compressing the original spatially dimensional feature map into a one-dimensional vector containing only the channel dimension. The advantage of global average pooling is that it preserves the overall feature information of each channel while eliminating specific details of spatial location, making the feature representation somewhat invariant to the precise location of the vehicle in the image. After global average pooling, the verification and correction module obtains a fixed-dimensional vehicle morphology description vector. Each element of this vector corresponds to a channel of the fused localization feature map, and the value reflects the average activation intensity of the channel's features throughout the entire vehicle region, effectively summarizing the overall visual features of the target vehicle.

[0038] Step 7.2 involves extracting the proportional components representing the geometric proportions of the vehicle outline and the structural components representing the morphological form of the vehicle body from the vehicle morphology description vector. These proportional and structural components are then concatenated to form visual morphological attribute features. Specifically, the verification and correction module performs further feature decomposition and semantic parsing on the generated vehicle morphology description vector, decomposing the original high-dimensional feature vector into feature components with clear physical meaning. Specifically, the verification and correction module uses a pre-trained feature decomposition network or dimensionality reduction algorithms such as Principal Component Analysis (PCA) to project the vehicle morphology description vector into different semantic feature subspaces. The proportional component subspace is specifically used to represent the geometric proportions of the vehicle outline, including geometric metric parameters such as the vehicle's length-to-width ratio, height-to-width ratio, wheelbase-to-length ratio, and the relative dimensions of the front and rear. These proportional features effectively distinguish different types of vehicles, such as sedans, SUVs, trucks, and buses, because different vehicle models have significant differences in geometric proportions. The structural component subspace is specifically used to characterize the morphological features of the vehicle body structure, including the shape and outline of the body, the curvature of the roof, the number and distribution of doors, and the degree of separation between the cargo box and the cab. These structural features reflect the vehicle's functional purpose and load capacity. For example, trucks typically have a distinct cargo box structure, while buses have a taller body and multiple doors. The verification and correction module extracts the corresponding proportional and structural components from the vehicle morphological description vector. Each component is a subset or linear combination of the original feature vector, maintaining the correspondence with the original features. Finally, the verification and correction module concatenates the extracted proportional and structural components, merging them into a unified visual morphological attribute feature in the feature dimension. This feature contains both the vehicle's geometric proportions and structural morphological information, providing a comprehensive visual feature foundation for subsequent load inference.

[0039] Step 7.3 involves inputting the visual morphological attribute features into a pre-trained interval regression network, which outputs the upper and lower limits of the visually inferred load interval. Specifically, the verification and correction module inputs the constructed visual morphological attribute features into a specially designed interval regression network for load interval prediction. This network is a deep learning model trained under supervision using visual features from a large number of vehicle samples and actual load data. The interval regression network architecture typically includes multiple fully connected layers, batch normalization layers, and activation function layers, enabling it to learn the complex mapping relationship from vehicle visual morphological attribute features to load intervals. The network training process utilizes abundant vehicle sample data, covering vehicle images and corresponding real load values ​​under different vehicle models and load conditions, allowing the network to learn the statistical regularity between vehicle appearance and load capacity. The output layer of the interval regression network is designed with two neurons, outputting the upper and lower limits of the visually inferred load interval, respectively. The upper limit represents the maximum load that the vehicle can potentially bear based on its current visual morphological features, and the lower limit represents the inferred minimum load. These two values ​​together constitute a load interval range. During training, the network not only learns the standard load ranges for different vehicle models but also the visual change patterns of vehicles under different load conditions. For example, heavy-duty vehicles may exhibit specific visual characteristics in terms of vehicle height, tire deformation, and suspension compression. When the visual morphological attribute features are input into the interval regression network, the network outputs the upper and lower limits of the visually inferred load range corresponding to the vehicle based on the learned mapping relationship. This range reflects the load capacity range inferred from the vehicle's visual appearance, providing an important reference for subsequent load data verification.

[0040] In this embodiment of the invention, the technical means of cropping the overall vehicle feature region corresponding to the target vehicle region in the fused localization feature map and performing global average pooling to obtain the vehicle morphology description vector, extracting the proportional component representing the geometric proportion of the contour and the structural component representing the morphology of the carriage from the vehicle morphology description vector and splicing them into visual morphological attribute features, and inputting the visual morphological attribute features into a pre-trained interval regression network to output the upper and lower limits of the visual inference load interval, overcomes the technical problems of existing load inference methods that lack effective means for evaluating load capacity based on the visual appearance of the vehicle, that simple image features cannot effectively represent the geometric proportion and structural morphology information of the vehicle, and that the lack of a dedicated load interval prediction model leads to inaccurate and unreliable inference results. Thus, it achieves the technical effect of effectively extracting the overall visual features of the vehicle through global average pooling of the overall vehicle feature region, obtaining morphological attribute features with clear physical meaning through the decomposition and splicing of proportional and structural components, and realizing the accurate mapping from visual features to load intervals through the interval regression network, providing a reliable visual inference basis for load data verification.

[0041] In a preferred embodiment of the present invention, the above-mentioned consistency verification and correction annotation of standard load data based on visual inference of load range may include: Step 8.1 determines whether the standard load data falls between the upper and lower limits of the visually inferred load range. Specifically, the verification and correction module performs a consistency comparison analysis between the standard load data obtained by the database retrieval module and the visually inferred load range generated by the verification and correction module. First, the verification and correction module identifies the value of the standard load data, which is the official vehicle load parameter retrieved from the vehicle load database based on license plate text information, typically expressed in tons or kilograms. Simultaneously, the verification and correction module obtains the upper and lower limits of the visually inferred load range. These two values ​​are the load capacity range inferred from the vehicle's visual morphological attributes through an interval regression network, also expressed in corresponding weight units. Next, the verification and correction module performs a range inclusion relationship judgment operation, checking whether the value of the standard load data is within the closed interval formed by the upper and lower limits of the visually inferred load range. The specific judgment condition is: lower limit ≤ standard load data ≤ upper limit. If the standard load data satisfies this inequality, it indicates that the standard load data in the database is consistent with the load capacity range inferred from visual appearance, and the load data has high reliability. If the standard load data does not satisfy this condition, i.e., the standard load data is less than the lower limit or greater than the upper limit, it indicates that there is a data inconsistency, requiring further analysis and processing. The verification and correction module will record the judgment results to provide a basis for subsequent verification and correction operations.

[0042] Step 8.2: If the standard load data does not fall within the visually inferred load range, the deviation ratio between the standard load data and the center value of the visually inferred load range is calculated. Specifically, when the standard load data does not fall within the visually inferred load range, the verification and correction module further analyzes the degree of deviation between the two to determine an appropriate processing strategy. First, the verification and correction module calculates the center value of the visually inferred load range using the formula: Center value = (Upper limit + Lower limit) / 2. This center value represents the best estimate of the vehicle load capacity inferred based on visual features. Next, the verification and correction module calculates the absolute deviation between the standard load data and the center value of the visually inferred load range using the formula: Absolute deviation = |Standard load data - Center value|. This value reflects the magnitude of the difference between the two load estimates. To better assess the relative degree of deviation, the verification and correction module calculates the deviation ratio using the formula: Deviation ratio = Absolute deviation / Center value. This ratio is a dimensionless relative indicator that eliminates the influence of the absolute magnitude of the load value, more objectively reflecting the severity of the deviation. For example, for heavy vehicles with large load capacities, an absolute deviation of a few tons might be acceptable, but for light vehicles with smaller load capacities, the same absolute deviation could indicate serious inconsistency. By calculating the deviation ratio, the verification and correction module can uniformly assess the consistency of load data for different types of vehicles, providing a quantitative evaluation index for subsequent anomaly detection and correction labeling.

[0043] Step 8.3: When the deviation ratio exceeds a preset deviation threshold, an anomaly label is generated and attached to the standard load data. Simultaneously, the center value of the visually inferred load interval is marked as the suggested reference load. The suggested reference load and standard load data are output together. Specifically, the verification and correction module compares the calculated deviation ratio with a preset deviation threshold. This threshold is typically determined based on statistical analysis of a large amount of historical data, aiming to identify genuine data anomalies while avoiding excessive false alarms due to overly strict thresholds. Common deviation threshold settings range from 0.2 to 0.5, indicating that a deviation ratio exceeding 20% ​​to 50% is considered a significant data inconsistency. When the deviation ratio exceeds the preset deviation threshold, the verification and correction module determines that the current standard load data is abnormal, possibly due to database errors, vehicle modifications, data entry errors, etc. After confirming the anomaly, the verification and correction module generates a corresponding anomaly label. This label contains detailed information such as the anomaly type (e.g., "Load data deviation anomaly"), deviation ratio value, and detection timestamp, and attaches this anomaly label to the original standard load data, forming a load data record with an anomaly mark. Simultaneously, the verification and correction module uses the center value of the visually inferred load range as the load reference value inferred based on the vehicle's visual appearance, labeling it as the suggested reference load. This value represents the load capacity estimate obtained based on visual feature analysis and can serve as a supplementary reference or alternative to the standard load data. Finally, the verification and correction module outputs both the standard load data with anomaly tags and the suggested reference load, providing complete load information for the user or subsequent processing modules. This preserves the original database load data for traceability while providing the load inference results based on visual analysis as a reference, ensuring the integrity and reliability of the load information.

[0044] In this embodiment of the invention, by employing a technical approach that determines whether the standard load data falls between the upper and lower limits of the visually inferred load interval, and if not, calculates the deviation ratio between the standard load data and the center value of the visually inferred load interval, and generates an anomaly label and attaches it to the standard load data when the deviation ratio exceeds a preset deviation threshold, and simultaneously marks the center value of the visually inferred load interval as a suggested reference load and outputs it together with the standard load data, the invention overcomes the technical problems of existing load data management systems lacking an effective data quality verification mechanism, being unable to identify and mark potentially erroneous load data, and lacking means of supplementing and correcting load data based on visual analysis. This achieves the technical effect of effectively identifying load data anomalies through consistency verification, objectively assessing the severity of data inconsistencies through deviation ratio calculation, and providing reliable technical support for load data quality management and error correction through anomaly labeling and suggested reference loads.

[0045] In a preferred embodiment of the present invention, after generating the anomaly label and attaching the anomaly label to the standard payload data, the process may further include: Step 9.1 involves storing the standard load data carrying the anomaly tag and the corresponding image frame of the target vehicle area into the anomaly review queue. Specifically, after generating and attaching the anomaly tag, the verification and correction module collects and stores the relevant anomaly data in a dedicated anomaly review queue for subsequent batch manual review and data correction. Specifically, the verification and correction module uses the standard load data carrying the anomaly tag as the core content of the anomaly record. This data includes the original license plate text information, the standard load value retrieved from the database, the attached anomaly tag information (such as anomaly type, deviation ratio, detection time, etc.), and the system-inferred suggested reference load, among other complete information. Simultaneously, to facilitate intuitive judgment and verification by manual reviewers, the verification and correction module also stores the corresponding image frame of the target vehicle area into the anomaly review queue. This image frame is an image area containing complete vehicle information, cropped from the original vehicle video frame, clearly showing the vehicle's appearance, model characteristics, load status, and other visual information. The anomaly review queue uses a first-in, first-out (FIFO) data structure to ensure that anomaly records are arranged in chronological order of discovery, facilitating sequential processing by reviewers. Each anomaly record is assigned a unique identifier and detailed metadata information, including detection time, monitoring device source, system version, and processing status, providing a complete information foundation for subsequent data traceability and quality analysis.

[0046] Step 9.2: When the number of records in the anomaly review queue reaches a preset threshold, the records in the anomaly review queue are sent in batches to the verification terminal for the verification terminal to update and verify the standard load data in the vehicle load database. Specifically, the verification correction module continuously monitors the number of records accumulated in the anomaly review queue. When the number of anomaly records in the queue reaches a preset batch processing threshold, the system automatically triggers the batch processing mechanism. The preset threshold needs to be set by comprehensively considering the balance between processing efficiency and timeliness. If the threshold is too small, frequent batch processing operations will increase system overhead; if the threshold is too large, the processing cycle of anomaly data will be prolonged. It is usually set between 100 and 1000 records based on the system's processing capacity and business needs. When the number of records in the queue reaches the preset threshold, the verification correction module will batch package all records in the anomaly review queue to generate a standardized data transmission format, ensuring data integrity and format consistency. During the packaging process, data integrity verification will be performed on each record to confirm the integrity of information such as image frames, load data, and anomaly tags, avoiding the impact of missing data on the quality of subsequent manual review. Next, the verification and correction module will send the packaged anomaly records in batches to the designated verification terminal via a secure data transmission channel. The verification terminal is typically a workstation operated by professional data administrators or a centralized data auditing platform. After receiving the anomaly records, the verification terminal will provide auditors with a graphical auditing interface, displaying vehicle images, load data comparisons, anomaly analysis results, and other information. This allows auditors to update, correct, or confirm the standard load data in the vehicle load database based on the actual situation, thereby achieving continuous optimization and quality improvement of the load database.

[0047] In this embodiment of the invention, by employing a technical means of storing standard load data carrying anomaly tags and corresponding target vehicle area image frames into an anomaly review queue, and sending the records in batches to the verification terminal when the number of records in the anomaly review queue reaches a preset threshold for updating and verifying the standard load data in the vehicle load database, the technical problems of existing data management systems—such as lack of a systematic anomaly data collection and review mechanism, low anomaly data processing efficiency and easy omissions, and lack of effective database quality feedback and update mechanism—are overcome. This achieves the technical effect of systematically collecting and managing anomaly data through the anomaly review queue, improving the processing efficiency of anomaly data through a batch processing mechanism, and continuously optimizing and improving the quality of the load database through the manual review mechanism of the verification terminal, thus forming a complete closed loop of data quality management.

[0048] In a preferred embodiment of the present invention, after retrieving the vehicle load database based on the license plate text information and obtaining the standard load data corresponding to the license plate text information, the method may further include: Step 10.1 records the total time taken from the input of the vehicle video frame to the acquisition of standard payload data. Specifically, the database retrieval module deploys a precise time monitoring mechanism throughout the entire vehicle recognition processing flow to track and record the system's processing performance in real time. Specifically, the time monitoring mechanism sets a start timestamp at the moment the vehicle video frame is input to the data acquisition module, marking the beginning of the entire recognition processing flow. Subsequently, the monitoring mechanism tracks the flow of data between various processing modules, including multi-scale feature map generation in the feature extraction module, occlusion perception processing in the feature fusion module, candidate region generation in the region detection module, vehicle license plate association in the association and binding module, license plate text recognition in the character recognition module, and payload data acquisition in the database retrieval module. When the database retrieval module successfully acquires the standard payload data corresponding to the license plate text information, the monitoring mechanism sets an end timestamp, marking the completion point of the entire recognition processing flow. By calculating the difference between the end timestamp and the start timestamp, the monitoring mechanism can accurately obtain the total time elapsed from the input of vehicle video frames to the acquisition of standard load data. This time elapsed data is recorded in milliseconds, reflecting the actual processing performance of the system under the current hardware and software configuration. The monitoring mechanism also records the total time elapsed data along with the segmented time elapsed for each processing module, forming a detailed performance analysis report to provide data support for subsequent performance optimization.

[0049] Step 10.2: When the total processing time exceeds a preset latency threshold, the number of channels in some convolutional layers of the lightweight object detection model's backbone network is dynamically pruned. The pruned model is then used to perform feature extraction in subsequently acquired vehicle video frames. Specifically, the performance monitoring module compares the recorded total processing time with a preset latency threshold in real time. This threshold is determined based on the system's performance requirements and the real-time needs of the application scenario. For example, for real-time traffic monitoring applications, the latency threshold might be set to 500 milliseconds to ensure the system can respond to and process traffic flow promptly. When the total processing time exceeds the preset latency threshold, the system automatically initiates a performance optimization mechanism to dynamically adjust computationally intensive steps in the processing flow. Since the lightweight object detection model in the feature extraction module is usually the most computationally complex part of the entire processing flow, the performance optimization mechanism prioritizes optimizing this model. Specifically, the optimization mechanism performs dynamic pruning on the backbone network of the lightweight object detection model. The pruning algorithm analyzes the importance and contribution of each convolutional layer in the backbone network, identifying redundant channels that have a smaller impact on the final recognition performance. The pruning process selectively removes redundant channels from some convolutional layers based on pre-trained importance evaluation metrics, such as channel activation intensity, gradient magnitude, and weight values, thereby reducing the number of model parameters and computational complexity. The pruning operation adheres to the principle of maintaining recognition accuracy, ensuring that processing speed is improved without significantly impacting vehicle recognition accuracy. After pruning, the system replaces the original model with the optimized model. In subsequent processing of acquired vehicle video frames, the pruned lightweight model is used to perform feature extraction, effectively reducing processing time and improving the system's real-time response capability.

[0050] In this embodiment of the invention, by recording the total time from the input vehicle video frame to the acquisition of standard payload data, dynamically pruning the number of channels in some convolutional layers of the backbone network of the lightweight target detection model when the total time exceeds a preset delay threshold, and using the pruned model to perform feature extraction operations in subsequent vehicle video frame processing, the technical problems of existing vehicle recognition systems—such as lack of real-time performance monitoring and adaptive optimization mechanisms, fixed model configurations that cannot be dynamically adjusted according to actual performance requirements, and excessive system processing latency affecting real-time application effects—are overcome. This achieves the technical effect of timely detection of processing latency issues through real-time performance monitoring, effectively reducing model complexity and computation time while maintaining recognition accuracy through dynamic pruning technology, realizing adaptive optimization of system processing performance, and ensuring that the vehicle recognition system maintains good real-time response capabilities under different load conditions.

[0051] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0052] It should be noted that all formulas in this manual are calculated by removing dimensions and taking their numerical values. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0053] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A vehicle precise positioning and identification system integrating multi-scale features, characterized in that, include: The data acquisition module is used to acquire vehicle video frames collected by traffic monitoring equipment; The feature extraction module is used to input the vehicle video frame into a pre-built lightweight target detection model, and extract multi-scale feature maps of the vehicle video frame at different scales through the backbone network of the lightweight target detection model. The feature fusion module is used to generate an occlusion perception suppression mask based on the response difference distribution between adjacent scale feature maps in the multi-scale feature map, and to perform feature fusion on the multi-scale feature map using the occlusion perception suppression mask to obtain a fused localization feature map. The region detection module is used to generate candidate vehicle regions and candidate license plate regions based on the fused localization feature map, and to extract local feature slices corresponding to the candidate license plate regions. The association binding module is used to construct a geometric offset vector based on the spatial mapping relationship between the candidate vehicle region and the candidate license plate region on the fused localization feature map, and calculate the matching affinity in combination with the local feature slices, so as to complete the association binding between the candidate vehicle region and the candidate license plate region based on the matching affinity, and determine the target vehicle region and the corresponding target license plate region. The character recognition module is used to perform character recognition on the target license plate area to obtain license plate text information; The database retrieval module is used to retrieve the vehicle load database based on the license plate text information and obtain the standard load data corresponding to the license plate text information. The verification and correction module is used to extract the visual morphological attribute features of the target vehicle area, generate a visual inference load range based on the visual morphological attribute features, and perform consistency verification and correction annotation on the standard load data based on the visual inference load range.

2. The system according to claim 1, characterized in that, Based on the response difference distribution between adjacent scale feature maps in the multi-scale feature map, an occlusion-aware suppression mask is generated, and the multi-scale feature map is fused using the occlusion-aware suppression mask to obtain a fused localization feature map, including: Calculate the channel response difference between the first-scale feature map and the corresponding second-scale feature map in the multi-scale feature map after upsampling and alignment, and normalize the channel response difference to obtain the difference response map. The differential response map is input into a learnable mapping layer to generate the occlusion-aware suppression mask; The occlusion-aware suppression mask is multiplied element-wise with the first-scale feature map to obtain the suppressed first-scale feature map. The suppressed first-scale feature map is then concatenated and convolved with the second-scale feature map to reduce dimensionality, generating the fused localization feature map.

3. The system according to claim 2, characterized in that, The calculation of the channel response difference between the first-scale feature map and the corresponding second-scale feature map in the multi-scale feature map after upsampling alignment, and the normalization of the channel response difference to obtain the difference response map, includes: Bilinear interpolation upsampling is performed on the first scale feature map to obtain an upsampled feature map with the same spatial size as the second scale feature map; The sum of the absolute differences between the upsampled feature map and the second-scale feature map in each channel direction is calculated to obtain the initial difference matrix; The initial difference matrix is ​​normalized by a minimum-maximum value, and the normalized initial difference matrix is ​​determined as the difference response map.

4. The system according to claim 1, characterized in that, The step of constructing a geometric offset vector based on the spatial mapping relationship between the candidate vehicle region and the candidate license plate region on the fused localization feature map, and calculating the matching affinity in combination with the local feature slices, includes: Determine the coordinates of the first center point of the candidate vehicle area and the coordinates of the second center point of the candidate license plate area, calculate the coordinate difference between the first center point coordinates and the second center point coordinates, and determine the coordinate difference as the geometric offset vector; The local feature slices of the candidate license plate region are mapped to the feature space of the candidate vehicle region to obtain the projected feature response map; The spatial distance penalty factor is calculated based on the geometric offset vector, and the feature similarity score is calculated based on the projected feature response map. The spatial distance penalty factor and the feature similarity score are then weighted and fused to generate the matching affinity.

5. The system according to claim 4, characterized in that, The process of associating and binding the candidate vehicle region with the candidate license plate region based on the matching affinity, and determining the target vehicle region and the corresponding target license plate region, includes: Construct a bipartite graph with the candidate vehicle area as the first type of node and the candidate license plate area as the second type of node, and use the matching affinity as the weight of the corresponding edge in the bipartite graph; Based on the weights, a maximum weighted bipartite graph matching operation is performed on the bipartite graph to filter out matching edges that meet a preset affinity threshold. The first type of node and the second type of node connected by the matching edge are respectively determined as the target vehicle area and the target license plate area, thereby completing the association binding.

6. The system according to claim 1, characterized in that, The step of performing character recognition on the target license plate area to obtain license plate text information includes: The target license plate area is subjected to perspective transformation correction to obtain the corrected license plate image; Extract the local stroke direction features of each character in the corrected license plate image, and construct a character feature sequence based on the local stroke direction features; The character feature sequence is input into the sequence recognition model. The probability distribution corresponding to each character is determined based on the output of the sequence recognition model. The character combination with the highest probability is selected as the license plate text information based on the probability distribution.

7. The system according to claim 1, characterized in that, The step of extracting the visual morphological attribute features of the target vehicle region and generating a visual inference load range based on the visual morphological attribute features includes: In the fused localization feature map, the overall vehicle feature region corresponding to the target vehicle region is cropped out, and a global average pooling operation is performed on the overall vehicle feature region to obtain the vehicle morphology description vector. Extract the proportional component used to characterize the geometric proportion of the vehicle outline and the structural component used to characterize the shape of the carriage from the vehicle shape description vector, and concatenate the proportional component and the structural component to form the visual shape attribute feature. The visual morphological attribute features are input into a pre-trained interval regression network, which outputs the upper and lower limits of the visual inference load interval.

8. The system according to claim 7, characterized in that, The consistency verification and correction annotation of the standard load data based on the visually inferred load range includes: Determine whether the standard load data falls between the upper and lower limits of the visually inferred load range; If the standard load data does not fall within the visual inferred load range, then calculate the deviation ratio between the standard load data and the center value of the visual inferred load range; When the deviation ratio exceeds a preset deviation threshold, an anomaly label is generated and attached to the standard load data. At the same time, the center value of the visually inferred load interval is marked as a suggested reference load, and the suggested reference load and the standard load data are output together.

9. The system according to claim 8, characterized in that, After generating anomaly labels and appending the anomaly labels to the standard payload data, the method further includes: Store the standard load data carrying the anomaly label and the corresponding image frame of the target vehicle area into the anomaly review queue. When the number of records in the abnormal audit queue reaches a preset threshold, the records in the abnormal audit queue are sent in batches to the verification terminal so that the verification terminal can update and verify the standard load data in the vehicle load database.

10. The system according to claim 1, characterized in that, After retrieving the vehicle load database based on the license plate text information and obtaining the standard load data corresponding to the license plate text information, the method further includes: Record the total time taken from inputting the vehicle video frame to acquiring the standard load data; When the total time exceeds a preset delay threshold, the number of channels in some convolutional layers of the backbone network of the lightweight target detection model is dynamically pruned, and the pruned model is used to perform feature extraction operations in the subsequently acquired vehicle video frames.

Images