Infrared trigger-based license plate recognition timing system

Abstract

This invention relates to the field of intelligent traffic timing technology and discloses an infrared-triggered license plate recognition timing system. The system comprises a five-layer architecture: an infrared sensing layer acquires vehicle information using a biomimetic visual sensing mechanism, generates a vehicle presence heatmap via a dynamic focusing network, and divides the sensing area; a license plate recognition mapping layer deploys infrared camera arrays by region, emits modulated infrared signals, and generates a license plate identification image using an optical character recognition algorithm; a cross-modal data fusion layer establishes a spatiotemporal mapping, synchronizes the clock using a precision clock synchronization protocol, and generates a reliability score via an attention fusion network; a timing control decision layer generates timing commands based on a multi-objective decision algorithm and sends them to the timing terminal in real time via a lightweight protocol; and an execution feedback layer monitors the terminal status, calculates deviations to generate a sensing strategy effectiveness index, and dynamically optimizes the strategy until the index is optimal. This system can improve the accuracy and reliability of recognition and timing in complex environments and is adaptable to diverse traffic scenarios.

Description

Technical Field

[0001] This invention relates to the field of intelligent traffic timing technology, specifically to a license plate recognition timing system based on infrared triggering. Background Technology

[0002] In current traffic management and parking operation scenarios, license plate recognition timing systems are widely used in parking lots, highway toll stations and other places. Existing systems mostly rely on ultrasonic sensors, ordinary video surveillance or single infrared probes to realize vehicle sensing, and complete license plate recognition by combining visible light camera equipment with image processing algorithms. Data fusion adopts simple weighted average or logical judgment methods, and timing control is based on fixed rules and relies on traditional communication protocols.

[0003] Traditional vehicle sensing technologies are susceptible to environmental temperature, humidity, and lighting conditions. They have limited sensing distance, lack the ability to dynamically capture vehicle outlines, and cannot adaptively divide sensing areas, leading to issues such as false triggering, missed detections, or overlapping detection of multiple vehicles. License plate recognition suffers from overexposure and blurring in complex environments such as strong light, backlighting, and inclement weather, resulting in low character recognition accuracy. Infrared technology is only used for supplementary lighting and lacks deep synergy. Furthermore, the infrared sensing and license plate recognition modules operate independently, lacking a unified clock synchronization mechanism, resulting in inaccurate spatiotemporal mapping, insufficient exploitation of complementary data fusion, and inadequate real-time timing command issuance, all of which affect timing accuracy. Summary of the Invention

[0004] The purpose of this invention is to provide an infrared-triggered license plate recognition timing system to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides an infrared-triggered license plate recognition timing system, the system comprising:

[0006] The infrared sensing layer acquires vehicle sensing information based on a biomimetic visual sensing mechanism, applies a dynamic focusing network to the vehicle sensing information, generates a vehicle presence heat map, and divides the vehicle presence heat map into sensing areas.

[0007] The license plate recognition mapping layer deploys an infrared camera array according to the divided sensing area, emits modulated infrared signals to cover the vehicle surface, and analyzes the license plate image signal through an optical character recognition algorithm to generate a license plate identification map;

[0008] The cross-modal data fusion layer establishes a spatiotemporal mapping between infrared sensing and license plate recognition, synchronizes the clock using a precision clock synchronization protocol, and fuses the vehicle presence heatmap and license plate identification map through an attention fusion network to generate a credibility score.

[0009] The timing control decision layer converts the credibility score into executable timing instructions based on a multi-objective decision algorithm, and sends the executable timing instructions to the timing terminal in real time through a lightweight communication protocol.

[0010] The execution feedback layer monitors the changes in timing status after the timing terminal executes the timing command in real time, calculates the deviation between the timing status change and the preset timing status change threshold, generates a sensing strategy effectiveness index, and dynamically optimizes the license plate recognition sensing strategy until the sensing strategy effectiveness index reaches its optimal value.

[0011] Preferably, the method for acquiring vehicle sensing information based on a biomimetic visual sensing mechanism includes:

[0012] The license plate recognition timing system deploys three types of sensors: inductive loop sensors, infrared camera sensors, and radar ranging sensors, which collect multimodal sensing information of vehicles passing by.

[0013] Simulates the non-uniform perception mechanism of biological vision and dynamically adjusts the detector sensitivity and sampling frequency;

[0014] Based on the vehicle traffic trajectory model, the front positioning area, license plate positioning area and rear positioning area are designated as high-sensitivity focusing areas;

[0015] The remaining area of ​​the vehicle is designated as the low-sensitivity peripheral area.

[0016] Dynamically define the sensitivity function to divide the vehicle surface into a high-sensitivity focusing area and a low-sensitivity peripheral area;

[0017] The spatial analysis network analyzes vehicle features in multimodal sensing information in real time and dynamically adjusts the focusing area based on these features.

[0018] Preferably, the method for generating a vehicle presence heatmap by applying a dynamic focusing network to the vehicle sensing information includes:

[0019] The multimodal sensing information is subjected to dynamic range normalization, and the normalized multimodal sensing information is stacked into a multidimensional tensor according to the feature dimension.

[0020] A dynamic focusing network structure is constructed, which includes feature extraction path, feature enhancement path and feature fusion connection;

[0021] Multimodal sensing information is input into a dynamic focusing network structure. In the feature extraction path, a convolutional neural network is used to extract features from the input multimodal sensing information to generate feature maps at different scales.

[0022] In the feature enhancement path, the spatial resolution of the high-level feature map is gradually restored through resolution reconstruction;

[0023] A feature fusion connection is established between the feature extraction path and the feature enhancement path to fuse feature maps of the same scale.

[0024] Preferably, the method for dividing the vehicle's heat map into sensing areas includes:

[0025] The vehicle presence status score is calculated for each region in the vehicle presence heatmap. The vehicle presence status score is obtained by weighted fusion of vehicle temperature distribution data, vehicle surface reflection data and vehicle spatial distance data included in the multimodal sensing information.

[0026] A first threshold and a second threshold for vehicle presence status scoring are preset, and the vehicle presence status score is compared with the preset first threshold and the preset second threshold respectively.

[0027] Different colors are used to divide the vehicle's heat map into sensing areas, with different colors representing different sensing areas.

[0028] Preferably, the method for generating the license plate identification image includes:

[0029] Infrared camera sensors of varying densities are deployed on the road surface according to the defined sensing areas.

[0030] The infrared camera sensor is an array-type infrared camera sensor, employing differentiated shooting strategies for different sensing areas;

[0031] An initial global image is captured by emitting modulated infrared signals onto the vehicle surface using an infrared camera sensor.

[0032] The infrared shooting angle is dynamically adjusted based on the sensing area of ​​the vehicle's thermal map.

[0033] Preferably, the method for establishing the spatiotemporal mapping between infrared sensing and license plate recognition includes:

[0034] Establish a world coordinate system with any point on the road surface as the origin;

[0035] The infrared camera sensor is calibrated using a calibration board to obtain its intrinsic and extrinsic parameters.

[0036] The pixel coordinates of the vehicle's heatmap are converted to sensor coordinates using the intrinsic parameters of the infrared camera sensor, and then the sensor coordinates are converted to world coordinates using the extrinsic parameters of the infrared camera sensor.

[0037] Using the origin of the world coordinate system as a reference point, the camera sensor is calibrated to obtain its extrinsic parameters.

[0038] The license plate identification image is converted into a world coordinate system using the extrinsic parameters of the camera sensor.

[0039] Preferably, the method for fusing vehicle presence heatmaps and license plate identification maps through an attention fusion network includes:

[0040] A spatiotemporal map is constructed based on the vehicle existence heat map and license plate identification map in the world coordinate system;

[0041] An attention fusion network is constructed using a multi-layer attention structure based on a multi-head attention mechanism to fuse vehicle existence heatmaps and license plate identification maps in the world coordinate system.

[0042] Attention fusion networks consist of an input layer, a feature transformation layer, an attention interaction layer, and an output layer.

[0043] The spatiotemporal graph is used as input to the input layer of the attention fusion network, and a credibility score is generated through the output layer.

[0044] Preferably, the method for constructing the spatiotemporal graph includes:

[0045] Each sensing area of ​​a vehicle in the heat map in the world coordinate system is taken as an infrared node, and the feature vector of each sensing area is extracted as the feature of the infrared node.

[0046] Each recognition region of the license plate identification map in the world coordinate system is taken as a camera node, and the feature vector of each recognition region is extracted as the feature of the camera node.

[0047] Collect all infrared nodes and camera nodes to obtain the node set;

[0048] Calculate the spatial distance between any two infrared nodes in the world coordinate system;

[0049] A preset infrared distance threshold is established, and connection edges between infrared nodes are created based on the relationship between spatial distance and the infrared distance threshold.

[0050] Calculate the spatial distance between any two camera nodes in the world coordinate system;

[0051] A preset camera distance threshold is established, and connection edges between camera nodes are created based on the relationship between spatial distance and the camera distance threshold.

[0052] Find the nearest camera node for each infrared node and establish the connection edge between the infrared node and the camera node;

[0053] Construct a spatiotemporal graph based on the obtained set of nodes and the set of connecting edges.

[0054] Preferably, the method for converting credibility scores into executable timing instructions based on a multi-objective decision-making algorithm includes:

[0055] The vehicle passage timeline is divided into time units, and each unit records the current timestamp and credibility score.

[0056] It also lists all adjustable timing control parameters, including the adjustment range, computational cost, and correlation of historical impact on timing for each parameter;

[0057] Three decision-making objectives are established, including a primary time accuracy objective, a secondary time accuracy objective, and an efficiency objective.

[0058] Two types of constraints are set: rigid constraints and flexible constraints.

[0059] Using a multi-objective decision-making algorithm, multiple sets of timing control parameter optimization schemes are randomly generated. The completion status of each scheme is evaluated for the three decision objectives, and the completion status scores of the three decision objectives are obtained. The completion status scores of the three decision objectives are combined to obtain an overall score. The scheme with the highest overall score is selected from the multiple schemes as the final optimization scheme.

[0060] Preferably, the method for generating the sensing strategy effectiveness index includes:

[0061] Collect timing command parameters executed by the timing terminal, synchronously monitor changes in timing status, calculate the deviation between the changes in timing status and the preset threshold for changes in timing status, and generate a standardized sensing strategy effectiveness index.

[0062] Preset a first threshold and a second threshold for the effectiveness index of the standardized sensing strategy;

[0063] The effectiveness of the sensing strategy is determined by the relationship between the standardized sensing strategy effectiveness index and the preset threshold. The license plate recognition sensing strategy is dynamically optimized and warning information is generated.

[0064] Compared with the prior art, the beneficial effects of the present invention are:

[0065] At the vehicle sensing level, the system employs an infrared sensing layer based on a biomimetic visual sensing mechanism. This layer simulates the ability of biological vision to capture dynamic targets and processes vehicle sensing information through a dynamic focusing network to generate a vehicle presence heatmap. This heatmap not only visually reflects the vehicle's position and outline but also adaptively divides the sensing area based on the vehicle's real-time status (such as driving or parked), effectively solving the problems of limited sensing range, false triggering, or missed triggering associated with traditional single sensors. Compared to traditional infrared sensing technology, this layer can more accurately lock onto vehicle areas, providing a clear target range for the subsequent license plate recognition module, reducing interference from irrelevant areas in the recognition process, and improving the targeting and efficiency of license plate recognition.

[0066] At the license plate recognition level, the license plate recognition mapping layer deploys an infrared camera array according to the divided sensing areas. It covers the vehicle surface by emitting modulated infrared signals, rather than relying solely on auxiliary lighting. The modulated infrared signals can accurately locate the license plate area on the vehicle surface. Utilizing the characteristic that infrared signals are insensitive to ambient light, even in harsh environments such as nighttime, backlighting, and heavy rain, the difference between the license plate characters and the background can be clearly highlighted, significantly improving the quality of the license plate image signal acquisition. Combined with optical character recognition algorithms, license plate information can be analyzed more accurately to generate a license plate identification image, effectively solving the problem of low accuracy of traditional visible light recognition under complex lighting conditions, ensuring the stability and reliability of license plate identification.

[0067] The design of the cross-modal data fusion layer further enhances the scientific rigor and accuracy of the system's data processing. This layer establishes a spatiotemporal mapping between infrared sensing and license plate recognition, employing a precise clock synchronization protocol to achieve clock synchronization between the two, completely eliminating the time discrepancy between the two modalities in traditional systems and laying a precise spatiotemporal foundation for subsequent data fusion. Simultaneously, the application of an attention fusion network automatically focuses on key information in both datasets (such as the core vehicle area in the infrared heatmap and character details in the license plate recognition image), fully leveraging the complementarity of the data to generate a scientific reliability score. Compared to traditional simple weighting or logical judgment, this fusion method can more comprehensively assess the reliability of the recognition results, providing an accurate basis for the generation of subsequent timing commands and avoiding timing deviations caused by errors in a single data point.

[0068] The timing control decision layer, based on a multi-objective decision-making algorithm, can generate executable timing commands according to credibility scores and the needs of actual application scenarios (such as temporary parking and long-term parking). Compared with traditional fixed-rule decision-making methods, this algorithm has stronger adaptability and can flexibly respond to different vehicle passage and parking scenarios, ensuring the rationality and relevance of timing commands. Simultaneously, the adoption of a lightweight communication protocol significantly reduces data transmission redundancy, improves the real-time performance of timing command issuance, ensures that the timing terminal can respond quickly to commands, avoids timing errors caused by communication delays, and further improves the system's timing efficiency and accuracy.

[0069] The real-time monitoring and dynamic optimization functions of the feedback layer ensure the long-term stable operation of the system. This layer can track the state changes of the timing terminal after executing timing commands in real time, calculate the deviation between the actual state and the preset timing state threshold, and generate a sensing strategy effectiveness index based on the deviation. According to this index, the system can automatically adjust key strategies such as the sensing area division of the infrared sensing layer and the infrared signal parameters of the license plate recognition mapping layer, realizing dynamic optimization of the sensing strategy. This closed-loop optimization mechanism can promptly correct deviations in the system operation process, continuously improve the accuracy of license plate recognition and the precision of timing, ensure that the system maintains high performance during long-term operation, avoid the performance degradation problem caused by the lack of optimization mechanisms in traditional systems, extend the effective service life of the system, reduce later maintenance costs, and also improve the adaptability of the system in different application scenarios, expanding the application scope of the system. Attached Figure Description

[0070] Figure 1 This is a timing diagram of the infrared-triggered license plate recognition timing system described in this invention;

[0071] Figure 2 A flowchart for generating a heatmap of vehicle existence;

[0072] Figure 3 A flowchart for dividing the sensing area. Detailed Implementation

[0073] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0074] Please see Figure 1 The present invention provides an infrared-triggered license plate recognition timing system, the system comprising: an infrared sensing layer, a license plate recognition mapping layer, a cross-modal data fusion layer, a timing control decision layer, and an execution feedback layer.

[0075] The infrared sensing layer acquires vehicle sensing information based on a biomimetic visual sensing mechanism, applies a dynamic focusing network to generate a vehicle presence heat map based on the vehicle presence heat map, and divides the vehicle presence heat map into sensing areas.

[0076] The license plate recognition mapping layer deploys an infrared camera array according to the divided sensing area, emits modulated infrared signals to cover the vehicle surface, and generates a license plate identity recognition map by parsing the license plate image signal through an optical character recognition algorithm.

[0077] A cross-modal data fusion layer establishes a spatiotemporal mapping between infrared sensing and license plate recognition, uses a precision clock synchronization protocol to synchronize the clock, and uses an attention fusion network to fuse vehicle presence heatmaps and license plate identification maps to generate a credibility score.

[0078] The timing control decision layer converts the credibility score into executable timing instructions based on a multi-objective decision algorithm, and sends the executable timing instructions to the timing terminal in real time through a lightweight communication protocol.

[0079] The execution feedback layer monitors the changes in timing status after the timing terminal executes the timing command in real time, calculates the deviation between the timing status change and the preset timing status change threshold to generate a sensing strategy effectiveness index, and dynamically optimizes the license plate recognition sensing strategy until the sensing strategy effectiveness index reaches the optimal level.

[0080] See Figure 2 In the actual deployment of a license plate recognition timing system, the selection and layout of the inductive detectors constitute the physical basis for the system's perception. Inductive loop sensors are precisely buried beneath the lane surface; their working principle is to detect electromagnetic field disturbances caused by the passage of metal parts of vehicles, thereby generating a raw electrical signal characterizing the vehicle's presence. Infrared camera sensors are installed at a certain angle above the lane side; their optical receivers continuously receive infrared radiation energy reflected from the vehicle surface and convert the light intensity signal into a voltage signal output. The radar ranging sensor uses frequency-modulated continuous wave mode, calculating the real-time distance to the vehicle surface by determining the frequency difference between the emitted and reflected waves. These three detectors asynchronously collect electromagnetic, optical, and frequency-modal multimodal sensing information about the passing vehicles.

[0081] The non-uniform perception mechanism simulating biological vision is a dynamic adaptive process. The system kernel maintains a sensitivity adjustment strategy library. When the radar ranging sensor detects an increased vehicle approach rate, the strategy library dynamically increases the sampling frequency of the inductive loop to capture more subtle magnetic field changes. Simultaneously, the integration time of the infrared camera sensor is shortened to adapt to light intensity fluctuations caused by rapid vehicle movement. The vehicle trajectory model is built by analyzing a large amount of historical data. This model defines the common height range of license plates, the typical horizontal position of the front bumper, and the geometric characteristics of the vehicle's departure angle as the basis for calculating the high-sensitivity focusing area. The boundary of the focusing area is not fixed but elastically expands and contracts based on real-time vehicle speed and vehicle model prediction data.

[0082] The definition of the dynamic sensitivity function relies on the fusion calculation of multi-source data. This function uses radar ranging data as the vertical coordinate reference, the trigger position of the inductive loop as the horizontal coordinate reference, and combines the thermal distribution characteristics fed back by the infrared camera sensor to jointly define the three-dimensional spatial range of the high-sensitivity focusing area. The remaining parts of the vehicle surface are assigned lower sampling weights to save computational resources. The spatial analysis network uses a lightweight convolutional structure to analyze multimodal sensing information in real time. The network first performs frequency domain transformation on the electromagnetic signal of the inductive loop to extract disturbance features, normalizes the infrared light intensity signal to eliminate ambient light interference, and performs moving average filtering on the radar distance signal to smooth jitter. Then, the three types of feature vectors are concatenated and input into the fully connected layer. The output layer generates dynamic adjustment parameters for the focusing area range to achieve optimized tracking of vehicle features. Dynamic range normalization is implemented to address the differences in physical dimensions among different sensors. The voltage signal output by the inductive loop is linearly mapped to the [0,1] interval. The light intensity readings of the infrared camera sensor are scaled proportionally based on historical maximum and minimum values. The radar distance value is compressed to a fixed range using a sigmoid function. All normalized data are aligned by timestamp and stacked along the channel dimension to form a three-dimensional tensor structure. The feature extraction path of the dynamic focusing network adopts a multi-scale convolutional kernel design. The first-level convolutional layer uses a large kernel receptive field to capture the overall contour features of the vehicle. The second level uses dilated convolution to expand the feature range without increasing the number of parameters. The third level uses grouped convolution to separate and extract features from multimodal data. Each convolutional level is followed by a batch normalization layer and a nonlinear activation function.

[0083] The feature enhancement path employs a progressive upsampling structure. High-level feature maps are first upscaled using nearest-neighbor interpolation, then skip-connected with feature maps of the corresponding scale from the feature extraction path. Before skip-connection, a 1x1 convolution is used to adjust the channel count. The fused feature map is then further refined through transposed convolution to enhance spatial details. The feature fusion connection uses an attention-weighted mechanism, applying a spatial attention module to the output of each layer of the feature extraction path. This module obtains channel statistics through global average pooling, then generates a spatial weight mask through two fully connected layers. Only after weighting is the feature map element-wise added to the same-scale features from the feature enhancement path. The convolutional neural network's feature extraction process uses a residual connection design. Each convolutional block contains two 3x3 convolutional layers, and short-circuit connections directly pass the original input across these two layers. This structure alleviates the gradient vanishing problem in deep networks, allowing the network to reach the necessary depth. The resolution reconstruction process employs a sub-pixel convolution algorithm, which rearranges low-resolution feature maps along the channel dimension to generate high-resolution output. Compared to bilinear interpolation, this algorithm better preserves high-frequency detail information. Before fusing the reconstructed feature maps with shallow features, the feature importance needs to be recalibrated using a channel attention module. The final vehicle presence heatmap is generated using multi-scale feature fusion. The network output layer contains three branches corresponding to heatmap predictions at different resolutions: the low-resolution branch perceives the global vehicle presence probability, the medium-resolution branch locates key component regions, and the high-resolution branch refines edge details. The outputs of the three branches are weighted and summed to generate the final heatmap. The weighting coefficients are dynamically adjusted based on real-time computational load to balance accuracy and efficiency.

[0084] See Figure 3The vehicle presence status score for each region in the vehicle thermal map is calculated through multi-source data fusion. The system acquires temperature distribution data of the vehicle surface from infrared camera sensors, reflecting temperature characteristic changes in different parts of the vehicle due to engine heat radiation and differences in surface materials. Vehicle surface reflectance data is derived from the echo intensity of modulated infrared signals. Different paint materials and metal parts exhibit quantifiable differences in reflectivity for specific wavelengths of infrared light, and these optical characteristics are converted into a numerical reflectance coefficient matrix. The spatial distance data provided by the radar ranging sensor includes not only the straight-line distance between the vehicle and the sensor but also a distance gradient map formed by the microscopic unevenness features of the vehicle surface, calculated through multi-antenna phase interferometry. The weighted fusion process employs an adaptive weight allocation algorithm. The weight coefficients of temperature distribution data are dynamically adjusted based on the statistical difference between ambient temperature and normal vehicle operating temperature. In low-temperature environments, the characteristics of engine heat radiation are more significant, thus receiving a higher weight. The weight of surface reflectance data is negatively correlated with ambient light intensity. Under strong light conditions, the background noise of the infrared signal increases, thus reducing the contribution ratio of reflectance data. The weight of distance data remains relatively stable because it is less affected by environmental interference. The weighted data from the three categories are linearly superimposed to generate a preliminary vehicle presence score, which is then compressed to a standardized range of 0-1 using the sigmoid function.

[0085] The first threshold for vehicle presence status scoring is set to 0.7, corresponding to a certain confidence level of vehicle presence. The second threshold is set to 0.3 to distinguish potential noise interference or small objects. The system compares the score of each region with these two thresholds in real time. Regions with scores higher than the first threshold are identified as areas with strong vehicle presence, regions with scores between the two thresholds are marked as areas awaiting confirmation, and regions below the second threshold are considered background regions. The color coding scheme uses the RGB color space. Areas with strong vehicle presence are rendered with highly saturated red, areas awaiting confirmation are represented by a gradient of yellow to indicate uncertainty, and background regions are displayed in their original grayscale. The sensing area division not only considers the score value but also introduces spatial continuity constraints. Adjacent high-scoring regions are merged into unified sensing blocks using a connected component analysis algorithm. The geometric center of each block is calculated as a reference point for subsequent camera sensor aiming. The block boundaries are smoothed using morphological closing operations to eliminate jagged edges caused by score fluctuations. Simultaneously, an edge detection algorithm is used to identify the contour features of the blocks for matching and verification with the vehicle geometric model. The deployment density of infrared camera sensors is configured hierarchically according to the importance of the sensing area. In areas with strong vehicle presence, four high-definition infrared camera sensors are deployed per square meter to form a dense observation array. In areas awaiting confirmation, two standard-resolution sensors are deployed per square meter, while in background areas, only one basic sensor is deployed per square meter. The array-type infrared camera sensor adopts a modular design. Each module contains three infrared lenses with different focal lengths, covering far, medium, and near observation levels respectively. The module integrates an infrared LED array to provide active illumination.

[0086] The differentiated shooting strategy is reflected in the combination of exposure parameters and shooting modes. For areas with strong vehicle presence, a high-speed continuous shooting mode combined with short exposure times is used to capture details of rapidly moving license plates. For areas awaiting confirmation, an interval shooting mode is used to save storage space. Background areas are only captured in single shots when the score changes. The transmission of modulated infrared signals uses a time-division multiplexing mechanism. Each sensor module is allocated a unique time slice to send a specifically coded infrared pulse sequence. This code includes the sensor ID and timestamp information for subsequent data association. The initial global shot uses a wide-angle lens to cover the entire lane width, with shooting parameters set to automatic exposure mode and medium resolution to obtain overall vehicle outline information. The global image serves as a spatial reference frame for subsequent close-up shots. The actuator for dynamically adjusting the infrared shooting angle uses a two-degree-of-freedom gimbal design, with a horizontal rotation angle range of ±45 degrees and a pitch angle adjustment range of -30 degrees to +15 degrees. The gimbal control algorithm calculates the target azimuth angle in real time based on the coordinates of the sensing area in the vehicle presence heatmap.

[0087] The optical character recognition algorithm employs a multi-stage processing flow. First, the captured infrared images undergo preprocessing, including grayscale conversion, contrast enhancement, and noise suppression. Then, edge detection and morphological operations are used to locate candidate license plate regions. These candidate regions are corrected for perspective distortion using affine transformation. In the character segmentation stage, a vertical projection method combined with connected component analysis is used to separate individual characters. Finally, a deep convolutional neural network is used to recognize the segmented characters, outputting the probability distribution of each character and the confidence score of the overall license plate number. The generation of the license plate identification image integrates multiple frames of recognition results. The system recognizes multiple images of the same license plate captured consecutively, then selects the most frequently recognized result as the final output through a voting mechanism. Simultaneously, the average and variance of the recognition confidence scores are recorded as input parameters for subsequent confidence scoring. A spatial mapping relationship is established between the recognition results and the sensing areas of the vehicle's heatmap. Each recognized license plate number is associated with the coordinates of a sensing block in the heatmap. This mapping provides a spatiotemporal alignment basis for subsequent cross-modal data fusion.

[0088] The world coordinate system is established using a specific physical point on the road surface as the spatial reference origin. This origin is typically chosen as the actual geographical location of the intersection of the lane centerline and the stop line, and its latitude and longitude coordinates are measured by a high-precision GPS receiver and fixed in the system configuration. The coordinate system adopts the right-hand Cartesian coordinate system rule, with the positive X-axis pointing in the direction of vehicle movement, the positive Y-axis pointing to the right side of the lane, and the positive Z-axis perpendicular to the ground and upward. All spatial coordinates use this coordinate system as a unified reference framework. The calibration plate uses a black and white checkerboard pattern. The physical dimensions of the grid are measured by precision instruments and their actual millimeter-level dimensions are recorded. During calibration, the calibration plate is placed in different positions and orientations within the field of view of the infrared camera sensor. The intrinsic parameter calibration of the infrared camera sensor is calculated using multiple sets of checkerboard images. The intrinsic parameter matrix includes the focal length parameter, optical center coordinates, and radial and tangential distortion coefficients. The extrinsic parameter calibration is calculated based on the relative positional relationship between the sensor and the world coordinate system, including a rotation matrix and a translation vector. The rotation matrix is ​​defined by three Euler angles, and the translation vector represents the three-dimensional coordinates of the sensor's optical center in the world coordinate system. The transformation from pixel coordinates to sensor coordinates uses a perspective projection model, and the transformation from sensor coordinates to world coordinates is achieved through an extrinsic parameter matrix. The entire process follows the camera calibration principle in computer vision.

[0089] The extrinsic parameter calibration of the camera sensor uses the same world coordinate system origin and employs a larger calibration plate to cover a wider field of view. During calibration, multiple sets of images from different perspectives are acquired by moving the calibration plate. The extrinsic parameters of the camera sensor include its rotation matrix and translation vector relative to the world origin. These parameters are used to transform the pixel coordinates in the license plate recognition image to a unified world coordinate system. Image distortion correction needs to be considered during coordinate transformation. First, the intrinsic parameters of the camera sensor are used to perform distortion correction on the original image, and then the extrinsic parameters are applied for coordinate transformation. The construction of the spatiotemporal map discretizes the vehicle presence heatmap in the world coordinate system into grid nodes. Each grid node corresponds to a fixed-size physical region (e.g., 10cm × 10cm) in the world coordinate system. The node feature vector contains the mean, variance, and time series features of the vehicle presence status score within that region. The license plate recognition image is also discretized into grid nodes. The node feature vector contains the license plate recognition confidence score, character clarity score, and the spatial coordinates of the license plate region in the world coordinate system. The node set contains all infrared and camera nodes, each with timestamp and spatial coordinate information. Connections are established based on spatial proximity; connections between infrared nodes are calculated using Euclidean distance in the world coordinate system. An edge is established when the distance between two infrared nodes is less than a set threshold, and the edge weight is inversely proportional to the distance value. Connections between camera nodes are also based on spatial distance, but the threshold is set more leniently to account for potentially large displacements of license plates. The connection between infrared and camera nodes employs a nearest neighbor matching algorithm, finding the nearest spatially distant camera node for each infrared node to establish a cross-modal connection. This connection allows information to flow between two different types of sensor data.

[0090] The attention fusion network employs an encoder-decoder structure. The input layer linearly transforms the feature vectors of all nodes in the spatiotemporal graph and adds positional encoding, which is calculated based on the 3D coordinates of the nodes in the world coordinate system. The feature transformation layer performs a nonlinear transformation on the features of each node using a multilayer perceptron, enhancing the expressive power of the features and unifying the feature scale of different modalities. The attention interaction layer uses a multi-head self-attention mechanism, where each attention head calculates the association weights between nodes. The weight calculation is based on a composite function of node feature similarity and spatial distance.

[0091]

[0092] Where: w ij h represents the attention weight between node i and node j. i and h j These are the feature vectors of nodes i and j, respectively, W Q and W K It is a learnable query and key transformation matrix, where d is the dimension of the feature vector, and p iand p j These are the spatial coordinates of the two nodes in the world coordinate system. β and γ are learnable parameters that adjust the degree of influence of spatial distance. The softmax function ensures that the sum of all weights is 1.

[0093] The attention interaction layer contains multiple attention heads, each focusing on different aspects of the relationships between nodes. The outputs of all heads are concatenated and linearly transformed before being passed to the next layer. The network also includes a cross-modal attention mechanism specifically for handling the interaction between infrared nodes and camera nodes, allowing for deep fusion of vehicle presence information and license plate recognition information. The output layer employs a graph convolutional network structure, updating node features through multiple rounds of message passing. Each node ultimately outputs a confidence score, representing the degree of confidence in the presence of a vehicle and the reliability of the license plate recognition result in that area. The generation of the confidence score comprehensively considers the information of all nodes in the spatiotemporal graph. The score values ​​are continuously distributed between 0 and 1, with a high score indicating that the area simultaneously has a high probability of vehicle presence and a high confidence in license plate recognition. Spatial continuity constraints are maintained during the score calculation process, preventing drastic jumps in scores between adjacent nodes. This smoothness ensures the stability of subsequent decisions. The final output confidence score map has the same spatial resolution as the original vehicle presence heatmap, but the value of each pixel incorporates the influence of license plate recognition information.

[0094] The construction of the spatiotemporal map begins with the discretization of the vehicle presence heatmap in the world coordinate system. The system divides the physical space into 10cm × 10cm grid cells, generating an infrared node at the center point of each cell and recording its three-dimensional coordinates. Taking a parking lot entrance lane as an example, assuming the origin of the world coordinate system is set at the starting point of the lane centerline, the X-axis extends along the lane direction, the Y-axis is perpendicular to the lane direction, and the Z-axis is vertically upward. When a car enters, its engine compartment area shows a high brightness value in the heatmap, and the grid cells corresponding to the world coordinate interval [2.3m, 3.1m] and the Y interval [-0.5m, 0.5m] are marked as high-activity areas.

[0095] Each infrared node's feature vector contains four dimensions: the mean vehicle presence score (taken as the average of all pixels within the grid), the score variance (reflecting the stability of the area), the maximum temperature gradient (reflecting the rate of change in heat distribution), and the reflectivity feature value (characterizing surface material). For example, the mean vehicle presence score for the grid containing the engine compartment can reach 0.85, while the mean score for the top area of ​​the vehicle compartment is only 0.42. The license plate recognition map is also discretized into a grid, with each recognition area generating a camera node. Its feature vector contains the license plate recognition confidence score, character clarity score, license plate color feature value, and spatial coordinate data. The construction of the node set requires the integration of all sensing data. See Table 1, which shows the node data collected at a certain moment.

[0096] Table 1: Spatiotemporal graph node data

[0097]

[0098] The establishment of connection edges follows the principle of spatial proximity. Connections between infrared nodes are calculated based on Euclidean distance, with an infrared distance threshold set at 0.5 meters. When the spatial distance between two infrared nodes is less than this threshold, a connection edge is established, and the edge weight is inversely proportional to the distance. For example, the spatial distance between nodes IR001 and IR002 is 0.13 meters, less than the threshold of 0.5 meters, so a connection edge is established and assigned a weight of 0.87. The distance between nodes IR001 and IR003 reaches 0.82 meters, exceeding the threshold, so no direct connection is established. Connections between camera nodes use a more lenient threshold setting, with the camera distance threshold set at 1.0 meter to account for potential license plate displacement. When the spatial distance between two camera nodes is less than this value, a connection edge is established, and the weight calculation is also negatively correlated with distance. For example, the distance between nodes CAM001 and CAM002 is 0.61 meters, within the threshold range, so a connection edge is established with a weight of 0.72. Cross-modal connections are implemented using a nearest neighbor matching algorithm, finding the nearest spatially located camera node for each infrared node to establish a connection. During the matching process, a KD-tree spatial index structure is used to accelerate nearest neighbor search, ensuring computational efficiency even with a large number of nodes. Taking node IR001 as an example, its distance to camera node CAM001 is 0.04 meters, and its distance to CAM002 is 0.68 meters. Therefore, CAM001 is chosen to establish a cross-modal connection edge. This connection allows bidirectional flow of vehicle presence information and license plate recognition information. A high presence score for an infrared node can enhance the recognition confidence of adjacent camera nodes, and conversely, a high confidence recognition of a camera node can also improve the presence score of adjacent infrared nodes.

[0099] The weight calculation for connections employs a composite index, considering both spatial distance and feature similarity. For connections between infrared nodes, the weight calculation formula incorporates a vehicle presence score similarity term; the closer the scores of two nodes are, the higher the connection weight. For connections between camera nodes, the weight incorporates the recognition confidence correlation factor. The weight calculation for cross-modal connections considers both the matching degree of presence score and recognition confidence. The final spatiotemporal graph is constructed as a graph structure G=(V,E), where V is the set of all nodes, including infrared and camera nodes, and E is the set of all connections, including connections between nodes of the same type and cross-modal connections. Each node carries its feature vector and timestamp information, and each connection edge records its weight value and connection type. This graph structure serves as the input to the attention fusion network, achieving deep fusion of multimodal information through the message passing mechanism of the graph neural network. In actual operation, the spatiotemporal graph maintains a dynamic update mechanism, recalculating node features and connections every 100 milliseconds. Node features are updated based on the latest sensor data, and connections are dynamically adjusted according to changes in node positions, ensuring that the graph structure always reflects the current vehicle state. This dynamism enables the system to adapt to changes in spatial relationships caused by vehicle movement, ensuring the real-time nature and accuracy of the fusion results. The spatiotemporal graph construction process also includes an anomaly connection detection mechanism. When the feature values ​​of a node differ significantly from those of its neighboring nodes, the system marks the connection as pending verification and initiates a review process. For example, if an infrared node displays a high presence score but surrounding camera nodes all display low recognition confidence, this may indicate a false detection or a special vehicle type. The system will trigger additional sensor detection to confirm the actual situation. This mechanism improves the system's robustness to anomalies and prevents the propagation of erroneous connections from affecting the final decision.

[0100] The vehicle passage timeline is divided using a fixed time-slice mechanism, with each time unit set to 100 milliseconds and recorded with a timestamp accurate to the microsecond. Simultaneously, the confidence score calculated through fusion within that time period is stored. Taking an underground parking lot entrance system as an example, when a vehicle enters the recognition area, the system starts recording from time t=0. The confidence score is 0.76 in the first time unit, rises to 0.89 in the second time unit, and reaches 0.93 in the third time unit. These data, along with the timestamps, constitute a time-series database. Adjustable timing control parameters include twelve parameters such as infrared illumination intensity, character recognition confidence threshold, image acquisition frame rate, sensor sampling period, data fusion weight coefficient, and number of communication retries. Each parameter has a clearly defined adjustment range and boundary conditions.

[0101] The infrared illumination intensity can be adjusted from 10% to 100% of the rated power. The calculated cost is that each 10% increase in power requires an additional 3W of power. Historical data shows that this parameter has a 0.78 impact coefficient on nighttime recognition success rate. The character recognition confidence threshold can be set between 0.5 and 0.9. Increasing the threshold reduces the false recognition rate but may increase the risk of missed recognition. The computational cost is reflected in the 15ms increase in processing time required for each 0.1 increase in the threshold. The image acquisition frame rate can be adjusted from 5 to 30fps. Frame rate changes directly affect data throughput and processing latency. Historical records show that the correlation between frame rate and recognition accuracy exhibits a curve pattern of first increasing and then decreasing. The decision-making objectives include three dimensions: First, the time accuracy objective requires that the error between the trigger time of the timing command and the actual vehicle passage time not exceed ±50ms, ensuring timing accuracy. Second, the time accuracy objective requires that the latency from recognition to command issuance be controlled within 200ms, ensuring system response timeliness. Third, the efficiency objective requires that system power consumption be maintained below 70% of the rated power to avoid equipment overheating and extend service life. Rigid constraints include inviolable limitations such as the maximum operating current required by electrical safety regulations, the maximum data retention period required by data protection regulations, and the maximum adjustment range determined by the equipment's mechanical structure. Flexible constraints include soft limitations such as the ideal operating temperature range, recommended operating parameter range, and suggested maintenance cycle, which are allowed to be temporarily exceeded but will affect performance.

[0102] The multi-objective decision-making algorithm employs an improved NSGA-II algorithm. First, it generates 200 sets of random parameter combinations, each containing specific settings for twelve parameters. When evaluating the schemes, the first time accuracy objective is assessed by calculating the deviation between the theoretical recognition time and the actual time in a simulated vehicle passage scenario. The second time accuracy objective is evaluated by calculating the total delay of the data processing pipeline. The efficiency objective is derived by accumulating the power consumption values ​​corresponding to each parameter. The scores for the completion of the three objectives are standardized using Min-Max, uniformly converted to a 0-1 score range. The time accuracy objective score is inversely proportional to the error value, and the efficiency objective score is inversely proportional to the power consumption value. The overall scheme score is calculated using a weighted summation method, with a weight of 0.4 for the first time accuracy objective, 0.3 for the second time accuracy objective, and 0.3 for the efficiency objective. The weighted sum is the overall scheme score. The scheme with the highest overall score was selected from 200 schemes for application. Assuming that the optimal scheme is set with parameters such as infrared illumination intensity of 65%, recognition confidence threshold of 0.7, acquisition frame rate of 20fps, and sampling period of 50ms, the scheme is expected to achieve an operating effect of time error of ±35ms, processing delay of 180ms, and power consumption of 63% of rated power.

[0103] During execution, the system collects actual command parameters executed by the timing terminal, including command sending time, execution response time, duration, and power consumption. It synchronously monitors changes in timing status, including display update delay, voice prompt response, and gate lifting speed, calculating the deviation between actual status changes and preset values. The standardized sensing strategy effectiveness index is calculated using a multi-indicator fusion approach, encompassing three dimensions: time accuracy, response consistency, and energy efficiency. A weighted average of scores from each dimension yields a comprehensive index between 0 and 1. The preset first threshold for the standardized sensing strategy effectiveness index is 0.8; an index above this value indicates a valid strategy requiring no adjustment. The second threshold is 0.6; a value below this triggers a strategy optimization mechanism. The dynamic optimization process employs a reinforcement learning framework, adjusting the parameter search space based on the direction of index changes. When the index continuously decreases, the search range is expanded to find new solutions; when the index increases, the search range is narrowed for fine-tuning. Early warning information generation uses a tiered mechanism: attention-level warnings are generated when the index is between 0.6 and 0.8, and warning-level warnings are generated when it is below 0.6. Simultaneously, optimization history is recorded for analyzing strategy evolution trends. The system maintains a historical database of strategy effectiveness. Each optimization adjustment, including parameter settings and corresponding effectiveness indices, is recorded and archived, forming a strategy effectiveness knowledge base. When a similar scenario recurs, the system prioritizes retrieving the historically optimal solution from the knowledge base as the initial solution, accelerating the optimization convergence process. This continuous optimization mechanism enables the system to adapt to the effects of long-term factors such as seasonal changes, equipment aging, and environmental alterations, maintaining the stability of timing accuracy and control performance.

[0104] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0105] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A license plate recognition and timing system based on infrared triggering, characterized in that, include: The infrared sensing layer acquires vehicle sensing information based on a biomimetic visual sensing mechanism, applies a dynamic focusing network to the vehicle sensing information, generates a vehicle presence heat map, and divides the vehicle presence heat map into sensing areas. The license plate recognition mapping layer deploys an infrared camera array according to the divided sensing area, emits modulated infrared signals to cover the vehicle surface, and analyzes the license plate image signal through an optical character recognition algorithm to generate a license plate identification map; The cross-modal data fusion layer establishes a spatiotemporal mapping between infrared sensing and license plate recognition, synchronizes the clock using a precision clock synchronization protocol, and fuses the vehicle presence heatmap and license plate identification map through an attention fusion network to generate a credibility score. The timing control decision layer converts the credibility score into executable timing instructions based on a multi-objective decision algorithm, and sends the executable timing instructions to the timing terminal in real time through a lightweight communication protocol. The execution feedback layer monitors the changes in timing status after the timing terminal executes the timing command in real time, calculates the deviation between the timing status change and the preset timing status change threshold, generates the sensing strategy effectiveness index, and dynamically optimizes the license plate recognition sensing strategy until the sensing strategy effectiveness index reaches the optimal level. The method for acquiring vehicle sensing information based on a biomimetic visual sensing mechanism includes: The license plate recognition timing system deploys three types of sensors: inductive loop sensors, infrared camera sensors, and radar ranging sensors, which collect multimodal sensing information of vehicles passing by. Simulates the non-uniform perception mechanism of biological vision and dynamically adjusts the detector sensitivity and sampling frequency; Based on the vehicle traffic trajectory model, the front positioning area, license plate positioning area and rear positioning area are designated as high-sensitivity focusing areas; The remaining area of ​​the vehicle is designated as the low-sensitivity peripheral area. Dynamically define the sensitivity function to divide the vehicle surface into a high-sensitivity focusing area and a low-sensitivity peripheral area; The spatial analysis network analyzes vehicle features in multimodal sensing information in real time and dynamically adjusts the focusing area based on the vehicle features. The method for generating a license plate identification image includes: Infrared camera sensors of varying densities are deployed on the road surface according to the defined sensing areas. The infrared camera sensor is an array-type infrared camera sensor, employing differentiated shooting strategies for different sensing areas; An initial global image is captured by emitting modulated infrared signals onto the vehicle surface using an infrared camera sensor. The infrared shooting angle is dynamically adjusted based on the sensing area of ​​the vehicle's thermal map. The method for establishing the spatiotemporal mapping between infrared sensing and license plate recognition includes: Establish a world coordinate system with any point on the road surface as the origin; The infrared camera sensor is calibrated using a calibration board to obtain its intrinsic and extrinsic parameters. The pixel coordinates of the vehicle's heatmap are converted to sensor coordinates using the intrinsic parameters of the infrared camera sensor, and then the sensor coordinates are converted to world coordinates using the extrinsic parameters of the infrared camera sensor. Using the origin of the world coordinate system as a reference point, the camera sensor is calibrated to obtain its extrinsic parameters. The license plate identification image is converted into a world coordinate system using the extrinsic parameters of the camera sensor.

2. The license plate recognition and timing system based on infrared triggering according to claim 1, characterized in that, The method for generating a vehicle presence heatmap by applying a dynamic focusing network to vehicle sensing information includes: The multimodal sensing information is subjected to dynamic range normalization, and the normalized multimodal sensing information is stacked into a multidimensional tensor according to the feature dimension. A dynamic focusing network structure is constructed, which includes feature extraction path, feature enhancement path and feature fusion connection; Multimodal sensing information is input into a dynamic focusing network structure. In the feature extraction path, a convolutional neural network is used to extract features from the input multimodal sensing information to generate feature maps at different scales. In the feature enhancement path, the spatial resolution of the high-level feature map is gradually restored through resolution reconstruction; A feature fusion connection is established between the feature extraction path and the feature enhancement path to fuse feature maps of the same scale.

3. The license plate recognition and timing system based on infrared triggering according to claim 1, characterized in that, The method for dividing the sensing area based on the heat map of the vehicle includes: The vehicle presence status score is calculated for each region in the vehicle presence heatmap. The vehicle presence status score is obtained by weighted fusion of vehicle temperature distribution data, vehicle surface reflection data and vehicle spatial distance data included in the multimodal sensing information. A first threshold and a second threshold for vehicle presence status scoring are preset, and the vehicle presence status score is compared with the preset first threshold and the preset second threshold respectively. Different colors are used to divide the vehicle's heat map into sensing areas, with different colors representing different sensing areas.

4. The license plate recognition and timing system based on infrared triggering according to claim 1, characterized in that, The method for fusing vehicle presence heatmaps and license plate identification maps using an attention fusion network includes: A spatiotemporal map is constructed based on the vehicle existence heat map and license plate identification map in the world coordinate system; An attention fusion network is constructed using a multi-layer attention structure based on a multi-head attention mechanism to fuse vehicle existence heatmaps and license plate identification maps in the world coordinate system. Attention fusion networks consist of an input layer, a feature transformation layer, an attention interaction layer, and an output layer. The spatiotemporal graph is used as input to the input layer of the attention fusion network, and a credibility score is generated through the output layer.

5. The license plate recognition and timing system based on infrared triggering according to claim 4, characterized in that, The method for constructing the spatiotemporal graph includes: Each sensing area of ​​a vehicle in the heat map in the world coordinate system is taken as an infrared node, and the feature vector of each sensing area is extracted as the feature of the infrared node. Each recognition region of the license plate identification map in the world coordinate system is taken as a camera node, and the feature vector of each recognition region is extracted as the feature of the camera node. Collect all infrared nodes and camera nodes to obtain the node set; Calculate the spatial distance between any two infrared nodes in the world coordinate system; A preset infrared distance threshold is established, and connection edges between infrared nodes are created based on the relationship between spatial distance and the infrared distance threshold. Calculate the spatial distance between any two camera nodes in the world coordinate system; A preset camera distance threshold is established, and connection edges between camera nodes are created based on the relationship between spatial distance and the camera distance threshold. Find the nearest camera node for each infrared node and establish the connection edge between the infrared node and the camera node; Construct a spatiotemporal graph based on the obtained set of nodes and the set of connecting edges.

6. The license plate recognition and timing system based on infrared triggering according to claim 1, characterized in that, The method for converting credibility scores into executable timing instructions based on a multi-objective decision-making algorithm includes: The vehicle passage timeline is divided into time units, and each unit records the current timestamp and credibility score. It also lists all adjustable timing control parameters, including the adjustment range, computational cost, and correlation of historical impact on timing for each parameter; Three decision-making objectives are established, including a primary time accuracy objective, a secondary time accuracy objective, and an efficiency objective. Two types of constraints are set: rigid constraints and flexible constraints. Using a multi-objective decision-making algorithm, multiple sets of timing control parameter optimization schemes are randomly generated. The completion status of each scheme is evaluated for the three decision objectives, and the completion status scores of the three decision objectives are obtained. The completion status scores of the three decision objectives are combined to obtain an overall score. The scheme with the highest overall score is selected from the multiple schemes as the final optimization scheme.

7. The license plate recognition and timing system based on infrared triggering according to claim 6, characterized in that, The method for generating the effectiveness index of the sensing strategy includes: Collect timing command parameters executed by the timing terminal, synchronously monitor changes in timing status, calculate the deviation between the changes in timing status and the preset threshold for changes in timing status, and generate a standardized sensing strategy effectiveness index. Preset a first threshold and a second threshold for the effectiveness index of the standardized sensing strategy; The effectiveness of the sensing strategy is determined by the relationship between the standardized sensing strategy effectiveness index and the preset threshold. The license plate recognition sensing strategy is dynamically optimized and warning information is generated.

Images