Light-weight truck axle type detection method based on infrared imaging

Abstract

The invention discloses a light-weight truck axle type detection method based on infrared imaging, and the method specifically comprises the steps: installing a camera on a road portal frame, installing an infrared thermal imager and an edge calculation device on a road side, enabling the camera to carry out the snapshot of a truck and the recognition of a license plate number, enabling the infrared thermal imager to obtain temperature data, enabling the edge calculation device to be responsible for calculation, and carrying out the detection of the axle type of the truck. According to the method, a pseudo-color image is generated according to temperature data, image enhancement is carried out, based on the enhanced image, a lightweight YOLOv11 model improved based on a dynamic pruning strategy and an adaptive feature enhancement module is adopted to carry out truck axle type detection, and the reliability and real-time performance of all-weather detection are remarkably improved.

Description

Technical Field

[0001] The present invention relates to axle type detection technology, in particular to a light truck axle type detection method based on infrared imaging. Background Art

[0002] With the rapid development of the modern logistics industry, truck traffic continues to increase, bringing with it increasing challenges in traffic safety and road management. Trucks, particularly on highways and urban roads, can easily create significant road pressure due to their large size and heavy loads, making their regulation increasingly important.

[0003] Although the traditional truck axle type detection method based on visible light images can identify the truck axle type to a certain extent, its detection effect is not ideal in complex environments due to factors such as lighting and weather conditions. In addition, existing infrared imaging technology effectively overcomes the limitations of visible light by capturing temperature differences, but its images still face problems such as a wide temperature range, low contrast, and high noise interference, resulting in limited detection accuracy. Moreover, when deployed on edge devices, the computing resource bottleneck of deep learning models is further highlighted: traditional model compression methods (such as static pruning) cannot dynamically adjust the computing load according to the scene, and redundant channels need to be retained in complex environments to maintain accuracy, while redundant calculations in simple scenes lead to a surge in delays. How to combine the temperature characteristics of infrared images to achieve adaptive lightweighting of the model has become a key challenge to improving detection efficiency and accuracy in complex environments. Summary of the Invention

[0004] In order to solve the above technical defects in the prior art, the present invention proposes a lightweight truck axle type detection method based on infrared imaging.

[0005] The technical solution for achieving the purpose of the present invention is: a method for detecting the axle shape of a lightweight truck based on infrared imaging, comprising:

[0006] The camera captures visible light images of the trigger line position and lane lines in real time. When a vehicle is detected crossing the trigger line, the camera is triggered to take a photo. After the camera detects a truck and recognizes its license plate number and lane, it triggers the infrared thermal imager to obtain temperature data.

[0007] The edge computing device obtains the license plate number, lane, and temperature data, generates a pseudo-color image based on the temperature data, and performs contrast-limited adaptive histogram equalization image enhancement on the pseudo-color image.

[0008] A lightweight YOLOv11 model improved based on dynamic pruning strategy and adaptive feature enhancement module is used to detect truck axle types in pseudo-color images after image enhancement.

[0009] Compared with the prior art, the present invention has the following significant advantages:

[0010] (1) In combination with infrared imaging technology, the present invention can detect truck axle profiles during the day and at night, and in various climate conditions. The use of infrared thermal imagers effectively overcomes the issues with traditional visible light imaging being affected by light and weather, providing accurate monitoring capabilities in all weather conditions and at all times.

[0011] (2) The present invention innovatively adopts a temperature-driven dynamic pruning strategy, dynamically adjusts the model complexity based on the temperature distribution characteristics of infrared images, and combines pre-trained hierarchical pruning templates to achieve low-latency inference on edge devices, significantly improving real-time performance and energy efficiency.

[0012] (3) The present invention improves the YOLOv11 network by introducing an adaptive feature enhancement module, which can significantly improve the detection accuracy of truck axle types, especially in complex backgrounds, low contrast and multi-scale scenes.

[0013] (4) The present invention uses contrast-limited adaptive histogram equalization image enhancement technology to effectively improve the quality of infrared images, reduce the effects of wide temperature range, low contrast and noise interference, optimize image details and local contrast, and make the detection of truck axle types more accurate and reliable.

[0014] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description or be understood by practicing the present invention. The purposes and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a workflow diagram of the present invention.

[0016] Figure 2 This is the improved YOLOv11 network structure diagram with the addition of an adaptive feature enhancement module.

[0017] Figure 3 This is the structure diagram of the adaptive feature enhancement module.

[0018] Figure 4 This is a truck axle type detection effect diagram of the present invention.

[0019] Figure 5 It is a schematic diagram of the equipment installation position of the present invention. DETAILED DESCRIPTION

[0020] The present invention will be described below with reference to the accompanying drawings.

[0021] The present invention relates to a method for detecting axle type of lightweight truck based on infrared imaging, the flow chart of which is as follows: Figure 1 As shown, the following steps are included:

[0022] S1. A short horizontal bar is extended outward from the gantry column 4m above the ground toward the outside of the road. The infrared thermal imager is installed on the bar. Specific installation requirements are as follows: the horizontal direction is tilted 45° toward the direction of the truck's movement, and the plumb direction is tilted 10° downward from the horizontal to ensure that the infrared thermal imager can fully capture the truck. The camera is installed on the gantry at a height of 8m from the ground, with the horizontal direction facing the truck's movement and the plumb direction tilted 15° downward from the horizontal to ensure that the camera can fully capture the vehicle at the capture position.

[0023] S2. Install the infrared thermal imager control program and the camera control program on the edge computing device, set the camera capture trigger line position and lane line, and set the two to be linked.

[0024] S3. The camera has computing and recognition capabilities and acquires visible light images in real time. When a vehicle is detected passing the trigger line, the camera is triggered to take a photo. After the camera detects the truck and identifies its license plate number and lane, it transmits the detection event to the edge computing device. The edge computing device then transmits the signal to the infrared thermal imager, triggering the infrared thermal imager to obtain temperature data.

[0025] S4. The edge computing device obtains the license plate number, lane, and temperature data, generates a pseudo-color image based on the temperature data, and performs contrast-limited adaptive histogram equalization image enhancement on the pseudo-color image. The specific steps are as follows:

[0026] To process temperature data using the GPU, the temperature data is first converted into a tensor on the GPU. The temperature data is normalized, mapping the temperature range to a grayscale value of [0, 255], thus generating a pseudo-color image. The normalized temperature data is then subjected to min-max standardization to expand the data's dynamic range. This allows the temperature data to be mapped to the color gradient of the pseudo-color image, allowing different temperature values ​​to be displayed more clearly. In actual temperature data processing, the temperature slope and offset are taken into account to ensure accurate conversion of temperature values.

[0027] The infrared pseudo-color image I(x,y) is divided into K×L non-overlapping local regions {B k,l}, each sub-block has a size of M×N, satisfying:

[0028] B k，l ={I(x,y)|(k-1)M <x≤kM,(l-1)N<y≤lN}

[0029] Where k = 1, 2, ..., K, l = 1, 2, ..., L. To avoid loss of boundary pixels, the extended area of ​​the image is mirror-filled with filling widths of M / 2 and N / 2.

[0030] For each sub-block Bk，l Perform grayscale histogram statistics and contrast limit clipping as follows:

[0031] Count the number of pixels at each gray level i:

[0032]

[0033] Where δ(·) is the Kronecker function (takes 1 when the parameter is 0, otherwise it is 0), and L is the number of gray levels.

[0034] Set the clipping threshold T to avoid noise amplification:

[0035]

[0036] Crop each grayscale level in the histogram and redistribute the crop amount:

[0037] Δ i =max(H k,l (i)-T,0)

[0038] H′ k，l (i) = min(H k，l (i),T)

[0039]

[0040] Compute the cumulative distribution function:

[0041]

[0042] Normalize the cumulative distribution function and map it to the full grayscale range:

[0043]

[0044] Apply the mapping function to obtain the enhanced sub-block:

[0045]

[0046] To avoid discontinuity at the sub-block boundaries, bilinear interpolation is used to eliminate the blocking effect and find the center points of the four adjacent sub-blocks where (x, y) are located. Where m, n∈{0, 1}. The weight is calculated based on the distance between the pixel position and the sub-block center:

[0047]

[0048] Weighted fusion finally enhances pixel value:

[0049]

[0050] The CLAHE-processed image is converted to a color pseudo-color image using the pseudo-color mapping function in OpenCV. The color spectrum allows different temperature values ​​to be represented in different colors, making the temperature distribution in the image more intuitive and easier to analyze.

[0051] S5. Use the improved lightweight YOLOv11 model based on dynamic pruning strategy and adaptive feature enhancement module to detect the axle type of trucks on the pseudo-color image after image enhancement. The improved YOLOv11 network structure is shown in the attached Figure 2 The lightweight YOLOv11 model mainly consists of three parts: backbone network, neck network and detection head, specifically:

[0052] The backbone network consists of multiple sets of basic convolutional layers, four sets of C3K2 modules, an adaptive feature enhancement module (AFE) at the end, a spatial pyramid fast pooling module (SPPF), and a parallel spatial attention module (C2PSA). The processing flow is as follows: After the input image extracts primary features through the basic convolutional layers, four sets of alternating basic convolutional layers and C3K2 modules gradually optimize mid- and high-level semantic features. The C3K2 module is optimized based on the C3 module. By specifying a parameter of 2, two C3Ks are connected in series, further improving the efficiency and capability of feature extraction. At the end, the adaptive feature enhancement module strengthens multi-scale details, the spatial pyramid fast pooling module integrates global context, and the parallel spatial attention module assigns weights to key areas, ultimately outputting highly discriminative features, significantly improving the accuracy and robustness of truck axle detection in complex scenarios.

[0053] In a further embodiment, the adaptive feature enhancement module consists of four key parts: first, the convolutional embedding, which processes the input features through layer normalization and convolutional embedding layers, and compresses the number of channels by half through 1x1 convolution, reducing the computational burden and promoting feature fusion; then, the spatial context module, which expands the receptive field by using larger convolution kernels (such as 7x7 group convolutions) to capture a wider range of spatial context information, which is particularly suitable for scale changes in images; next is the feature refinement module, which optimizes feature extraction based on the concepts of image sharpening and contrast enhancement, can effectively capture low-frequency and high-frequency information, and enhance detail features; finally, the outputs of the spatial context module and the feature refinement module are fused through 1x1 convolution and convolutional multi-layer perceptron to generate the final enhanced features.

[0054] The neck network adopts a bidirectional cross-layer feature fusion architecture. First, the deep semantic features output by the end of the backbone network through spatial pyramid pooling and channel attention optimization are upsampled by 2 times, and channel-aligned and spliced ​​with the medium-resolution features output by the third C3k2 module of the backbone network. The features are processed by a feature optimization module containing depthwise separable convolution and channel attention to reduce computational redundancy. The fusion results are then input into the C3k2 module to extract features, and are upsampled twice and spliced ​​with the high-resolution shallow features output by the second C3k2 module of the backbone network. The features are then input into the C3k2 module to extract features, and cross-layer interaction is performed with the previous fusion features through convolution downsampling. Finally, three-level feature maps of 160×160 (fusion of shallow positioning details), 80×80 (combined middle-level structural features) and 40×40 (integration of deep global semantics) are output.

[0055] The detection head is the decision module of the YOLOv11 network, which is responsible for generating the final detection results.

[0056] The temperature-driven dynamic pruning strategy designs the pruning rate based on the scene complexity, ranks the L1 norm of the trained model channel weights by importance, generates multiple sets of pruning templates, retains key channels, and eliminates redundancy. During deployment, the model weights and templates are converted to ONNX format and integrated into the edge device. During inference, the dynamic pruning controller calculates the scene complexity index in real time and loads the corresponding templates on demand, achieving an adaptive balance between edge detection accuracy and efficiency, ensuring the reliability and real-time performance of truck axle detection in complex environments. The implementation details are as follows:

[0057] In order to evaluate the complexity of the image, this embodiment calculates the temperature complexity index (TCI) based on the temperature distribution, contrast and gradient of the infrared image.

[0058] First, the temperature matrix data of the truck is obtained through the infrared thermal imager And normalize it and scale the temperature range to the interval of 0 and 1.

[0059] The following features are calculated based on the normalized temperature values:

[0060] 1. Temperature entropy: measures the degree of discreteness of temperature distribution. The calculation formula is:

[0061]

[0062] where t i is the discretization interval of temperature value, p(t i ) is the frequency of the temperature value in interval i.

[0063] 2. Temperature contrast: Indicates the difference between the maximum and minimum temperature values ​​in the image. The calculation formula is:

[0064]

[0065] Where T max is the maximum value of the original temperature, T min 、T mean are the minimum and mean temperatures.

[0066] 3. Temperature gradient: describes the rate of change of temperature in the image. The Sobel operator is used to directly calculate the gradient amplitude map G of the temperature matrix T. The calculation formula is:

[0067]

[0068] The above features are combined to generate the comprehensive complexity index (TCI). The TCI calculation formula is:

[0069]

[0070] To dynamically select pruning templates based on image complexity, this embodiment sets multiple TCI thresholds. These thresholds are used to classify the input image during inference into different complexity scenarios. In this embodiment, the TCI thresholds θ1 and θ2 are determined according to the following steps: 1. Through experiments and analysis of a large amount of infrared image data, the TCI value range for different complexity scenarios is statistically analyzed; 2. Through cross-validation and model testing, the impact of different TCI thresholds on pruning accuracy and computational complexity is evaluated; 3. Based on the experimental results, TCI thresholds for low, medium, and high complexity scenarios are determined.

[0071] Based on the TCI value of the image, this embodiment predefines multiple pruning templates, each corresponding to a different pruning ratio, and selects the appropriate template based on different complexity scenarios. The specific template design is as follows:

[0072] High complexity scenario (TCI ≥ θ1): retain more channels to maintain detection accuracy, and the pruning ratio is r = r low ;

[0073] Medium complexity scenario (θ2≤TCI≤θ1): Balance accuracy and efficiency, and the pruning ratio is r=r mid ;

[0074] Low complexity scenario (TCI < θ2): Maximize computational efficiency, and the pruning ratio is r = r high .

[0075] Among them, θ1 and θ2 are the preset complexity thresholds (θ1>θ2), r low 、r mid 、r high is the corresponding pruning ratio.

[0076] Channel importance scoring and template generation (offline phase): The channel attention mechanism is used to evaluate the importance of each channel to assist in deciding which channels should be retained. The attention score of each channel reflects the importance of the channel in the current task. Specifically, the channel attention score calculation formula is as follows:

[0077]

[0078] For each channel c of the convolutional layer in the backbone network and the neck network, calculate the L1 norm of its weight, where W c Represents the weight tensor of the c-th channel.

[0079] Sort by channel attention score from high to low, retaining the first (1-r)×C backbone For each type of scene complexity, a corresponding pruning template file is generated to record the index of the retained channels in each layer.

[0080] In further implementation, the training and deployment process of the improved YOLOv11 model is as follows:

[0081] During the model training phase, infrared image data collected around the clock from trucks is captured by a camera, which captures visible light images and simultaneously uses an infrared thermal imager to obtain full-frame temperature data. After normalization, the temperature data is mapped to a pseudo-color image, generating an infrared dataset containing different axle types (two-axis, three-axis, ..., six-axis, and other axle types).

[0082] Use LabelImg software to annotate the pseudo-color image and generate a label file in YOLO format. Then, modify the output layer parameters of the YOLOv11 network, define the number of training rounds, batch size, and input size, and input the dataset for training.

[0083] After training is complete, the convolutional layers in the model's backbone and neck networks are ranked by importance based on the L1 norm of their weights, generating multiple sets of pruning templates. Each template is tailored to the complexity of different scenarios, retaining key channels that are most responsive to temperature characteristics while pruning redundant channels to reduce computational complexity. The template file records the index of the retained channels in each layer and is stored alongside the model weights.

[0084] During the deployment phase, the trained model weights and pruning templates are converted to the ONNX format and integrated into the edge computing device. The device has a built-in dynamic pruning controller that receives real-time temperature data from the infrared thermal imager, calculates the scene complexity index (TCI), and dynamically loads the corresponding pruning template based on the TCI value.

[0085] During inference, the pruned model optimizes execution efficiency through sparse computing techniques, skipping ineffective computations in pruned channels while also combining quantization and compression to reduce memory usage. If a sudden change in scene complexity (such as sudden extreme weather events) is detected, the system automatically switches to a conservative pruning template and triggers an alarm to maintain detection robustness. Through this process, the model achieves an adaptive balance between accuracy and efficiency at the edge, providing efficient and reliable technical support for truck axle detection in complex environments.

Claims

1. A method for detecting axle type of lightweight truck based on infrared imaging, characterized in that: include: The camera captures visible light images of the trigger line position and lane lines in real time. When a vehicle is detected crossing the trigger line, the camera is triggered to take a photo. After the camera detects a truck and recognizes its license plate number and lane, it triggers the infrared thermal imager to obtain temperature data. The edge computing device obtains the license plate number, lane, and temperature data, generates a pseudo-color image based on the temperature data, and performs contrast-limited adaptive histogram equalization image enhancement on the pseudo-color image. A lightweight YOLOv11 model improved based on dynamic pruning strategy and adaptive feature enhancement module is used to detect truck axle types in pseudo-color images after image enhancement.

2. The method for detecting axle type of a lightweight truck based on infrared imaging according to claim 1, characterized in that: The infrared thermal imager has a viewing angle that completely covers all lanes after installation, and can completely capture the truck at the capture position.

3. The method for detecting axle type of a lightweight truck based on infrared imaging according to claim 1, characterized in that: The camera, infrared thermal imager, and edge computing device are in the same local area network for communication and data transmission.

4. The method for detecting axle type of a lightweight truck based on infrared imaging according to claim 1, characterized in that: The edge computing device obtains the license plate number, lane, and temperature data, generates a pseudo-color image based on the temperature data, and performs contrast-limited adaptive histogram equalization image enhancement on the pseudo-color image. The specific method is as follows: Normalize the temperature data and map the temperature range to the grayscale value of [0,255] to obtain a pseudo-color image; Divide the pseudo-color image into several local areas; Calculate the histogram for each local area and clip the histogram according to the preset contrast limit threshold; Perform local equalization on the cropped histogram to obtain an enhanced local image; The bilinear interpolation method is used to fuse the boundaries of each local image to eliminate the blocking effect and form an overall enhanced image.

5. The method for detecting axle type of a lightweight truck based on infrared imaging according to claim 4, characterized in that: The specific method of calculating the histogram for each local area and clipping the histogram according to the preset contrast limit threshold is as follows: Count the number of pixels at each gray level i: Among them H k,l (i) represents the number of pixels with grayscale level i in the kth and lth local regions, where k and l refer to the coordinates of the local region of the image, i is the grayscale level, representing the pixel value, and δ(·) is the Kronecker function. When the pixel value I(x,y) equals i, the function returns 1, otherwise it returns 0. It is used to count the number of pixels at each grayscale level. L is the number of grayscale levels, indicating the total number of grayscale levels in the image. Set the clipping threshold T to avoid noise amplification: M×N is the size of the local area; Crop each grayscale level in the histogram and redistribute the crop amount: Δ i =max(H k,l (i)-T,0) H′ k，l (i)=min(H k，l (i),T) Δ i H′ is the grayscale change, which represents the difference between the current grayscale level and the threshold value, and is used for image enhancement. k,l (i) is the histogram value after clipping, H″ k,l (i) is the histogram value after smooth redistribution, Δ j is the number of pixels in the grayscale that are clipped.

6. The method for detecting axle profile of a lightweight truck based on infrared imaging according to claim 5, characterized in that: The specific method of performing local equalization processing on the cropped histogram to obtain the enhanced local image is as follows: Compute the cumulative distribution function: Normalize the cumulative distribution function and map it to the full grayscale range: Apply the mapping function to obtain the enhanced sub-block:

7. The method for detecting axle profile of a lightweight truck based on infrared imaging according to claim 5, characterized in that: The specific method of using bilinear interpolation method to fuse the boundaries of each local image and eliminate the blocking effect to form an overall enhanced image is as follows: Find the center points of the four adjacent sub-blocks where the enhanced pixel is located at (x, y) in the image Where m,n∈{0,1}; The weight is calculated based on the distance between the pixel position and the sub-block center: Weighted fusion finally enhances pixel value:

8. The method for detecting axle profile of a lightweight truck based on infrared imaging according to claim 1, characterized in that: The lightweight YOLOv11 model includes a backbone network, a neck network, and a detection head. The backbone network consists of multiple groups of basic convolutional layers, four groups of C3K2 modules, an adaptive feature enhancement module at the end, a spatial pyramid fast pooling module, and a parallel spatial attention module. After the input image is processed by the basic convolutional layer to extract primary features, the four groups of alternating basic convolutional layers and C3K2 modules are used to gradually optimize the mid- and high-level semantic features. The adaptive feature enhancement module strengthens multi-scale details, the spatial pyramid fast pooling module fuses global context, and the parallel spatial attention module assigns weights to key areas, ultimately outputting highly discriminative features. The neck network adopts a bidirectional cross-layer feature fusion architecture. First, the deep semantic features output by the end of the backbone network through spatial pyramid pooling and channel attention optimization are upsampled by a factor of 2. The deep semantic features are then aligned and spliced ​​with the medium-resolution features output by the third C3k2 module of the backbone network. The features are then processed by a feature optimization module containing depthwise separable convolution and channel attention. The fusion results are then input into the C3k2 module to extract features, upsampled twice, and spliced ​​with the high-resolution shallow features output by the second C3k2 module of the backbone network. Then the C3k2 module is input to extract features, and cross-layer interaction is performed with the previous fusion features through convolution downsampling, and finally a three-level feature map is output; The detection head is the decision-making module of the YOLOv11 network, responsible for generating the final detection results. It processes the three-level feature maps output by the neck network through three independent Detect layers. Each Detect layer simultaneously outputs the target bounding box coordinates, detection confidence, and classification probability. It fuses multiple scales through non-maximum suppression and outputs the axis detection results.

9. The method for detecting axle type of a lightweight truck based on infrared imaging according to claim 1, characterized in that: The pruning strategy of the lightweight YOLOv11 model is as follows: The pruning rate is determined based on the complexity of the scene, specifically: Calculate the comprehensive complexity index TCI of the image and determine the pruning ratio based on the comprehensive complexity index, specifically: High complexity scenario, i.e. TCI ≥ θ1: maintain pruning ratio r = r low ; Medium complexity scenario, i.e. θ2≤TCI≤θ1: the pruning ratio is r=r mid ; Low complexity scenario, i.e. TCI < θ2: pruning ratio is r = r high Where θ1θ2 are the first and second TCI thresholds, respectively, r min 、r mid 、r high is the set pruning ratio; The channel attention mechanism is used to evaluate the importance score of each channel to assist in the decision-making of pruning channels. Specifically: Calculate the channel attention score of channel c of each convolutional layer in the backbone network and the neck network. The specific formula is: In the formula, Score c is the attention score of channel c, which measures the importance of channel c. The higher the score, the greater the influence of the channel on the final model. c Represents the weight tensor of the cth channel, i, j, k represents the weight W c The index of each element in , i and j represent the spatial dimension, and k represents the channel; Sort by channel attention score from high to low, retaining the first (1-r)×C backbone A channel is used to generate corresponding pruning template files for each type of scene complexity.

10. The method for detecting axle type of a lightweight truck based on infrared imaging according to claim 9, characterized in that: The calculation formula of the comprehensive complexity index TCI of the image is: Where, temperature entropy t i is the discretization interval of temperature value, p(t i ) is the frequency of temperature values ​​in interval i; T max is the maximum value of the original temperature, T min 、T mean are the minimum and mean temperatures; is the maximum possible range of temperature, set by the maximum temperature outside the image; is the temperature gradient, which describes the rate of change of temperature in the image. The calculation formula is: G i，j is the temperature value of the image at (i, j); α, β, and γ are weight coefficients used to balance the contribution of the three parts in the overall index.

Images