Vehicle key area temperature monitoring method based on improved YOLOv9

Abstract

The present application relates to the technical field of image processing and visual positioning, and particularly relates to a vehicle key area temperature monitoring method based on improved YOLOv9. First, vehicle monitoring data of a high-speed driving area is acquired, and infrared images corresponding to each high-speed driving vehicle are accurately acquired according to the vehicle monitoring data; then the infrared images are input into an improved YOLOv9 recognition model, and multiple segmentation areas of the high-speed driving vehicle are accurately acquired by using the improved YOLOv9 recognition model; so that the multiple key area temperatures of the high-speed driving vehicle can be acquired according to the multiple segmentation areas and the temperature data, the key area temperature of the high-speed driving vehicle is accurately monitored, and further investigation and confirmation of the vehicle spontaneous combustion hidden danger are completed.

Description

Technical Field

[0001] This invention relates to the field of image processing and visual positioning technology, specifically to a method for monitoring the temperature of key areas of a vehicle based on an improved YOLOv9. Background Technology

[0002] Currently, the handling of highway safety accidents is shifting from post-accident response to proactive prevention and source control. Researching effective methods to reduce highway fire accidents and mitigate operational safety risks has become a hot topic in the industry. Vehicle spontaneous combustion is a major cause of highway fires, often preceded by early signs such as excessively high temperatures in critical areas like tires.

[0003] Therefore, to reduce tunnel fire accidents from the source of risk, it is necessary to promptly detect high-speed vehicles that are about to reach or have already reached their auto-ignition temperature, so as to take measures as early as possible. However, accurately monitoring the temperature of vehicles traveling at high speeds on highways is extremely difficult. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention proposes a vehicle critical area temperature monitoring method based on an improved YOLOv9, which can accurately monitor the temperature of critical areas of a vehicle during high-speed driving.

[0005] The technical solution adopted in this invention is an improved method for monitoring the temperature of key areas of a vehicle using YOLOv9.

[0006] In a first feasible approach, a method for monitoring temperature in critical areas of a vehicle based on an improved YOLOv9 includes:

[0007] Monitor vehicles in high-speed driving areas and obtain vehicle monitoring data;

[0008] Based on vehicle monitoring data, determine the infrared images corresponding to each high-speed vehicle, as well as the temperature data corresponding to the infrared images;

[0009] Infrared images are input into the improved YOLOv9 recognition model to obtain multiple segmented regions of high-speed vehicles;

[0010] The temperature of multiple key areas of a high-speed vehicle is obtained based on the segmented regions and temperature data.

[0011] In conjunction with the first feasible method, the second feasible method involves monitoring vehicles within the high-speed driving area and acquiring vehicle monitoring data, including:

[0012] In high-speed driving areas, high-definition cameras capture images of vehicles traveling at high speeds, radar speed detectors measure vehicle speeds, and infrared thermal imagers capture infrared video and temperature data of vehicles traveling at high speeds.

[0013] In the third feasible method, in conjunction with the second feasible method, the infrared image corresponding to each high-speed vehicle is determined based on vehicle monitoring data, including:

[0014] The license plate numbers of each high-speed vehicle were determined based on high-definition images of each vehicle traveling at high speed.

[0015] Determine the speed of the high-speed vehicle at the moment of each high-definition image capture;

[0016] The duration of the data transfer is determined based on the capture time and vehicle speed.

[0017] The infrared images corresponding to each high-speed vehicle were determined from the retrospective infrared video.

[0018] The infrared images of each high-speed vehicle are named and stored according to the license plate number and the time of capture.

[0019] In conjunction with the second feasible method, the fourth feasible method involves determining the infrared image corresponding to each high-speed vehicle based on vehicle monitoring data, including:

[0020] Infrared images of high-speed driving areas with no vehicles passing through are used as calibration infrared images;

[0021] The calibrated infrared images are updated in real time according to a preset interval; vehicles in high-speed driving areas are captured by a high-definition checkpoint camera, and infrared video within a preset time period is determined based on the capture time.

[0022] A preliminary comparison is made between each frame of infrared image in the infrared video within a preset time period and the calibrated infrared image, and the change area is delineated based on the comparison results.

[0023] The infrared image containing the region of greatest pixel fluctuation is identified as the infrared image corresponding to the high-speed vehicle.

[0024] In conjunction with the first feasible approach, the fifth feasible approach, before inputting the infrared image into the improved YOLOv9 recognition model, includes:

[0025] The temperature data corresponding to the infrared image is rendered and colored to obtain the rendered temperature data.

[0026] The SIN function is used to map the rendered temperature data to obtain an updated infrared image.

[0027] Combining the first feasible approach, the sixth feasible approach improves the YOLOv9 recognition model by including:

[0028] The input module resizes the input infrared image and performs data enhancement processing on the resized infrared image to obtain an enhanced image.

[0029] The feature extraction module performs convolution processing on the enhanced image to obtain vehicle features;

[0030] The feature fusion module fuses vehicle features through the SEMA module to obtain multi-scale features;

[0031] The output module inputs multi-scale features into three detectors, and outputs the segmented region of the high-speed vehicle through the detectors.

[0032] In the seventh possible implementation, combining the sixth implementation method, the SEMA module includes a first branch and a second branch. The first branch processes vehicle features to obtain a first branch result. The first branch result is then normalized, pooled, and activated to obtain first branch channel information. The second branch processes vehicle features to obtain a second branch result. The second branch result is then pooled and activated to obtain second branch channel information. The first and second branch channel information are interacted to obtain first and second interaction information. The first and second interaction information are fused, and the fused result is activated to obtain attention weights. These attention weights are then used to weight vehicle features to obtain multi-scale features.

[0033] In conjunction with the sixth feasible method, in the eighth feasible method, the first branch is used to encode the vehicle features in the vertical and horizontal directions; the vertical and horizontal encodings are convolved simultaneously to obtain the first convolution result; the first convolution result is decomposed into two vectors; the two vectors are activated; the vehicle features are weighted using the two activation vectors to obtain the first branch result.

[0034] Combining the sixth feasible approach, in the ninth feasible approach, the second branch is used to calculate the maximum and average values ​​of vehicle features and construct a spatial graph; the spatial graph is concatenated and convolved to obtain global spatial attention weights; the vehicle features are weighted using the global spatial attention weights to obtain vehicle spatial weighted features; the vehicle spatial weighted features are aggregated using a pooling pyramid to obtain the result of the second branch.

[0035] In the tenth possible implementation, combining the first feasible method, the infrared image is input into the improved YOLOv9 recognition model to obtain multiple segmented regions of the high-speed vehicle, including:

[0036] The infrared image is input into the improved YOLOv9 recognition model to obtain the tire area;

[0037] The infrared image is segmented based on the tire area to obtain multiple segmented regions.

[0038] As can be seen from the above technical solution, the beneficial technical effects of the present invention are as follows:

[0039] 1. First, acquire vehicle monitoring data in the high-speed driving area, and accurately obtain infrared images corresponding to each high-speed vehicle based on the vehicle monitoring data; then input the infrared images into the improved YOLOv9 recognition model, and use the improved YOLOv9 recognition model to accurately obtain multiple segmented regions of the high-speed vehicle; thus, based on each segmented region and temperature data, obtain the temperature of multiple key areas of the high-speed vehicle, and achieve accurate monitoring of the temperature of key areas of the vehicle during high-speed driving, so as to ensure the further investigation and confirmation of potential vehicle fire hazards.

[0040] 2. The improved YOLOv9 recognition model adopts the SEMA attention module, which pays more attention to spatial information. This module focuses on global spatial location information, enhances the ability to focus on foreground targets, and can play a role in multi-scale aggregation of upstream tasks. This greatly improves the accuracy of the improved YOLOv9 recognition model, which helps to solve the problem of missed detection and false detection caused by the similarity of foreground and background information in vehicle images in high-speed driving scenarios, and further improves the accuracy of monitoring the temperature of key areas of vehicles in high-speed driving. Attached Figure Description

[0041] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0042] Figure 1 This is a schematic diagram of a vehicle critical area temperature monitoring method based on an improved YOLOv9 provided in this embodiment;

[0043] Figure 2 This embodiment provides a layout diagram of a vehicle critical area temperature monitoring system based on an improved YOLOv9.

[0044] Figure 3 This embodiment provides a network structure diagram of a SEMA module;

[0045] Figure 4 This embodiment provides a network structure diagram for CAP.

[0046] Figure 5 This embodiment provides a network structure diagram for SAP.

[0047] Figure label:

[0048] 1-Gantry frame, 2-First infrared thermal imager, 3-High-definition bayonet camera, 4-Second infrared thermal imager, 5-Radar speedometer, 6-Switch, 7-Industrial control computer, 8-Hard disk recorder, 9-Cloud server. Detailed Implementation

[0049] The embodiments of the technical solution of the present invention will now be described in detail with reference to the accompanying drawings. These embodiments are merely illustrative of the technical solution of the present invention and are therefore intended to limit the scope of protection of the present invention.

[0050] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application should have the ordinary meaning understood by those skilled in the art. The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for implementation of the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. Unless otherwise stated, the term "a plurality of" means two or more. In this disclosure, the character " / " indicates an "or" relationship between the preceding and following objects. For example, A / B means: A or B. The term "and / or" describes an association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B. The term "corresponding" can refer to an association or binding relationship; A corresponding to B means that there is an association or binding relationship between A and B.

[0051] Combination Figure 1 As shown, this embodiment provides a method for monitoring the temperature of key areas of a vehicle based on an improved YOLOv9, including:

[0052] Step S01: Monitor vehicles in the high-speed driving area and obtain vehicle monitoring data;

[0053] Step S02: Determine the infrared image corresponding to each high-speed vehicle and the temperature data corresponding to the infrared image based on the vehicle monitoring data;

[0054] Step S03: Input the infrared image into the improved YOLOv9 recognition model to obtain multiple segmented regions of the high-speed vehicle;

[0055] Step S04: Obtain the temperature of multiple key areas of the high-speed vehicle based on the segmented areas and temperature data.

[0056] Optionally, vehicles in the high-speed driving area are monitored to obtain vehicle monitoring data, including: capturing images of vehicles in the high-speed driving area using a high-definition camera to obtain high-definition images of high-speed vehicles, measuring the speed of high-speed vehicles using a radar speed detector, and capturing images of vehicles using an infrared thermal imager to obtain infrared video and temperature data of high-speed vehicles.

[0057] Combination Figure 2 As shown, a temperature monitoring system for key areas of a high-speed vehicle based on an improved YOLOv9 includes a high-definition checkpoint camera 3, a first infrared thermal imager 2 and a second infrared thermal imager 4, a radar speedometer 5, a hard disk recorder 8, a switch 6, an industrial control computer 7, and a cloud server 9.

[0058] On the gantry 1 on the highway, a high-definition checkpoint camera 3 is deployed in the center of the horizontal bar, a first infrared thermal imager 2 and a second infrared thermal imager 4 are deployed on the left and right sides of the horizontal bar respectively, and a radar speed detector 5 is deployed in the center of the right vertical bar; the high-definition checkpoint camera 3, the first infrared thermal imager 2 and the second infrared thermal imager 4 and the radar speed detector 5 are connected to an industrial control computer 7 through a switch 6; the industrial control computer 7, the hard disk recorder 8 and the cloud server 9 are connected in sequence.

[0059] Optionally, the high-definition camera supports the function of quickly capturing license plates.

[0060] Combination Figure 1 As shown, the first and second infrared thermal imagers are high-speed infrared thermal imagers that can cover three lanes, including the fast lane, the normal driving lane and the emergency lane, with a field of view of 100m to 150m and selectable resolutions of 384*288 and 640*512.

[0061] Optionally, the switch supports 10 Gigabit Ethernet, ensuring real-time output of infrared video and full-screen temperature data from high-speed infrared thermal imagers to the local industrial control computer, and output of surveillance video from high-definition camera cameras to a hard disk recorder. Infrared images are acquired by extracting frames from infrared video, and high-definition images are acquired by extracting frames from surveillance video.

[0062] Optionally, the infrared images corresponding to each high-speed vehicle are determined based on vehicle monitoring data, including: determining the license plate number of each high-speed vehicle based on the high-definition images of each high-speed vehicle; determining the speed of the high-speed vehicle at the time of capture of each high-definition image; determining the backtracking time based on the capture time and speed; determining the infrared images corresponding to each high-speed vehicle from the infrared video based on the backtracking time; and naming and storing the infrared images corresponding to each high-speed vehicle by license plate number and capture time.

[0063] Optionally, it also includes controlling the field of view of the infrared thermal imager within the range of 100m to 150m. A field of view greater than 100m ensures that a complete photo of the vehicle can be captured, while a field of view less than 150m ensures that the vehicle image in the infrared image can be identified.

[0064] Optionally, the playback duration can be determined based on the capture time and vehicle speed, including:

[0065] Where t0 is the backtracking duration, t1 is the capture time of the high-definition image, and v1 is the vehicle speed.

[0066] In some embodiments, the high-definition checkpoint camera has a license plate recognition function. The high-definition image captured by the camera identifies the license plate number 'a' of a high-speed vehicle, and the high-speed vehicle is named 'a'. The speed measured by the radar speed detector at the capture time is determined; this speed is the speed of high-speed vehicle 'a'. The backtracking time is calculated based on the vehicle speed and the capture time. An infrared image frame containing the backtracking time position is selected from the infrared video captured by the infrared thermal imager; this selected infrared image is the infrared image of high-speed vehicle 'a'; the temperature data corresponding to the selected infrared image is the temperature data corresponding to high-speed vehicle 'a'. The selected infrared image and the corresponding temperature data are named by the license plate number and the capture time, and stored separately on the local industrial control computer. Thus, based on the capture time of the high-definition checkpoint camera, the speed of the high-speed vehicle is measured by the radar speed detector, and the required backtracking time of the infrared video is calculated. The infrared image of the high-speed vehicle is then obtained from the backtracking time of the infrared video. Compared to capturing infrared images by triggering a snapshot, this solution involves retrospectively analyzing the infrared video to obtain the infrared image from the retrospective duration, resulting in smaller time errors and greater accuracy.

[0067] Optionally, the infrared image corresponding to each high-speed vehicle is determined based on vehicle monitoring data, including: using infrared images of high-speed driving areas without vehicles as calibration infrared images; updating the calibration infrared images in real time at preset intervals; capturing images of vehicles in high-speed driving areas using high-definition camera sensors, and determining infrared video within a preset time period based on the capture time; performing a preliminary comparison between each frame of infrared images in the infrared video within the preset time period and the calibration infrared images, and delineating the change areas based on the comparison results; and determining the infrared image of the area with the largest pixel fluctuation as the infrared image corresponding to the high-speed vehicle.

[0068] Optionally, it also includes: determining the license plate number of the high-speed vehicle based on the high-definition image; naming the infrared image and the corresponding temperature data based on the license plate number and the capture time, and storing them on a local industrial control computer.

[0069] Optionally, the preset time period is 2 seconds, and infrared video within 2 seconds before and after the capture time is selected. The infrared images of each frame in the selected infrared video are compared with the calibrated infrared images.

[0070] In some embodiments, infrared images of the high-speed driving area traversed by the vehicle are calibrated to obtain calibrated infrared images, which are then normalized. The calibrated infrared images are then updated in real time at preset intervals, which are 10 minutes / time, 30 minutes / time, 60 minutes / time, or 1 day / time. This periodic updating of the calibrated images prevents the influence of surrounding interference, such as the changing shadows of gantries, nearby billboards, etc., and the movement of sunlight, thereby avoiding significant pixel deviations in the calibrated infrared images.

[0071] Optionally, before inputting the infrared image into the improved YOLOv9 recognition model, the process includes: rendering and coloring the temperature data corresponding to the infrared image to obtain rendered temperature data; and using the SIN function to map the rendered temperature data to obtain an updated infrared image.

[0072] Optionally, rendering and colorizing the temperature data corresponding to the infrared image includes:

[0073] Among them, T norm T represents the rendered temperature data, and T represents the temperature data before rendering. max For the maximum temperature data, T min This is the minimum temperature data.

[0074] Optionally, the SIN function is used to map the rendered temperature data, including:

[0075] T sin =sin(T) norm ); where T sin This refers to the mapped rendering temperature data, i.e., the updated temperature data.

[0076] In some embodiments, the temperature data is the original temperature matrix. Since the infrared image is obtained by rendering the original temperature matrix, it is a secondary processing of the original temperature matrix. Based on this, this scheme re-renders and colors the original temperature matrix corresponding to the infrared image, normalizing the temperature to... Within the specified range, the sine function is then applied to map the temperatures. By mapping the temperature matrix onto the coloring curve, image details are enhanced. Due to the varying slopes of the function, the rate of change is greater in the middle section of the curve, and the color changes are more pronounced between smaller temperature differences, thus making the contours in the infrared image clearer. The recolored vehicle compartment and other areas are more clearly distinguishable from the wheels, allowing for better segmentation of these areas.

[0077] Optionally, the infrared image is input into the improved YOLOv9 recognition model to obtain multiple segmented regions of the high-speed vehicle, including: inputting the infrared image into the improved YOLOv9 recognition model to obtain the tire region; and performing key region segmentation on the infrared image based on the tire region to obtain multiple segmented regions.

[0078] Optionally, the infrared image is segmented into key regions based on the tire area to obtain multiple segmented regions, including: excluding the headlights in the tire area to obtain the correct positions of at least two tires of the high-speed vehicle; determining the segmented region of the vehicle bottom area based on the upper and lower boundary areas of the two tires; determining the segmented region of the vehicle body area based on the more obvious texture features of the tires and the upper part of the vehicle body; and subtracting the area of ​​the vehicle bottom area and the vehicle body area from the complete vehicle area in the infrared image to obtain the remaining part as the vehicle front area.

[0079] Optionally, the complete vehicle region in the infrared image is obtained from the output of the improved YOLOv9 recognition model.

[0080] Optionally, excluding the headlights in the tire area, obtaining the correct position of the two tires of the high-speed vehicle includes: connecting each pair of candidate tires in the tire area with a line, intersecting each tire line with its nearest lane line, and obtaining the angle between the tire line and the lane line. If the angle is within a preset angle range, the two candidate tires forming the tire line are the tires of the high-speed vehicle; if the angle is not within the preset angle range, at least one of the two candidate tires forming the tire line is a headlight.

[0081] In some embodiments, the preset angle range is 0° to 30°. After detecting the wheel area, pairs of wheels are combined to obtain a straight line between them, which is then extended to intersect the nearest lane line to obtain the angle between them. If the angle is greater than 30°, at least one of the targets is considered to be a headlight; if it is less than 30°, both are considered to be tires. In this way, in nighttime scenes, the outline of truck headlights is highly similar to that of tires. This solution combines candidate tires in pairs and uses the lane line as a reference to eliminate areas where headlights are falsely detected, thus improving accuracy.

[0082] Optionally, the improved YOLOv9 recognition model includes: an input module that resizes the input infrared image and performs data augmentation on the resized infrared image to obtain an enhanced image; a feature extraction module that performs convolution processing on the enhanced image to obtain vehicle features; a feature fusion module that fuses the vehicle features through the SEMA module to obtain multi-scale features; and an output module that inputs the multi-scale features into three detection heads respectively and outputs the tire region of the high-speed vehicle through the detection heads.

[0083] Optionally, the improved YOLOv9 recognition model identifies the vehicle tire region, and based on the tire region, key regions including the vehicle underbody region, the cargo box region, and the front region are obtained. The improved YOLOv9 recognition model employs a CNN architecture based on invertible functions, designing PGI and auxiliary invertible branches. The improved YOLOv9 recognition model has fewer modules and channels, achieving lower resource consumption and faster response times. Compared to the lightweight YOLOv8 model, the improved YOLOv9 recognition model has higher parameter utilization and higher accuracy.

[0084] In some embodiments, the input module of the improved YOLOv9 recognition model uniformly adjusts the size of the input infrared image to 640×640, and then performs data augmentation on the adjusted image using operations such as rotation, scaling, cropping, and Mosaic enhancement to obtain an enhanced image. The feature extraction module extracts spatial and semantic features containing vehicles and their key regions from the enhanced image step by step through convolution. Then, the feature fusion module fuses, transforms, and enhances the extracted features through cross-scale connections and weighted fusion mechanisms, embedding SEMA attention during this process to improve the feature extraction capability for foreground targets. Finally, the output module outputs the features from the above multiple different scales to the head. Each detection head contains three branches: classification, object, and location. The classification and object branches use binary cross-entropy loss, while the regression branch combines DFL (Distance Focal Loss) loss and CIoU (Complete Intersection over Union) loss.

[0085] In some embodiments, the SEMA module is an improvement on the EMA module. The first branch of the EMA converges spatial information from feature maps from two directions before encoding, which can embed positional information into the channel attention map. However, further improvement is still needed in the interaction between global spatial locations. Therefore, this scheme designs a SEMA module based on the EMA (Efficient Multi-Scale Attention) attention module, using two parallel paths to extract attention weights for grouped feature maps. The SEMA module designs a first branch CAP (Coordinate Attention Path, CAP) and a second branch SAP (Spatial Attention and pool Path, SAP). The second branch extracts global spatial location information of vehicle features and uses a pooling pyramid to converge multi-scale information. Then, the cross-channel information of the two branch channels interacts, fusing contextual information of different scales to generate better pixel-level attention for the feature map, thereby capturing the dependencies between all channels and reducing computational cost.

[0086] Optionally, combined Figure 3 As shown, the SEMA module includes a first branch CAP and a second branch SAP. The first branch processes vehicle features to obtain the first branch result. After normalization, pooling, and activation processing of the first branch result, the first branch channel information is obtained. The second branch processes vehicle features to obtain the second branch result. After pooling and activation processing of the second branch result, the second branch channel information is obtained. The first and second branch channel information are interacted to obtain first and second interaction information, respectively. The first and second interaction information are fused, and the fused result is activated to obtain attention weights. The attention weights are used to weight vehicle features to obtain multi-scale features.

[0087] Optionally, combined Figure 4 As shown, the first branch CAP is used to encode vehicle features in the vertical and horizontal directions; the vertical and horizontal encodings are convolved simultaneously to obtain the first convolution result; the first convolution result is decomposed into two vectors; the two vectors are activated; the vehicle features are weighted using the two activation vectors to obtain the first branch result.

[0088] Alternatively, the formula for the first branch CAP is expressed as:

[0089]

[0090]

[0091] In the above formula, h and w represent the h-th row and w-th column of the vehicle feature map C×H×W, i represents the i-th column, c represents the c-th channel, x and y represent the input and output of each pixel, respectively, and σ is the activation function.

[0092] Optionally, combined Figure 5 As shown, the second branch SAP is used to calculate the maximum and average values ​​of vehicle features and construct a spatial graph; the spatial graph is concatenated and convolved to obtain global spatial attention weights; the vehicle features are weighted by the global spatial attention weights to obtain vehicle spatial weighted features; the vehicle spatial weighted features are aggregated by pooling pyramids to obtain the result of the second branch.

[0093] Alternatively, the SAP formula for the second branch is expressed as follows:

[0094] F s =F in ×Conv2d 7×7 [Max dim＝1 (F in Mean dim＝1 (F in)];

[0095]

[0096] In the above formula, F in ,F s ,F ch These represent the input features, the reweighted features, and the output features, respectively. Maxpool i This indicates that the input is subjected to max pooling i times consecutively, where Max and Mean represent taking the maximum value and average value, respectively.

[0097] Optionally, before inputting the infrared image into the improved YOLOv9 recognition model, the method further includes: training the initial improved YOLOv9 recognition model to obtain the improved YOLOv9 recognition model.

[0098] Optionally, training the improved YOLOv9 initial recognition model includes: obtaining a training set and a test set; setting the model parameters of the improved YOLOv9 initial recognition model; iteratively optimizing the model parameters of the improved YOLOv9 initial recognition model based on the training set; calculating the accuracy of the model using the test set; stopping the iteration when the accuracy meets the requirements; and obtaining the improved YOLOv9 recognition model.

[0099] In some embodiments, vehicle infrared image samples covering various scenarios, vehicle types, and other possible variations are collected. Conventional bounding box annotation software is difficult to apply to irregular vehicle images and infrared tire images, which are typically 640*640 pixels in size. This solution uses Labele software to annotate the collected vehicle infrared image samples by drawing circles and dots, marking the positions of the vehicle tires within the infrared image samples. The annotated vehicle infrared image sample data is then divided into training, testing, and validation sets in a ratio of 0.6:0.2:0.2.

[0100] In some embodiments, the model parameters of the improved YOLOv9 initial recognition model are set according to the recognition accuracy required by the cloud server and the edge device, and the learning rate is set to 0.01, the number of iterations is 100, the batch size is 16, and the optimizer used is SGD.

[0101] In some embodiments, the input image is standardized using proportional scaling combined with background filling, resulting in an input image size of 640*640*3. Feature information is extracted progressively from the input image. Bottom features contain rich location and contour information but lack semantic information, while top features extract high-level features that contain rich semantic information but lack location information, resulting in multi-scale feature maps of 80*80*256, 40*40*512, and 20*20*1024. A top-down and top-up approach is used to fuse, transform, and enhance the path aggregation network and the RepNCSPELAN4 module. This fuses feature maps of different scales output from the three stages, helping to aggregate shallow information into deep features. Finally, in the detection head, the detection and recognition of key vehicle regions at different scales are completed, resulting in segmented regions.

[0102] Optionally, it also includes evaluating the improved YOLOv9 recognition model.

[0103] In some embodiments, the evaluation metrics used in this scheme include precision, recall or retrieval rate, and mean precision (mAP), with mAP used as the primary evaluation metric. Detection speed is measured in FPS (frames per second). This scheme is compared with existing one-stage detection models YOLOv9s, YOLOv7, YOLOv8, SSD, and the two-stage algorithm Faster R-CNN. The comparison results are shown in Table 1. Table 1 shows that the mAP of this scheme reaches 95.2%, which is 5.8%, 7%, 2.9%, 13.6%, and 10.4% higher than the baseline models YOLOv9s, YOLOv7, YOLOv8, SSD, and Faster R-CNN, respectively. This significant improvement in mAP indicates that this scheme utilizes a scale-sensitive structure to fuse cross-scale features, improving the extraction capability of infrared vehicle features at different scales in highway scenes. The SEMA module enhances the feature extraction capability of foreground targets, thus improving the detection accuracy of key areas of overheated vehicles in highway scenes. In addition, the number of parameters and computational cost of this scheme is slightly higher than that of YOLOv9s, but much lower than that of other models. This is because the added SEMA attention brings a small amount of computation to the model, but it is almost negligible in practical application scenarios.

[0104] Table 1 Comparison of results between various detection algorithms and this scheme.

[0105] Method mAP@0.5 / % Params(M) FLOPs(G) YOLOv9s 89.4 1.763 4.1 SSD 81.6 23.5 30.4 Faster R-CNN 84.8 39.7 81.6 YOLOv7 88.2 4.613 11.2 YOLOv8 92.3 6.020 13.2 This plan 95.2 1.865 4.9

[0106] In some embodiments, this scheme compares different types of attention modules, such as Channel Attention (CA), Simimple (Parameter-Free Attention Module, SimAM), and EMA, by embedding them at the same location. The results are shown in Table 2. Table 2 shows that embedding most mainstream attention modules can improve detection accuracy. After adding the attention mechanism to the intermediate fusion node, the model can obtain richer channel and spatial information, thus improving the model's performance. Compared with CA, SimAM, and EMA attention modules, the model's mAP value is 95.2% after adding the SEMA module, which is 1.5% to 2.3% higher than other attention modules. SEMA improves the path based on EMA, can extract global spatial location information of grouped feature maps, and uses pooling pyramids to aggregate multi-scale information, effectively improving the detection accuracy of key areas of overheated vehicles.

[0107] Table 2 Comparison of detection performance with different attention embeddings

[0108]

[0109] Optionally, the temperatures of multiple key areas of the high-speed vehicle are obtained based on each segmented region and temperature data, including: determining the temperature of the key areas corresponding to each segmented region from the temperature data based on the positional correspondence between infrared images and temperature data.

[0110] In some embodiments, the temperature data is a full-screen temperature matrix, with a one-to-one correspondence between the infrared image and the full-screen temperature matrix. The temperature of the key area corresponding to the full-screen temperature matrix is ​​queried based on the segmented regions of the infrared image. For the temperature data corresponding to the bottom area of ​​the vehicle, the focus is primarily on the average temperature value, excluding areas with abnormal local temperatures, such as the exhaust emission devices and fuel tanks on the sides of the vehicle. For the cargo box area, the focus is primarily on the maximum temperature value to prevent the combustion of flammable materials in the cargo box. For the front area, the focus is primarily on the average temperature value, without observing the maximum temperature value, because the infrared thermal imaging detector is prone to some large interference points due to the influence of sunlight reflected from the windshield.

[0111] In some embodiments, the temperature data is a full-screen temperature matrix, with a one-to-one correspondence between the infrared image and the full-screen temperature matrix. The temperature of the key region corresponding to the full-screen temperature matrix is ​​queried based on the segmented regions of the infrared image. Since the target in the infrared image is relatively small, and considering that the temperature change is relatively small, the average temperature within the key region is considered as the temperature information of that key region.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

Claims

1. A method for monitoring temperature in key areas of a vehicle based on an improved YOLOv9, characterized in that, include: Monitor vehicles in high-speed driving areas and obtain vehicle monitoring data; Based on vehicle monitoring data, determine the infrared images corresponding to each high-speed vehicle, as well as the temperature data corresponding to the infrared images; Infrared images are input into the improved YOLOv9 recognition model to obtain multiple segmented regions of high-speed vehicles; The temperature of multiple key areas of a high-speed vehicle is obtained based on the segmented regions and temperature data. Improvements to the YOLOv9 recognition model include: The input module resizes the input infrared image and performs data enhancement processing on the resized infrared image to obtain an enhanced image. The feature extraction module performs convolution processing on the enhanced image to obtain vehicle features; The feature fusion module fuses vehicle features through the SEMA module to obtain multi-scale features; The output module inputs multi-scale features into three detection heads and outputs the segmented region of the high-speed vehicle through the detection heads; The SEMA module includes a first branch and a second branch. The first branch processes vehicle features to obtain the first branch result. After normalization, pooling, and activation processing of the first branch result, first branch channel information is obtained. The second branch processes vehicle features to obtain the second branch result. After pooling and activation processing of the second branch result, second branch channel information is obtained. The first and second branch channel information are interacted to obtain first and second interaction information, respectively. The first and second interaction information are fused, and the fused result is activated to obtain attention weights. The attention weights are then used to weight vehicle features to obtain multi-scale features. The first branch is used to encode vehicle features vertically and horizontally; the vertical and horizontal encodings are convolved simultaneously to obtain the first convolution result; the first convolution result is decomposed into two vectors; the two vectors are activated; the vehicle features are weighted using the two activation vectors to obtain the first branch result; The second branch is used to calculate the maximum and average values ​​of vehicle features and construct a spatial graph; the spatial graph is concatenated and convolved to obtain global spatial attention weights; the vehicle features are weighted using the global spatial attention weights to obtain vehicle spatial weighted features; the vehicle spatial weighted features are aggregated using a pooling pyramid to obtain the result of the second branch.

2. The method according to claim 1, characterized in that, Vehicles in high-speed driving areas are monitored to obtain vehicle monitoring data, including: In high-speed driving areas, high-definition cameras capture images of vehicles traveling at high speeds, radar speed detectors measure vehicle speeds, and infrared thermal imagers capture infrared video and temperature data of vehicles traveling at high speeds.

3. The method according to claim 2, characterized in that, Based on vehicle monitoring data, determine the infrared images corresponding to each high-speed vehicle, including: The license plate numbers of each high-speed vehicle were determined based on high-definition images of each vehicle traveling at high speed. Determine the speed of the high-speed vehicle at the moment of each high-definition image capture; The duration of the data transfer is determined based on the capture time and vehicle speed. The infrared images corresponding to each high-speed vehicle are determined from the infrared video based on the retrospective duration. The infrared images of each high-speed vehicle are named and stored according to the license plate number and the time of capture.

4. The method according to claim 2, characterized in that, Based on vehicle monitoring data, determine the infrared images corresponding to each high-speed vehicle, including: Infrared images of high-speed driving areas with no vehicles passing through are used as calibration infrared images; The calibrated infrared images are updated in real time according to a preset interval; vehicles in high-speed driving areas are captured by a high-definition checkpoint camera, and infrared video within a preset time period is determined based on the capture time. A preliminary comparison is made between each frame of infrared image in the infrared video within a preset time period and the calibrated infrared image, and the change area is delineated based on the comparison results. The infrared image containing the region of greatest pixel fluctuation is identified as the infrared image corresponding to the high-speed vehicle.

5. The method according to claim 1, characterized in that, Before inputting infrared images into the improved YOLOv9 recognition model, the following steps are included: The temperature data corresponding to the infrared image is rendered and colored to obtain the rendered temperature data. The SIN function is used to map the rendered temperature data to obtain an updated infrared image.

6. The method according to claim 1, characterized in that, Infrared images are input into an improved YOLOv9 recognition model to obtain multiple segmented regions of a high-speed vehicle, including: The infrared image is input into the improved YOLOv9 recognition model to obtain the tire area; The infrared image is segmented based on the tire area to obtain multiple segmented regions.

Images