Methods for detecting severe overloaded trucks based on infrared and lidar

Abstract

This invention discloses a method for detecting severe overloaded trucks based on infrared and lidar. Specifically, lidar is installed on the roadside of a highway to detect the truck's cargo box height (D) above the ground and its speed (V). Similarly, an infrared thermal imager and a camera are installed on the roadside to detect characteristic temperatures and identify the truck and its type, respectively. A processing terminal processes the sensor information and transmits it to the cloud. The camera uses the YOLO X deep learning algorithm to detect the truck and its type, while the infrared thermal imager detects the characteristic temperatures of the truck's engine and tires, as well as the ambient temperature. The processing terminal uses a hybrid strategy combining DBSCAN clustering and CatBoost to train the processed temperature data and the D and V data. Finally, based on the real-time temperature and D / V data, it determines whether the truck is severely overloaded and issues a warning for severely overloaded trucks. This method achieves preliminary detection of severe overloaded trucks on highways without contact, reducing safety hazards.

Description

Technical Field

[0001] This invention pertains to truck overload detection technology, specifically a method for detecting severe overload of trucks based on infrared and lidar. Background Technology

[0002] In recent years, my country's total expressway mileage has grown rapidly, playing a vital role in modern transportation due to its high speed and accessibility. However, the problem of overloaded trucks persists despite repeated bans, posing significant risks. On one hand, safety is compromised. Overloaded vehicles operate under excessive loads for extended periods, reducing braking and operational safety. Simultaneously, increased driver stress leads to more frequent and severe accidents. On the other hand, overloaded trucks cause significant damage to highway infrastructure and the transportation ecosystem. Among overloaded trucks, those exceeding 100% of their load limit or exceeding 100 tons pose the greatest danger.

[0003] Currently, most truck overload detection on highways still relies on manual methods, which are cumbersome and consume a lot of manpower. A few methods use dynamic weighing, but these are costly and require road resurfacing, making them difficult to maintain. Therefore, there is a need for a non-contact method with high detection accuracy for detecting severe truck overload. Summary of the Invention

[0004] To address the aforementioned technical deficiencies in the existing technology, this invention proposes a method for detecting severe overloading of trucks based on infrared and lidar.

[0005] The technical solution to achieve the objective of this invention is: a method for detecting severe overload of trucks based on infrared and lidar, comprising:

[0006] S1. The processing terminal receives the camera footage in real time. When a truck is detected in the footage, the truck model is identified, and the lidar and infrared thermal imager are controlled to start working.

[0007] S2. The infrared thermal imager detects the characteristic temperatures of the truck engine and tires, as well as the ambient temperature, and transmits this information to the processing terminal.

[0008] S3, Process terminal calculates relative characteristic temperature data;

[0009] S4. The lidar acquires the point cloud map of the truck, outputs the distance between the point cloud of the truck body and the point cloud of the ground, calculates the speed based on the displacement of the point cloud of the truck front within a set time, and transmits the speed to the processing terminal.

[0010] S5. Input the real-time detected and calculated temperature data, the distance between the truck compartment point cloud and the ground point cloud, and the speed data into the overload detection model of the corresponding vehicle type to determine whether the vehicle is seriously overloaded.

[0011] Preferably, the YOLO X algorithm is used to detect trucks and their models. The specific steps are as follows:

[0012] S1.1: Acquire video of trucks passing by through a camera, split the video into images frame by frame and save them in a folder to obtain several images of different vehicle types as a training set for object detection;

[0013] S1.2: Use annotation software to draw frames around the trucks in the image and label them with their vehicle types;

[0014] S1.3: Convert the XML format file generated by the annotation software into the JSON format file used by YOLO X;

[0015] S1.4: Modify the label count and label name parameters of the YOLO X algorithm to the actual values, and run the algorithm training section to train the dataset;

[0016] S1.5: Replace the pre-trained weight file with the weight file obtained from the training to obtain the truck model detection algorithm;

[0017] S1.6: Run the algorithm in step S1.5 to perform real-time truck detection and vehicle type recognition on the images captured by the camera.

[0018] Preferably, the engine and tire characteristic temperatures include the engine maximum temperature, the engine average temperature, the maximum temperature of all tires, and the average temperature of all tires.

[0019] Preferably, the YOLO X algorithm is used to detect the engine and tire regions on the infrared image, obtain the coordinate range of the regions, and use temperature data analysis software to obtain the characteristic temperature of the corresponding coordinate range.

[0020] Preferably, the difference between the characteristic temperature of the engine and tires and the ambient temperature is used as the characteristic value.

[0021] Preferably, a hybrid strategy combining the DBSCAN clustering algorithm and CatBoost is used to establish the overload detection model, and the steps are as follows:

[0022] S5.1: Obtain the distance, speed and characteristic temperature data of the truck body point cloud and the ground point cloud under different vehicle types and loads as a dataset, and set the target value label as "severe overload" and "non-severe overload";

[0023] S5.2: Use a hybrid strategy algorithm to train the dataset to obtain a binary classification model of severely overloaded trucks under different vehicle types;

[0024] S5.3: Based on the vehicle model detection results, call the binary classification model under that vehicle model to determine severe overloading.

[0025] Preferably, the specific steps for training the dataset using a hybrid strategy algorithm to obtain a binary classification model for severely overloaded trucks under different vehicle types are as follows:

[0026] S5.2.1: Use the DBSCAN algorithm to divide the non-severe overload class into k sub-clusters, and add labels 0, 1, ..., k-1 to each sub-cluster. Then, recombine the severe overload class with the sub-cluster into a new sample set, that is, the new sample set has k+1 categories.

[0027] S5.2.2 uses the CatBoost algorithm for multi-class classification. The formula for calculating the target statistic is as follows:

[0028]

[0029] In the formula, n represents the amount of data; For x i,k The generated numeric variable; x i,k Let x be the k-th sample category in the i-th class of data. j,k =x i,k At that time, [x j,k =x i,k The value is 1 if it is not 0 otherwise; Y j Let be the value of the j-th data label variable; P is the prior value, which is the mean of the target variable in the dataset; a is the weight of the prior values ​​that are greater than 0.

[0030] S5.2.3 Reclassify non-k-class labels as non-severe overload labels and k-class labels as severe overload labels, outputting a binary classification result, i.e., outputting whether the truck is severely overloaded.

[0031] Preferably, the value of k is selected according to the following formula:

[0032]

[0033] In the formula, N0 represents the majority class sample, N1 represents the minority class sample, and ceil represents the floor function.

[0034] Preferably, severe overloading is defined as the total weight of the truck exceeding 100% of its load limit or the total weight of the truck exceeding 100 tons.

[0035] Compared with the prior art, the significant advantages of this invention are:

[0036] (1) The non-contact method is used to detect and warn of severely overloaded trucks, which ensures traffic efficiency, is highly maintainable, and does not require road repaving.

[0037] (2) In vehicle model recognition, the YOLO X deep learning algorithm has higher accuracy than other detection algorithms in this project;

[0038] (3) From the perspective of sensor technology integration, it is more accurate than the traditional single overloaded truck detection technology and has a better application prospect.

[0039] (4) It can be integrated with other sensors to improve detection accuracy, and has strong inclusiveness.

[0040] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0041] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0042] Figure 1 This is a flowchart of the process of the present invention.

[0043] Figure 2 This is a diagram showing the placement of the equipment. Detailed Implementation

[0044] It is readily understood that, based on the technical solution of this invention, various embodiments of the invention can be conceived by those skilled in the art without altering the essential spirit of the invention. Therefore, the following detailed embodiments and accompanying drawings are merely illustrative examples of the technical solution of this invention and should not be considered as the entirety of the invention or as limitations or restrictions on the technical solution of this invention. Rather, these embodiments are provided to enable those skilled in the art to gain a more thorough understanding of the invention. Preferred embodiments of the invention are described below in conjunction with the accompanying drawings, which form part of this application and, together with the embodiments of the invention, serve to illustrate the innovative concept of the invention.

[0045] The present invention is conceived as a method for detecting severe overloading of trucks based on infrared and lidar, the specific steps of which are as follows:

[0046] S1. The processing terminal receives camera footage in real time. When a truck is detected in the footage, it identifies the vehicle type and controls the lidar and infrared thermal imager to start working.

[0047] Specifically, the camera used has night vision capability; the infrared thermal imager used has a temperature measurement error of ±2℃ or 2%, and the accompanying analysis software can correct the temperature data based on air temperature, humidity, visibility, distance, etc., to avoid the influence of environmental factors on the measured temperature data; the lidar used is centimeter-level with an error of ±2cm, is not affected by sunlight, and each sensor has a wireless transmission module.

[0048] In further implementation, the YOLO X algorithm is used to detect trucks and their models, as detailed below:

[0049] S1.1: Install cameras on the roadside in advance to capture videos of trucks passing by, split the videos into images frame by frame and save them in a folder, and obtain several images of different vehicle types as a training set for object detection;

[0050] S1.2: Use Labelimg annotation software to draw frames around the trucks in the image and label them with their vehicle types;

[0051] S1.3: Convert the XML format file generated by the Labelimg annotation software into the JSON format file used by YOLO X;

[0052] S1.4: Modify the YOLO X algorithm's parameters such as the number of labels and label names to their actual values, and run the algorithm's training section to train the dataset;

[0053] S1.5: Replace the pre-trained weight file Yolov5s.pt with the weight file obtained from the training to obtain the truck model detection algorithm;

[0054] S1.6: Run the algorithm in step S1.5 to perform real-time truck detection and vehicle type recognition on the images captured by the camera.

[0055] S2, the infrared thermal imager detects the characteristic temperature of the truck engine and tires, as well as the ambient temperature, and transmits this information to the processing terminal.

[0056] Specifically, the infrared thermal imager has a temperature measurement error of ±2℃ or 2%, and the accompanying analysis software can correct the temperature data based on air temperature, humidity, visibility, distance, etc., to avoid the influence of environmental factors on the measured temperature data, and it also has a wireless transmission module.

[0057] Specifically, engine and tire characteristic temperatures include engine maximum temperature, engine average temperature, maximum temperature of all tires, and average temperature of all tires.

[0058] Specifically, the YOLO X algorithm is used to detect the engine and tire regions on the infrared image, obtain the coordinate range of the regions, and then use temperature data analysis software to obtain the characteristic temperature of the corresponding coordinate range.

[0059] S3, Processing terminal calculates relative characteristic temperature data.

[0060] Specifically, to minimize the influence of ambient temperature, the difference between the ambient temperature and the characteristic value is used.

[0061] S4. The lidar acquires the point cloud map of the truck, outputs the distance D between the truck body point cloud and the ground point cloud, calculates the velocity V based on the displacement of the truck front point cloud within 1 second, and transmits it to the processing terminal.

[0062] Specifically, the lidar used has a ranging accuracy of centimeters with an error of ±2cm, is unaffected by sunlight, and has a wireless transmission module.

[0063] S5. Input the temperature data and D and V data obtained from real-time detection and calculation into the overload detection model of this vehicle model to determine whether the vehicle is seriously overloaded.

[0064] Specifically, severe overloading is defined as: a truck whose total weight exceeds its load limit by 100% or whose total weight exceeds 100 tons. Such trucks pose a great safety hazard and can cause serious damage to roads, and are key vehicles managed by various traffic management departments.

[0065] In a further embodiment, a hybrid strategy combining the DBSCAN clustering algorithm and CatBoost is used to establish a severe overload detection model, and the steps are as follows:

[0066] S5.1: A large amount of D, V and characteristic temperature data under different vehicle models and loads were obtained through experiments as a dataset, and the target value labels were located as "severe overload" and "non-severe overload".

[0067] S5.2: Use a hybrid strategy algorithm to train the dataset to obtain a binary classification model of severely overloaded trucks under different vehicle types;

[0068] The specific hybrid strategy algorithm used is as follows:

[0069] S5.2.1: Use the DBSCAN algorithm to divide the "non-severe overload" class into k subclusters, and add labels (0, 1, ..., k-1) to each subcluster. Then, recombine these subclusters with the "severe overload" class to form a new sample set, i.e., the new sample set has k+1 categories. The value of k is selected according to the following formula:

[0070]

[0071] In the formula, N0 represents the majority class sample, N1 represents the minority class sample, and ceil represents the floor function.

[0072] S5.2.2 uses the CatBoost algorithm for multi-class classification. CatBoost is a novel gradient-enhanced decision tree algorithm that reduces the impact of noise and low-frequency categorical data on the data distribution by adding prior distribution terms. The formula for calculating its target statistic is as follows:

[0073]

[0074] In the formula, n represents the amount of data; For x i,k The generated numeric variable; x i,k Let x be the k-th sample category in the i-th class of data. j,k =x i,k At that time, [x j,k =x i,k The value is 1 if it is not 0 otherwise; Y j Let be the value of the j-th data label variable; P is the prior value, which is the mean of the target variable in the dataset; a is the weight of the prior values ​​that are greater than 0.

[0075] S5.2.3 Reclassify non-k-class labels as "non-severe overloading" labels and k-class labels as "severe overloading" labels, outputting the binary classification result, i.e., outputting whether the truck is severely overloaded.

[0076] This strategy transforms the binary classification problem into a multi-class classification problem by using the DBSCAN algorithm to balance the dataset. Then, the CatBoost classification algorithm is used to effectively classify imbalanced samples, i.e., the insufficient number of severely overloaded samples, thus effectively eliminating the impact of imbalanced samples in the truck overload experiment data on classification accuracy.

[0077] S5.3: Based on the vehicle model detection results, call the binary classification model under that vehicle model to determine severe overloading.

[0078] If a truck is found to be severely overloaded, the processing terminal will transmit the information to the cloud, prompting inspectors to conduct a second inspection of the truck.

[0079] The definition of severe overloading is: a truck whose total weight exceeds its load limit by 100% or whose total weight exceeds 100 tons. Such trucks pose a great safety hazard and can cause serious damage to roads. They are vehicles that are subject to key management by various traffic management departments.

[0080] This invention can effectively avoid the problems of high cost, low accuracy and complicated process of existing detection methods, and has good detection effect on different types of trucks.

[0081] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

[0082] It should be understood that, in order to simplify the present invention and help those skilled in the art understand its various aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes described in a single embodiment or with reference to a single figure. However, the present invention should not be construed as including all features in the exemplary embodiments as essential technical features of the claims of this patent.

[0083] It should be understood that the modules, units, components, etc., included in the device of one embodiment of the present invention can be adaptively changed to be placed in a device different from that embodiment. Different modules, units, or components included in the device of the embodiment can be combined into a single module, unit, or component, or they can be divided into multiple sub-modules, sub-units, or sub-components.

Claims

1. A method for detecting severe overloading of trucks based on infrared and lidar, characterized in that, include: S1. The processing terminal receives the camera footage in real time. When a truck is detected in the footage, the truck model is identified, and the lidar and infrared thermal imager are controlled to start working. S2. The infrared thermal imager detects the characteristic temperatures of the truck engine and tires, as well as the ambient temperature, and transmits this information to the processing terminal. S3, Process terminal calculates relative characteristic temperature data; S4. The lidar acquires the point cloud map of the truck, outputs the distance between the point cloud of the truck body and the point cloud of the ground, calculates the speed based on the displacement of the point cloud of the truck front within a set time, and transmits the speed to the processing terminal. S5. Input the real-time detected and calculated temperature data, the distance between the truck bed point cloud and the ground point cloud, and the speed data into the overload detection model of the corresponding vehicle type to determine whether the vehicle is severely overloaded. The steps are as follows: S5.1: Obtain the distance, velocity, and characteristic temperature data of the truck body point cloud and the ground point cloud under different vehicle types and loads as a dataset, and set the target value labels as "severe overload" and "non-severe overload"; S5.2: Use a hybrid strategy algorithm to train the dataset to obtain a binary classification model for severely overloaded trucks under different vehicle types. The specific steps are as follows: S5.2.1: Use the DBSCAN algorithm to divide the non-severe overload class into k sub-clusters, and add labels 0, 1, ..., k-1 to each sub-cluster. Then, recombine the severe overload class with the sub-cluster into a new sample set, that is, the new sample set has k+1 categories. S5.2.2 uses the CatBoost algorithm for multi-class classification. The formula for calculating the target statistic is as follows: In the formula, n is the amount of data; For x i,k The generated numeric variable; x i,k Let x be the k-th sample category in the i-th class of data. j,k =x i,k At that time, [x j,k =x i,k The value is 1 if it is not 0 otherwise; Y j Let be the value of the j-th data label variable; P is the prior value, which is the mean of the target variable in the dataset; a is the weight of the prior values ​​that are greater than 0. The value of k is selected according to the following formula: In the formula, N0 represents the majority class samples, N1 represents the minority class samples, and ceil represents the floor function; S5.2.3 Reclassify non-k-class labels into non-severe overload labels and k-class labels into severe overload labels, outputting a binary classification result, i.e., outputting whether the truck is severely overloaded; S5.3: Based on the vehicle model detection results, call the binary classification model under that vehicle model to determine severe overloading.

2. The method for detecting severe overloaded trucks based on infrared and lidar according to claim 1, characterized in that, The YOLO X algorithm is used to detect trucks and their models. The specific steps are as follows: S1.1: Acquire video of trucks passing by through a camera, split the video into images frame by frame and save them in a folder to obtain several images of different vehicle types as a training set for object detection; S1.2: Use annotation software to draw frames around the trucks in the image and label them with their vehicle types; S1.3: Convert the XML format file generated by the annotation software into the JSON format file used by YOLO X; S1.4: Modify the label count and label name parameters of the YOLO X algorithm to the actual values, and run the algorithm training section to train the dataset; S1.5: Replace the pre-trained weight file with the weight file obtained from the training to obtain the truck model detection algorithm; S1.6: Run the algorithm in step S1.5 to perform real-time truck detection and vehicle type recognition on the images captured by the camera.

3. The method for detecting severe overloaded trucks based on infrared and lidar according to claim 1, characterized in that, Engine and tire characteristic temperatures include engine maximum temperature, engine average temperature, maximum temperature of all tires, and average temperature of all tires.

4. The method for detecting severe overloaded trucks based on infrared and lidar according to claim 1, characterized in that, The YOLO X algorithm was used to detect the engine and tire regions in the infrared image, and the coordinate range of the regions was obtained. Temperature data analysis software was then used to obtain the characteristic temperature of the corresponding coordinate range.

5. The method for detecting severe overloaded trucks based on infrared and lidar according to claim 1, characterized in that, The difference between the characteristic temperature of the engine and tires and the ambient temperature is used as the characteristic value.

6. The method for detecting severe overloaded trucks based on infrared and lidar according to claim 1, characterized in that, The term "severe overloading" is defined as a truck's total weight exceeding its load limit by 100% or the truck's total weight exceeding 100 tons.

Images