Intelligent empty van compartment identification and auditing method based on video stream features

Abstract

The application relates to the technical field of image recognition, and provides an intelligent truck empty compartment recognition and auditing method based on video stream features, which comprises five modules of video stream image acquisition, vehicle target detection and tracking, vehicle motion state monitoring, vehicle compartment state recognition and vehicle compartment state reporting, and the main processes are video stream image acquisition, vehicle target detection and tracking, vehicle motion state detection, vehicle compartment state recognition and vehicle compartment state reporting; the vehicle compartment state is recognized through an MCN network, the representation ability and the discrimination of the compartment features are improved, the IBN-Net structure is used to effectively solve the influence of environmental changes such as illumination and color on vehicle feature extraction, the generalization ability of the model between different domains is ensured, the SENet structure is used to guide the model to automatically learn the importance of the channel dimension of the feature map, and the perception ability of the model to the channel features is enhanced.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and more specifically to an intelligent method for identifying and verifying empty cargo compartments in freight trucks based on video stream features, which improves the accuracy and efficiency of empty cargo compartment identification. Background Technology

[0002] With the rapid development of my country's economy, the volume of my country's import and export trade has reached new highs, making it the world's largest trading nation. Every year, a large number of containers enter the country. In order to effectively ensure the safety of the import and export environment, the demand for container empty container inspection and identification is increasing day by day. Empty container inspection is the detection of whether the container is empty.

[0003] Currently, there are three main methods for empty container inspection: The first method involves manual inspection of containers upon entry and exit, based on customs declarations. This method requires dedicated personnel to intervene at the checkpoint, and manual inspection can take anywhere from four to five minutes, severely impacting the efficiency of goods entering and leaving the country. It suffers from low operational efficiency, a large amount of mechanical and ineffective inspection that easily becomes a mere formality, and potential corruption risks. The second method uses ultrasonic instruments developed by domestic manufacturers to inspect empty containers. The principle is to compare the ultrasonic characteristics of pre-collected empty containers with the measured ultrasonic characteristics of actual containers to determine whether they are empty. While the second method can improve upon the inefficiency and inability to conduct comprehensive inspections inherent in manual inspection, it still has the following drawbacks:

[0004] First, ultrasonic generators and sonar must scan the entire container, which makes vehicle inspection and customs clearance time-consuming.

[0005] Second, the varying thickness of the container material significantly affects the calibration and acquisition of ultrasonic echo characteristics, making accurate judgment impossible.

[0006] Third, the ultrasonic interference generated by checkpoint vehicles and other equipment during operation affects the equipment's ability to determine empty containers, resulting in low accuracy in practical applications.

[0007] The third method is to use a large X-ray machine for scanning. X-ray scanning requires manual image recognition to determine whether the container is empty. However, X-ray scanning is a non-intelligent processing method, and radiation is generated during the inspection process, which can cause serious physical harm to the driver and surrounding staff who pass the inspection. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention provides an intelligent truck empty compartment identification and verification method based on video stream features. By utilizing video stream images acquired by a camera, real-time intelligent analysis of the vehicle compartment is achieved. The analysis system and the monitoring system operate together to solve the vehicle compartment identification problems currently existing in the background technology.

[0009] The specific technical solution of this invention is as follows:

[0010] An intelligent method for identifying and verifying empty cargo compartments in freight trucks based on video stream features includes the following steps:

[0011] S101: Acquire video stream images. The front-end capture device acquires the video stream in real time and uploads it to the system server. After receiving the uploaded real-time video stream information, the corresponding frame image will be sent to the relevant processing module for subsequent intelligent truck empty compartment recognition.

[0012] S102: Vehicle target detection and tracking. After receiving the image to be processed in S101, vehicle target detection is performed based on the Yolov5 algorithm. After confirming the vehicle position information, the DeepSort algorithm is used to determine the motion trajectory of each vehicle target.

[0013] S103: Vehicle motion status monitoring. The determination of vehicle motion status is the prerequisite for subsequent vehicle body recognition. For vehicles exiting the checkpoint, after obtaining the specific position of the target vehicle on each input frame image and the motion trajectory of consecutive frames through S102, it is necessary to determine the motion status of the vehicle. The detection of vehicle motion status is based on the relative motion distance of vehicles in adjacent frames.

[0014] S104: Vehicle compartment status recognition. The vehicle compartment status recognition is the core link of the truck empty compartment recognition and verification method. The input of this link is the paused image of the vehicle compartment, and the output is the vehicle compartment recognition status. The vehicle compartment status recognition network MCN is a dual-structure classification network, which is built on SENet and IBN-Net networks. Specifically, it involves four parts: image input, data preprocessing, image feature extraction and feature classification.

[0015] S105: Vehicle compartment status reporting. The reporting of the compartment status is completed by the real-time early warning module. The core of this module is to record the identification status of each vehicle compartment by establishing a compartment status identification result table.

[0016] As a further improvement to this scheme, in step S102, the vehicle target detection process is executed based on the Yolov5 algorithm. By combining the ROI coordinate information corresponding to each camera, the target vehicles in the region of interest are obtained, and then the detected vehicle targets are filtered to retain only the vehicles in the ROI region, so as to reduce the interference of irrelevant targets.

[0017] As a further improvement to this scheme, in step S102, the motion trajectory of each vehicle target is determined based on the DeepSort algorithm. In the motion information association stage of different vehicle targets, deep vehicle re-identification features are introduced. These features are extracted by the offline-trained SCAN vehicle re-identification network. SCAN is a classic vehicle re-identification algorithm that innovatively proposes a cross-segmentation mechanism to ensure effective and complete representation learning of vehicle targets under multiple perspectives and scales. It also proposes a variety of side information fusion application strategies, which have achieved state-of-the-art (SOTA) results on multiple public vehicle re-identification datasets. The path information of the vehicle is updated through the dual association of motion information and deep representation features.

[0018] As a further improvement to this scheme, in step S103, when the vehicle is paused, the vehicle empty compartment recognition process is executed. The detection of the vehicle's motion state is based on the relative motion distance of the vehicles in adjacent frames. When the motion distance of the vehicles in adjacent frames is greater than the set distance threshold, the vehicle is considered to be in motion; otherwise, the vehicle is in a paused state. The distance threshold is set to T1.

[0019] As a further improvement to this scheme, in step S103, the vehicle's movement distance is equal to the distance between the center points of the main diagonal of the vehicle target position box in adjacent frames, specifically calculated as follows:

[0020]

[0021]

[0022]

[0023]

[0024] In equations 1 and 2, P t P t+1 Equation 1 represents the coordinates of the center point of the main diagonal of the target vehicle's position box in frames t and t+1, respectively. Here, 0 represents the starting point coordinate of the main diagonal of the vehicle's position box, and 3 represents the ending point coordinate of the main diagonal of the vehicle's position box. x and y correspond to the horizontal and vertical coordinates, respectively. In Equation 3, Dist represents the movement distance of the same vehicle in two adjacent frames. In Equation 4, the vehicle_state variable represents the vehicle's motion state. When it is equal to 0, it means that the vehicle is stopped, and when it is equal to 1, it means that the vehicle is in motion.

[0025] As a further improvement to this solution, in step S104, the input for identifying the empty cargo compartment of a freight truck is the image of the cargo compartment after the truck has been paused. For the target vehicle after the truck has been paused, the cargo compartment status can be divided into three types: no door, empty compartment and non-empty compartment. No door means that the cargo compartment door has not been opened after the vehicle has been paused. Empty compartment means that the cargo compartment is identified as empty. Non-empty compartment means that the cargo compartment is identified as non-empty.

[0026] As a further improvement to this scheme, in step S104, the core of the SENet network is to guide the model to focus on the differences in channel dimensions and automatically learn the nonlinear relationships between different channel features in order to improve the model's attention to important channel features. The core structure of the SENet network consists of two operations: Squeeze and Execution. The Squeeze module extracts global features on the channels through the Global Pooling operation, and then the Execution module obtains the importance weight parameters of each channel through FC, ReLU and Sigmoid operations. Finally, the weights are mapped to the features of each channel, thereby improving the model's attention to important channel features.

[0027] As a further improvement to this scheme, in step S104, the I BN-net network is used to improve the model's adaptability to changes in image appearance. Because the training set of the model cannot completely cover the data in all scenarios, the use of deep models may face the situation that there are differences between the training domain and the test domain. If the difference between the two domains is too large, it will lead to a deterioration in the actual application effect of the model.

[0028] As a further improvement to this solution, in step S105, a real-time warning threshold and a door-not-opened timeout warning threshold are set in the real-time warning module. The real-time warning threshold represents the number of consecutive corresponding state frames that confirm an empty or non-empty compartment, while the timeout warning threshold represents the number of consecutive door-not-opened state frames that confirm a door-not-opened result. The real-time warning threshold is denoted as T2, and the timeout warning threshold is denoted as T3.

[0029] As a further improvement to this scheme, in step S105, if the number of consecutive frames of the unopened state is greater than the timeout warning threshold, the corresponding carriage identification result is reported as unopened timeout; if the number of consecutive frames of the empty carriage state or the non-empty carriage state is greater than the real-time warning threshold, the empty carriage or non-empty carriage result is reported.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] 1. This invention introduces deep vehicle re-identification features at different stages of motion information association for vehicle targets, and establishes a deep association metric for judgment. These features are extracted by the offline-trained SCAN vehicle re-identification network. SCAN is a classic vehicle re-identification algorithm that solves the target loss problem in the vehicle tracking process and effectively improves the accuracy of vehicle tracking.

[0032] 2. This invention significantly improves the representation and discrimination capabilities of vehicle compartment features by adopting the design of MCN twin networks. It uses the IBN-Net structure to effectively solve the impact of environmental changes such as illumination and color on vehicle feature extraction, ensuring the model's generalization ability across different domains. At the same time, the SENet structure guides the model to automatically learn the importance of feature map channel dimensions, enhancing the model's ability to perceive channel features.

[0033] 3. This invention establishes a carriage status recognition result table through a real-time early warning module to record the recognition status of each vehicle carriage, constructs a carriage status recognition result statistics and monitoring mechanism, improves the accuracy of reported results through the real-time early warning module, effectively improves the accuracy and efficiency of vehicle empty carriage recognition, greatly saves manpower and other resource costs, and eliminates the need for X-ray scanning, thus improving the safety of staff. Attached Figure Description

[0034] Figure 1 This is a flowchart of the steps of the carriage identification and verification method of the present invention;

[0035] Figure 2 This is a flowchart of the MCN network identification process in the carriage identification and verification method of the present invention;

[0036] Figure 3 This is a flowchart of the SENet Model for identifying and verifying vehicle compartments according to the present invention.

[0037] Figure 4 This is a flowchart of the operation of the IBN-Net Model for identifying and verifying carriages according to the present invention;

[0038] Figure 5 This is a diagram illustrating the operation of the real-time early warning module in the carriage identification and verification method of this invention.

[0039] Figure 6 This is a flowchart illustrating the implementation process of the carriage identification and verification method of the present invention. Detailed Implementation

[0040] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and should not be construed as limiting the scope of the invention.

[0041] In this embodiment, as Figure 1-6As shown, an intelligent truck empty cargo compartment identification and verification method based on video stream features is presented. This method mainly comprises five modules: video stream image acquisition, vehicle target detection and tracking, vehicle motion state monitoring, vehicle cargo compartment status identification, and vehicle cargo compartment status reporting. The specific steps are as follows:

[0042] Step 1: Acquire video stream images

[0043] like Figure 1 As shown, the system first acquires the video stream in real time and uploads it to the system server. Then, it uses the real-time streaming image data captured by the target camera to access the subsequent processing flow.

[0044] Step 2: Vehicle Target Detection and Tracking

[0045] After receiving the real-time streaming image data, the vehicle target detection process starts working, detecting vehicle targets in the input image and saving the vehicle's position information, namely the four coordinate points of the vehicle target box: the upper left corner, the lower left corner, the lower right corner, and the upper right corner.

[0046] The specific vehicle target detection and tracking process is as follows: after receiving the image to be processed, the vehicle target detection process is first executed based on the Yolov5 algorithm. Then, the target vehicles in the region of interest are obtained by combining the ROI coordinate information of each camera. The detected vehicle targets are filtered to retain only the vehicles in the ROI region, so as to reduce the interference of irrelevant targets and save machine resources and improve the efficiency of subsequent processing.

[0047] It should be noted that setting a region of interest can effectively avoid interference from objects outside the region; and setting a region of interest can help save machine resources and improve the efficiency of subsequent processing.

[0048] Simultaneously, after obtaining the location information of the target vehicles, the DeepSort algorithm is used to determine the motion trajectory of each vehicle target. In the motion information association stage of different vehicle targets, deep vehicle re-identification features are introduced. These features are extracted by the offline-trained SCAN vehicle re-identification network. SCAN is a classic vehicle re-identification algorithm that innovatively proposes a cross-segmentation mechanism to ensure effective and complete representation learning of vehicle targets under multiple perspectives and scales. It also proposes a variety of side information fusion application strategies, achieving state-of-the-art (SOTA) results on multiple public vehicle re-identification datasets. The vehicle path information is updated through the dual association of motion information and deep representation features.

[0049] like Figure 6As shown, by combining the vehicle target location information detected in the input image, a tracking vehicle target is created. The Mahalanobis distance between the predicted motion state of the vehicle and the target detection result is calculated based on the Kalman filter algorithm to complete the association of vehicle motion information. In addition, the vehicle appearance information is associated based on the deep vehicle representation features obtained by the vehicle re-identification network. Finally, the association results of motion information and vehicle depth features are combined to determine the assignment ID of the vehicle in the next frame, thus completing the establishment of the vehicle motion trajectory.

[0050] Step 3: Vehicle Motion Status Detection

[0051] Determining the vehicle's motion status is the prerequisite for executing the subsequent vehicle empty compartment recognition module. For vehicles exiting the checkpoint, after obtaining the target vehicle's specific position on each input frame image and the motion trajectory of consecutive frames in the second step, it is necessary to determine the vehicle's motion status. The vehicle empty compartment recognition process can only be executed after the vehicle stops.

[0052] The detection of vehicle motion status is based on the relative motion distance of vehicles in adjacent frames. After the motion trajectory of the same vehicle is obtained, the motion distance of the vehicle in two adjacent frames is calculated, that is, the Euclidean distance between the center points of the main diagonal of the vehicle target box in two frames. Then it is compared with the known distance threshold T1 to obtain and record the vehicle status information, that is, whether the vehicle is paused or moving.

[0053] The vehicle is considered to be in motion when the distance traveled by the vehicle in an adjacent frame is greater than the set distance threshold T1; otherwise, the vehicle is in a paused state. Here, the distance traveled by the vehicle is equal to the distance between the center points of the main diagonal lines of the vehicle's target location box in adjacent frames. The specific calculation is as follows:

[0054]

[0055]

[0056]

[0057]

[0058] In equations 1 and 2, P t P t+1 Equation 1 represents the coordinates of the center point of the main diagonal of the target vehicle's position box in frames t and t+1, respectively. Here, 0 represents the starting point coordinate of the main diagonal of the vehicle's position box, and 3 represents the ending point coordinate of the main diagonal of the vehicle's position box. x and y correspond to the horizontal and vertical coordinates, respectively. In Equation 3, Dist represents the movement distance of the same vehicle in two adjacent frames. In Equation 4, the vehicle_state variable represents the vehicle's motion state. When it is equal to 0, it means that the vehicle is stopped, and when it is equal to 1, it means that the vehicle is in motion.

[0059] Step 4: Vehicle Body Status Recognition

[0060] 4.1 The cargo compartment status recognition is the core link of the truck empty compartment recognition and verification method. The input of this link is the cargo compartment image after the vehicle has been paused, and the output is the cargo compartment recognition status. For the vehicle target after the vehicle has been paused, the cargo compartment status can be divided into three types: door not open, empty compartment and not empty compartment. Among them, door not open means that the cargo compartment door has not been opened after the vehicle has been paused, empty compartment means that the vehicle compartment is recognized as empty compartment, and not empty compartment means that the vehicle compartment is recognized as not empty compartment.

[0061] 4.2 After the vehicle stops, the image of the passenger compartment is extracted and input into the passenger compartment status recognition network (MCN). The passenger compartment status recognition network (MCN) is a dual-branch classification network. The specific process of the MCN network is as follows: Figure 2 As shown, it is built upon SENet and IBN-Net networks. The core structures of SENet and IBN-Net in the MCN network are as follows: Figure 3 , Figure 4 As shown, the model involves four parts: image input, data preprocessing, image feature extraction, and feature classification. The image input module receives images of the train carriages. To improve the model's performance and generalization ability, the data preprocessing module performs size transformation, random horizontal flipping, color dithering, image filling, and random cropping operations on the input train carriage images. The image feature extraction part extracts the depth feature vectors corresponding to the input images based on branch networks of different paths. Under the synergistic effect of SE and IBN branches, the final train carriage features f are obtained. Finally, the feature classification layer obtains the train carriage state classification result corresponding to the input train carriage image.

[0062] 4.3 Typically, CNN network optimization mainly involves increasing the receptive field size of convolutional operations or establishing multi-scale mechanisms to encourage the model to extract more spatial information, thereby improving its performance. In contrast, convolutional operations generally fuse all features across the channel dimension by default, lacking awareness and extraction of the importance of different channels. The core of the SENet network in this solution is to guide the model to focus on the differences in the channel dimension and automatically learn the nonlinear relationships between features of different channels. The core structure of the SENet network consists of two operations: Squeeze and Excitation. The Squeeze module extracts global features across channels through Global Pooling, while the Excitation module obtains the importance weight parameters of each channel through FC, ReLU, and Sigmoid operations. Finally, the weights are mapped to the features of each channel, thereby improving the model's focus on important channel features.

[0063] 4.4 The input images of deep neural networks are captured by actual devices. Therefore, differences in lighting, color, style, and other factors in different scenes will cause the image appearance to vary. In addition, the training set of the model cannot completely cover the data in all scenes. All of these factors may cause the use of deep models to face the situation where there is a difference between the training domain and the test domain. If the difference between the two domains is too large, it will lead to a deterioration in the actual application performance of the model. In summary, how to improve the model's ability to adapt to changes in image appearance has become one of the keys to the actual performance of the model. IBN-net is designed to solve this problem.

[0064] Shallow feature differences in a model often manifest as differences in the appearance of an image, while deep feature differences manifest as differences in the content of the image. Introspection (IN) is typically used for low-level visual tasks because it can filter out complex differences in image appearance, such as image stylization. Batch normalization (BN), on the other hand, is used for high-level visual tasks, such as object detection and image recognition. IBN-Net innovatively combines the two, using IN and BN together to address shallow feature differences and BN to address deep feature differences, thereby improving the model's learning and generalization abilities.

[0065] 4.5. The MCN dual-sub-network for empty compartment recognition obtains the empty compartment recognition result (door not open, empty compartment, or not empty compartment), and establishes and updates the vehicle compartment status recognition result table, inserting the recognition result into the corresponding vehicle ID in the recognition result table. An example of the compartment status recognition result table is shown below:

[0066]

[0067] Step 5: Reporting the status of the vehicle body

[0068] The number of times the "unopened door," "empty compartment," and "non-empty compartment" categories are identified in the compartment status recognition result table is counted. The category with the highest number of identifications is considered the empty compartment identification result for that vehicle, and then the result is reported.

[0069] The specific process is as follows: Figure 5 As shown: The reporting of the carriage status is completed by the real-time early warning module. The core of this module is to record the identification status of each vehicle carriage by establishing a carriage status identification result table. At the same time, two parameters are set: the real-time early warning threshold T2 and the door not opened timeout early warning threshold T3. The real-time early warning threshold represents the number of consecutive corresponding status frames that can confirm the result of an empty or non-empty carriage, while the timeout early warning threshold represents the number of consecutive door not opened status frames that can confirm the result of a door not opened.

[0070] The embodiments of the present invention are given for the purposes of illustration and description, and are not intended to be exhaustive or to limit the invention to the forms disclosed. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features.

Claims

1. An intelligent method for identifying and verifying empty cargo compartments in freight trucks based on video stream features, characterized in that, Includes the following steps: S101: Acquire video stream images. The front-end capture device acquires the video stream in real time and uploads it to the system server. After receiving the uploaded real-time video stream information, the corresponding frame image will be sent to the relevant processing module for subsequent intelligent truck empty compartment recognition. S102: Vehicle target detection and tracking. After receiving the image to be processed in S101, vehicle target detection is performed based on the Yolov5 algorithm. After confirming the vehicle position information, the DeepSort algorithm is used to determine the motion trajectory of each vehicle target. In step S102, the motion trajectory of each vehicle target is determined based on the DeepSort algorithm. In the motion information association stage of different vehicle targets, deep vehicle re-identification features are introduced. These features are extracted by the offline-trained SCAN vehicle re-identification network. S103: Vehicle motion status monitoring. The determination of vehicle motion status is the prerequisite for subsequent vehicle body recognition. For vehicles exiting the checkpoint, after obtaining the specific position of the target vehicle on each input frame image and the motion trajectory of consecutive frames through S102, it is necessary to determine the motion status of the vehicle. The detection of vehicle motion status is based on the relative motion distance of vehicles in adjacent frames. S104: Vehicle compartment status recognition. The vehicle compartment status recognition is the core link of the truck empty compartment recognition and verification method. The input of this link is the paused image of the vehicle compartment, and the output is the vehicle compartment recognition status. The vehicle compartment status recognition network MCN is a dual-structure classification network, which is built on SENet and IBN-Net networks. Specifically, it involves four parts: image input, data preprocessing, image feature extraction and feature classification. S105: Vehicle compartment status reporting. The reporting of the compartment status is completed by the real-time early warning module. The core of this module is to record the identification status of each vehicle compartment by establishing a compartment status identification result table.

2. The intelligent truck empty compartment identification and verification method based on video stream features as described in claim 1, characterized in that: In step S102, the vehicle target detection process is executed based on the Yolov5 algorithm. By combining the ROI coordinate information corresponding to each camera, the target vehicle in the region of interest is obtained. The setting of the region of interest can effectively avoid interference caused by objects outside the region.

3. The intelligent truck empty compartment identification and verification method based on video stream features as described in claim 1, characterized in that: In step S103, the vehicle motion state is detected based on the relative motion distance of vehicles in adjacent frames. When the motion distance of vehicles in adjacent frames is greater than the set distance threshold, the vehicle is considered to be in motion; otherwise, the vehicle is in a paused state. The distance threshold is set to T1.

4. The intelligent truck empty compartment identification and verification method based on video stream features as described in claim 3, characterized in that: In step S103, the vehicle's movement distance is equal to the distance between the center points of the main diagonal of the vehicle's target location box in adjacent frames, calculated using the following formula: （1） （2） （3） （4） In equations 1 and 2, P t P t+1 Equation 1 represents the center coordinates of the main diagonal of the target vehicle's position box in frames t and t+1, respectively. Here, 0 represents the starting coordinate of the main diagonal of the vehicle's position box, and 3 represents the ending coordinate of the main diagonal of the vehicle's position box. x and y correspond to the horizontal and vertical coordinates, respectively. In Equation 3, Dist represents the distance the same vehicle travels in two adjacent frames. In Equation 4, the vehicle_state variable represents the vehicle's motion state. When it is equal to 0, it means the vehicle is stopped, and when it is equal to 1, it means the vehicle is in motion.

5. The intelligent truck empty compartment identification and verification method based on video stream features as described in claim 1, characterized in that: In step S104, the input for identifying the empty cargo compartment of a truck is the image of the cargo compartment after it has been paused. For the target vehicle after it has been paused, the cargo compartment status can be divided into three types: no door, empty compartment and not empty compartment. No door means that the cargo compartment door has not been opened after the vehicle has been paused. Empty compartment means that the cargo compartment is identified as empty. Not empty compartment means that the cargo compartment is identified as not empty.

6. The intelligent truck empty compartment identification and verification method based on video stream features as described in claim 5, characterized in that: In step S104, the core of the SENet network is to guide the model to focus on the differences in channel dimensions and automatically learn the nonlinear relationships between different channel features in order to improve the model's attention to important channel features.

7. The intelligent truck empty compartment identification and verification method based on video stream features as described in claim 6, characterized in that: In step S104, the IBN-net network is used to improve the model's ability to adapt to changes in image appearance.

8. The intelligent truck empty compartment identification and verification method based on video stream features as described in claim 1, characterized in that: In step S105, a real-time warning threshold and a timeout warning threshold are set in the real-time warning module. The real-time warning threshold represents the number of consecutive corresponding state frames that can confirm whether the car is empty or not, while the timeout warning threshold represents the number of consecutive unopened state frames that can confirm the unopened result.

9. The intelligent truck empty compartment identification and verification method based on video stream features as described in claim 8, characterized in that: In step S105, if the number of consecutive frames of the unopened state exceeds the timeout warning threshold, the corresponding carriage identification result is reported as unopened timeout; if the number of consecutive frames of the empty carriage state or the non-empty carriage state exceeds the real-time warning threshold, the empty carriage or non-empty carriage result is reported.

Images