Goods state monitoring and driving behavior linkage tracing method and system based on infrared imaging and Internet of Vehicles

Abstract

The invention provides a cargo state monitoring and driving behavior linkage tracing method and system based on infrared imaging and Internet of Vehicles. The method comprises the following steps: respectively acquiring video streams and driving behavior data through an infrared camera in a cargo box and a vehicle-mounted T-BOX; after the data are uploaded to the cloud, cargo state analysis and driving behavior analysis are executed in parallel; the cargo state analysis adopts a customized YOLOv8 model to identify cargos, and the state change is quantified by calculating the centroid offset and the size variation; and when an abnormal cargo state, dangerous driving or illegal door opening event is detected, a storage mechanism is intelligently triggered, and video clips before and after the event and synchronous driving data are stored in an associated manner. According to the invention, accurate perception of the cargo state and correlation analysis of the driving behavior are realized, the problems of monitoring blind areas, data isolation and difficult tracing in the prior art are solved, and the efficiency of transportation safety management and responsibility determination is significantly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent Internet of Vehicles, and particularly relates to a cargo state monitoring and driving behavior linkage tracing method and system based on infrared imaging and Internet of Vehicles. BACKGROUND

[0002] The existing cargo transportation monitoring technology has the following defects: Data isolation: relying on GPS or a single sensor, unable to real-time perceive the displacement of the cargo in the vehicle compartment or the opening of the cargo box.

[0003] Lack of correlation analysis: driving behavior data (such as sudden acceleration and sudden braking) is not associated with cargo state changes, making it difficult to determine accident liability.

[0004] Low tracing efficiency: after an abnormal event occurs, video and data retrieval takes a long time, and it is difficult to quickly locate the key node.

[0005] Therefore, a solution is needed that can integrate multi-source data, realize real-time linkage analysis, and efficiently trace. SUMMARY

[0006] The present application relates to the technical field of intelligent Internet of Vehicles, and particularly relates to a cargo state monitoring and driving behavior linkage tracing method and system based on infrared imaging and Internet of Vehicles.

[0007] To achieve the above-mentioned purpose, the present application adopts the following technical solutions: A cargo state monitoring and driving behavior linkage tracing method based on infrared imaging and Internet of Vehicles, comprising the following steps: S1: collecting video streams through infrared cameras deployed in the cargo box of the transportation vehicle, and collecting driving behavior data of the vehicle through the vehicle-mounted intelligent terminal T-BOX; The infrared camera adopts a wide-angle lens and is fixedly installed on the top central axis of the cargo box, ensuring that the field of view can completely cover the main cargo carrying area inside the cargo box.

[0008] S2: uploading the video stream and driving behavior data collected in S1 to a cloud processing platform; The cloud processing platform processes the video stream by video slicing, cuts the continuous video stream into equal-length segments (such as 5 seconds per segment), and adds accurate time stamps.

[0009] The cloud processing platform encapsulates the driving behavior data into data packets conforming to JT / T808 or its extended protocol, and each data packet has a time stamp synchronized with the video stream.

[0010] S3: Analyze the video stream and driving behavior data on the cloud processing platform and detect abnormal situations; the analysis includes: Cargo status analysis: The trained object detection model is used to identify and locate cargo in the video stream, and the status change index of the cargo is calculated based on the identification results; The "trained object detection model" is a customized version of the YOLOv8s model. This model is trained using a dataset of infrared images containing various types of goods (such as boxes, pallets, and bags) in truck cargo scenarios, specifically for identifying and locating goods.

[0011] Driving behavior analysis: Based on the driving behavior data, determine whether a preset type of dangerous driving behavior has occurred; S4: When the cloud processing platform detects an abnormal situation, it automatically triggers a video storage command, then extracts the corresponding video clips and driving behavior data, and stores them; The triggering conditions include: The status change index of the goods exceeds a first preset threshold; The system determines that a dangerous driving behavior of the preset type has occurred; A cargo box door opening event was detected; The event triggering unit supports logical judgment: for example, when both conditions of "cargo status indicators exceeding the threshold" and "dangerous driving behavior occurring simultaneously" are met, the event level can be increased, or a longer video retention period can be triggered. Furthermore, for "cargo box opening events," if GPS data shows the vehicle is located in a fixed logistics park or warehouse, this event may not trigger an emergency alarm, but will only be logged to avoid generating a large number of invalid alarms during normal loading and unloading.

[0012] Based on the timestamp T_event of the trigger time, the system automatically retrieves all video clips within the time interval [T_event - 60s, T_event + 60s] from the uploaded video clips and seamlessly stitches them into a complete traceable video file. A unique safety event ID is generated simultaneously. This ID, the stitched video file, and all driving behavior data within the time interval [T_event - 60s, T_event + 60s] are packaged together into a single data packet and stored in cloud object storage. The database records the event ID, trigger time, trigger type, and associated data packet path.

[0013] Furthermore, in the cargo status analysis step, calculating the cargo status change index specifically includes: Each frame of the video stream is defined as I. t The object detection model is used to identify a set of bounding boxes Bt={b} for n goods.i t |i=1,2,...,n}, where the i-th bounding box b i t From its upper left corner coordinate (x 1i t ,y 1i t ) and the coordinates of the lower right corner (x 2i t ,y 2i t )definition; Calculate the overall centroid coordinates C of the cargo set in the current frame t. t =(X c t ,Y c t The calculation formula is as follows: ; Calculate the average cargo size S in the current frame t. t The calculation formula is as follows: ; The centroid offset Dc and the size change Ds are calculated separately as the state change indicators: ; ,in( , )and The overall centroid coordinates and average size of the cargo collection for the reference frame.

[0014] Furthermore, the triggering condition "the status change index of the goods exceeds a first preset threshold" is defined as: The centroid offset D c greater than the preset displacement threshold T shift , and / or the size change D s Size change greater than the preset threshold T size。

[0015] Furthermore, in the driving behavior analysis step, the preset type of dangerous driving behavior includes at least one of rapid acceleration, sudden braking, and sharp turning; Among them, rapid acceleration or sudden braking is identified by determining whether the longitudinal acceleration value exceeds the second preset threshold T2; Sharp turns are identified by determining whether the lateral acceleration value or the steering wheel angle exceeds a third preset threshold T3.

[0016] Furthermore, prior to the data upload step, the following steps are also included: The video stream is sliced ​​at the vehicle-mounted edge computing node to generate continuous video segments of equal duration; In the event triggering and storage step, the captured video segment is composed of the continuous video segments.

[0017] Furthermore, the method also includes a traceability step: Receive a query request, which includes the event time or event type; Based on the query request, the video clips and driving behavior data associated with the specified time period are retrieved from the storage system and output, wherein the video clips and driving behavior data are aligned and played synchronously using a unified timestamp.

[0018] A cargo status monitoring and driving behavior linkage traceability system based on infrared imaging and vehicle networking includes: The vehicle-mounted data acquisition module, including an infrared camera and a vehicle-mounted intelligent terminal T-BOX, is used to collect video streams inside the cargo box and vehicle driving behavior data. Cloud processing platforms include: A communication unit is used to receive the video stream and driving behavior data; A cargo status analysis unit is used to perform the cargo status analysis. A driving behavior analysis unit is used to perform the driving behavior analysis. An event triggering unit is used to determine whether the triggering conditions are met based on the analysis results and to generate a video storage instruction. Cloud storage service, used in response to the video storage instruction, to store the video clip and associated driving behavior data.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows: Infrared imaging technology enables all-weather visual monitoring of cargo status. Combined with precise mathematical quantification models (centroid shift, dimensional changes), it can sensitively detect minute abnormal movements of cargo and provide timely warnings.

[0020] By conducting parallel cloud-based analysis and correlation of visual changes in cargo and driving behavior data, a causal relationship model of "driving behavior - cargo status" was established, providing an intuitive and reliable chain of composite evidence for liability determination.

[0021] By adopting an "event-triggered" storage strategy, only key video clips before and after abnormal events are saved and packaged and associated with the corresponding driving data. This not only significantly reduces cloud storage costs but also shortens the time for accident review and liability determination from hours to minutes, significantly improving traceability efficiency. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating the implementation of the cargo status monitoring and driving behavior linkage traceability method based on infrared imaging and vehicle networking of the present invention. Detailed Implementation

[0023] To provide a further understanding of the purpose, structure, features, and functions of the present invention, detailed descriptions are provided below with reference to specific embodiments.

[0024] A method for cargo status monitoring and driving behavior linkage and traceability based on infrared imaging and vehicle networking includes the following steps: S1: Video streams are collected by infrared cameras deployed in the cargo box of transport vehicles, and driving behavior data of the vehicles are collected by the vehicle-mounted intelligent terminal T-BOX; The infrared camera uses a wide-angle lens and is fixedly installed on the central axis of the top of the cargo box to ensure that the field of view can completely cover the main cargo area inside the cargo box.

[0025] S2: Upload the video stream and driving behavior data collected in S1 to the cloud processing platform; The cloud processing platform performs video slicing on the video stream, cutting the continuous video stream into segments of equal length (e.g., 5 seconds each) and adding precise timestamps.

[0026] The cloud processing platform encapsulates the driving behavior data into data packets conforming to JT / T808 or its extended protocols, with each data packet carrying a timestamp synchronized with the video stream.

[0027] S3: Analyze the video stream and driving behavior data on the cloud processing platform and detect abnormal situations; the analysis includes: Cargo status analysis: The trained object detection model is used to identify and locate cargo in the video stream, and the status change index of the cargo is calculated based on the identification results; The "trained object detection model" is a customized version of the YOLOv8s model. This model is trained using a dataset of infrared images containing various types of goods (such as boxes, pallets, and bags) in truck cargo scenarios, specifically for identifying and locating goods.

[0028] Driving behavior analysis: Based on the driving behavior data, determine whether a preset type of dangerous driving behavior has occurred; S4: When the cloud processing platform detects an abnormal situation, it automatically triggers a video storage command, then extracts the corresponding video clips and driving behavior data, and stores them; The triggering conditions include: The status change index of the goods exceeds a first preset threshold; The system determines that a dangerous driving behavior of the preset type has occurred; A cargo box door opening event was detected; The event triggering unit supports logical judgment: for example, when both conditions of "cargo status indicators exceeding the threshold" and "dangerous driving behavior occurring simultaneously" are met, the event level can be increased, or a longer video retention period can be triggered. Furthermore, for "cargo box opening events," if GPS data shows the vehicle is located in a fixed logistics park or warehouse, this event may not trigger an emergency alarm, but will only be logged to avoid generating a large number of invalid alarms during normal loading and unloading.

[0029] Based on the timestamp T_event of the trigger time, the system automatically retrieves all video clips within the time interval [T_event - 60s, T_event + 60s] from the uploaded video clips and seamlessly stitches them into a complete traceable video file. A unique safety event ID is generated simultaneously. This ID, the stitched video file, and all driving behavior data within the time interval [T_event - 60s, T_event + 60s] are packaged together into a single data packet and stored in cloud object storage. The database records the event ID, trigger time, trigger type, and associated data packet path.

[0030] In summary, to address the issue of "inability to perceive cargo status in real time," this application utilizes a customized YOLOv8s infrared vision model and precise mathematical quantification indicators (centroid shift, dimensional changes) to transform the vague "cargo movement" into quantifiable and accurate data that triggers an alarm when thresholds are exceeded. This enables automated, high-precision, real-time perception of abnormal states such as cargo displacement, tilting, and collapse within the cargo container, overcoming the blind spots of GPS and single sensors in monitoring the interior of the cargo container.

[0031] To address the issue of "lack of correlation analysis between driving behavior and cargo status," this application no longer views video data and vehicle data in isolation. Instead, it synchronizes them through timestamps, and event-triggered units execute complex logical judgments (such as AND logic). This enables the system to intelligently identify the strong correlation between the "cause" (dangerous driving behavior) and the "effect" (abnormal cargo status), providing an irrefutable chain of causal evidence for liability determination and completely changing the previous situation of isolated data.

[0032] To address the issue of "difficulty in quickly locating the responsible party after an abnormal event occurs," this application generates a unique safety event ID and packages and associates all evidence (videos, driving data, location information). This allows investigators to obtain a complete, multi-dimensional, and spatiotemporally aligned "accident report" within 5 minutes by simply querying the event ID, without having to review lengthy video recordings after an accident. This greatly improves the efficiency and accuracy of tracing.

[0033] Furthermore, in the cargo status analysis step, calculating the cargo status change index specifically includes: Each frame of the video stream is defined as I. t The object detection model is used to identify a set of bounding boxes Bt={b} for n goods. i t |i=1,2,...,n}, where the i-th bounding box b i t From its upper left corner coordinate (x 1i t ,y 1i t ) and the coordinates of the lower right corner (x 2i t ,y 2i t )definition; Calculate the overall centroid coordinates C of the cargo set in the current frame t. t =(X c t ,Y c t The calculation formula is as follows: ; Calculate the average cargo size S in the current frame t. t The calculation formula is as follows: ; The centroid offset Dc and the size change Ds are calculated separately as the state change indicators: ; ,in( , )and The overall centroid coordinates and average size of the cargo collection for the reference frame.

[0034] This method utilizes an object detection model to obtain the precise bounding box of the goods, and calculates the overall centroid coordinates (C0) of the goods set. t ) and average size (S) t This digitally represents the cargo status from two dimensions: "location" and "shape." It is compared with previously stored stable reference frame data (C... ref , S ref By performing real-time comparisons, the system can sensitively detect the shift of the centroid (D). c (and the change in the dimensions of the stacked goods).

[0035] Furthermore, the triggering condition "the status change index of the goods exceeds a first preset threshold" is defined as: The centroid offset Dc greater than the preset displacement threshold T shift , and / or the size change D s Size change greater than the preset threshold T size。 By setting a displacement threshold (T) shift ) and size change threshold (T) size The system implements data-driven automated alarms, avoiding the subjectivity and lag of manual inspections, and greatly reducing false alarms and missed alarms.

[0036] Furthermore, in the driving behavior analysis step, the preset type of dangerous driving behavior includes at least one of rapid acceleration, sudden braking, and sharp turning; Specifically, rapid acceleration or sudden braking is identified by determining whether the longitudinal acceleration value exceeds a second preset threshold T2; the second preset threshold T2 is set to 0.45g ~ 0.55g (approximately 4.4 m / s² ~ 5.4 m / s²). When the absolute value of the longitudinal acceleration continuously exceeds this threshold for 0.5 seconds, it is determined to be a dangerous behavior.

[0037] Sharp turns are identified by determining whether the lateral acceleration value or the steering wheel angle exceeds a third preset threshold T3. The third preset threshold T3 is set to 0.4g (approximately 3.9 m / s²) for lateral acceleration or 90° / second for steering wheel angle.

[0038] Furthermore, prior to the data upload step, the following steps are also included: The video stream is sliced ​​at the vehicle-mounted edge computing node to generate continuous video segments of equal duration; In the event triggering and storage steps, the extracted video segments are composed of consecutive video segments. When the cloud triggers a storage command, the system does not need to perform complex cropping and encoding conversion on an extremely long video file. Instead, it only needs to quickly retrieve and combine existing video slices within the time interval [T_event - 60s, T_event + 60s] based on the trigger timestamp T_event, like assembling building blocks. This makes generating a complete traceability video file fast and cost-effective.

[0039] Furthermore, the method also includes a traceability step: Receive a query request, which includes the event time or event type; Based on the query request, the video clips and driving behavior data associated with the specified time period are retrieved from the storage system and output, wherein the video clips and driving behavior data are aligned and played synchronously using a unified timestamp.

[0040] Users no longer need to manually review hours of video footage; they only need to input key information, and the system can locate the target event and all related data within seconds. This greatly improves the efficiency of accident review and liability determination, compressing investigation work that might have previously taken hours or even days into just a few minutes.

[0041] A cargo status monitoring and driving behavior linkage traceability system based on infrared imaging and vehicle networking includes: The vehicle-mounted data acquisition module, including an infrared camera and a vehicle-mounted intelligent terminal T-BOX, is used to collect video streams inside the cargo box and vehicle driving behavior data. Cloud processing platforms include: A communication unit is used to receive the video stream and driving behavior data; A cargo status analysis unit is used to perform the cargo status analysis. A driving behavior analysis unit is used to perform the driving behavior analysis. An event triggering unit is used to determine whether the triggering conditions are met based on the analysis results and to generate a video storage instruction. Cloud storage service, used in response to the video storage instruction, to store the video clip and associated driving behavior data.

[0042] The present invention has been described in the above-described embodiments; however, these embodiments are merely examples for implementing the present invention. It must be noted that the disclosed embodiments do not limit the scope of the present invention. Conversely, any modifications and refinements made without departing from the spirit and scope of the present invention are within the scope of patent protection of the present invention.

Claims

1. A method for cargo status monitoring and driving behavior linkage traceability based on infrared imaging and vehicle networking, characterized in that, Includes the following steps: S1: Video streams are collected by infrared cameras deployed in the cargo box of transport vehicles, and driving behavior data of the vehicles are collected by the vehicle-mounted intelligent terminal T-BOX; S2: Upload the video stream and driving behavior data collected in S1 to the cloud processing platform; S3: Analyze the video stream and driving behavior data on the cloud processing platform and detect abnormal situations; the analysis includes: Cargo status analysis: The trained object detection model is used to identify and locate cargo in the video stream, and the status change index of the cargo is calculated based on the identification results; Driving behavior analysis: Based on the driving behavior data, determine whether a preset type of dangerous driving behavior has occurred; S4: When the cloud processing platform detects an abnormal situation, it automatically triggers a video storage command, then extracts the corresponding video clips and driving behavior data, and stores them; The triggering conditions include: A: The status change index of the goods exceeds the first preset threshold; B: Determines that the preset type of dangerous driving behavior has occurred; C: A cargo box opening event was detected.

2. The cargo status monitoring and driving behavior linkage traceability method based on infrared imaging and vehicle networking as described in claim 1, characterized in that: The cargo status analysis step specifically includes calculating the cargo status change indicators: Each frame of the video stream is defined as I. t The object detection model is used to identify a set of bounding boxes Bt={b} for n goods. i t |i=1,2,...,n}, where the i-th bounding box b i t From its upper left corner coordinate (x 1i t ,y 1i t ) and the coordinates of the lower right corner (x 2i t ,y 2i t )definition; Calculate the overall centroid coordinates C of the cargo set in the current frame t. t =(X c t ,Y c t The calculation formula is as follows: ； Calculate the average cargo size S in the current frame t. t The calculation formula is as follows: ； The centroid offset Dc and the size change Ds are calculated separately as the state change indicators: ； ,in( , )and The overall centroid coordinates and average size of the cargo collection for the reference frame.

3. The cargo status monitoring and driving behavior linkage traceability method based on infrared imaging and vehicle networking as described in claim 2, characterized in that: The trigger condition "the status change index of the goods exceeds the first preset threshold" is defined as follows: The centroid offset D c Displacement greater than the preset threshold T shift , and / or the size change D s Size change greater than the preset threshold T size。 4. The cargo status monitoring and driving behavior linkage traceability method based on infrared imaging and vehicle networking as described in claim 1, characterized in that: In the driving behavior analysis step, the preset type of dangerous driving behavior includes at least one of rapid acceleration, sudden braking, and sharp turning; Among them, rapid acceleration or sudden braking is identified by determining whether the longitudinal acceleration value exceeds the second preset threshold T2; Sharp turns are identified by determining whether the lateral acceleration value or the steering wheel angle exceeds a third preset threshold T3.

5. The cargo status monitoring and driving behavior linkage traceability method based on infrared imaging and vehicle networking as described in claim 1, characterized in that: Prior to the data upload step, the following is also included: The video stream is sliced ​​at the vehicle-mounted edge computing node to generate continuous video segments of equal duration; In the event triggering and storage step, the captured video segment is composed of the continuous video segments.

6. The cargo status monitoring and driving behavior linkage traceability method based on infrared imaging and vehicle networking as described in claim 1, characterized in that: The method also includes a traceability step: Receive a query request, which includes the event time or event type; Based on the query request, the video clips and driving behavior data associated with the specified time period are retrieved from the storage system and output, wherein the video clips and driving behavior data are aligned and played synchronously using a unified timestamp.

7. A cargo status monitoring and driving behavior linkage traceability system based on infrared imaging and vehicle networking for implementing the method of any one of claims 1-6, characterized in that, include: The vehicle-mounted data acquisition module, including an infrared camera and a vehicle-mounted intelligent terminal T-BOX, is used to collect video streams inside the cargo box and vehicle driving behavior data. Cloud processing platforms include: A communication unit is used to receive the video stream and driving behavior data; A cargo status analysis unit is used to perform the cargo status analysis. A driving behavior analysis unit is used to perform the driving behavior analysis. An event triggering unit is used to determine whether the triggering conditions are met based on the analysis results and to generate a video storage instruction. Cloud storage service, used in response to the video storage instruction, to store the video clip and associated driving behavior data.

Images