Cargo transport abnormal event monitoring method and system based on load sensing and visual recognition

Abstract

This invention proposes a method and system for monitoring abnormal events in cargo transportation based on load sensing and visual recognition. The method includes: acquiring information about the planned operation area, collecting cargo weight data in real time and continuously monitoring it; when a weight change exceeding a threshold is detected and occurs outside the planned operation area, triggering an onboard intelligent camera to perform visual recognition analysis; using a lightweight target detection model deployed at the edge to identify whether preset abnormal behavior characteristics exist; if abnormal behavior is confirmed, automatically recording video clips and packaging them with time, location, and weight change data to generate structured evidence and pushing it to the cloud platform, and performing graded handling responses according to the severity of the anomaly. This invention achieves full-link automation from anomaly perception, cross-verification, evidence locking to closed-loop handling through a time-series linkage mechanism of load anomaly triggering visual verification, significantly reducing the false alarm rate and improving the level of safety management of cargo in transit.

Description

Technical Field

[0001] This invention relates to the field of intelligent logistics and transportation safety monitoring technology, and in particular to a method and system for monitoring abnormal events in cargo transportation based on load sensing and visual recognition. Background Technology

[0002] The safe management of goods during road transportation has long been a key challenge for the logistics industry. Traditional monitoring methods mainly rely on GPS tracking and manual reporting by drivers, lacking the ability to perceive the real-time status of the cargo container. This makes it difficult to detect and effectively prevent abnormal events such as cargo theft and substitution during transportation.

[0003] To address these issues, the industry has developed several technological solutions. One is weight monitoring based on load sensors, which continuously collects data on vehicle load to detect changes in cargo. However, single-dimensional weight data struggles to distinguish between normal loading / unloading and fraudulent unloading, and is easily affected by factors such as vehicle bumps and road slopes, resulting in a high false alarm rate. The second is visual monitoring based on video cameras, which can directly record images around the cargo container. However, limitations such as lighting conditions, obstructions, and the large volume of video data make it difficult to achieve real-time intelligent analysis throughout the process. Most solutions rely on post-event playback for review, leading to significant response delays and an inability to intervene promptly when anomalies occur. Third, some systems have attempted to combine load sensors with video surveillance (such as CN201611255261.3 and CN118294004A). The basic idea is to collect two types of data in parallel, but there is a lack of an effective mechanism for time-series linkage and cross-verification of the two types of information. The video call and the weight change event are decoupled, and it is impossible to achieve an automated logical closed loop from triggering anomalies to visual verification and finally to locking evidence. It still relies on manual post-event correlation review, making it difficult to achieve real-time identification and immediate response to abnormal loading and unloading behaviors during the journey.

[0004] Furthermore, existing solutions have the following shortcomings: a lack of automatic evidence-building mechanisms after anomalies occur, resulting in incomplete evidence chains and low review efficiency; and a disconnect between alarm information and response actions, failing to form a complete closed loop from discovery to intervention. Therefore, there is an urgent need for an intelligent monitoring solution capable of deep temporal fusion of load sensing and AI vision, possessing real-time cross-validation and automatic evidence locking capabilities. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a method for monitoring abnormal events in cargo transportation based on load sensing and visual recognition, comprising the following steps:

[0006] S1: Construct an electronic fence in the planned operation area of ​​the transportation task and obtain the location information of the transportation vehicles in real time.

[0007] The electronic fence is a virtual geographical boundary constructed based on geographic coordinates, used to define the planned operation area for transportation tasks.

[0008] The planned operation area is issued when the logistics order is dispatched, including loading points, unloading points, and key areas along the route.

[0009] S2: Collects cargo weight data from transport vehicles in real time and continuously monitors changes in weight data.

[0010] S3: When the change in weight data is detected to meet the preset conditions, the location information of the transport vehicle and the electronic fence are used to determine whether the current weight data change occurs outside the planned operation area.

[0011] S4: If S3 determines that the current weight data change occurs outside the planned work area, then trigger and execute visual recognition analysis; if S3 determines that the current weight data change occurs within the planned work area, then return to S2 to continue monitoring.

[0012] S5: If the visual recognition analysis confirms the existence of abnormal behavior characteristics, a video segment of a preset recording duration is automatically recorded, and the video segment is associated with and packaged with event metadata to generate structured abnormal event evidence; if the visual recognition analysis confirms that there are no abnormal behavior characteristics, the process returns to S2 to continue monitoring.

[0013] The event metadata includes the timestamp corresponding to the video clip, the location information of the transport vehicle, the type of abnormal behavior characteristics, the amount of change in weight data, the vehicle ID, and the driver information.

[0014] Preferably, the preset recording duration is set to 10-20 seconds; in addition, to improve the legal validity of the evidence, the video clips are encrypted end-to-end, and a digital signature is attached to the encrypted data packet to ensure the confidentiality and non-repudiation of the evidence data.

[0015] S6: Push the structured abnormal event evidence to the cloud. After receiving the evidence, the cloud determines the corresponding handling response level based on the mapping relationship between the structured abnormal event evidence and the preset abnormal level, and executes the handling action corresponding to the handling response level.

[0016] Furthermore, the preset conditions specifically involve setting a weight change threshold W and a time window T. If a transport vehicle, located outside the planned operation area, experiences a weight data change greater than W and a change duration less than T, it is determined to be a possible abnormal unloading event.

[0017] Furthermore, the visual recognition analysis specifically includes: scanning and collecting image data of the interior of the cargo container or the unloading area, performing visual recognition analysis on the image data, and determining whether there are any abnormal behavioral characteristics, including personnel intrusion, unauthorized opening of the container, and operation of loading and unloading equipment.

[0018] Furthermore, the preset anomaly level mapping relationship includes: The change in weight data is mapped to the first dimension interval, the type of abnormal behavior characteristics is mapped to the second dimension interval, and the location information of the current transport vehicle and the distance to the nearest planned operation area are mapped to the third dimension interval. Based on the mapping results of the structured abnormal event evidence in the first dimension interval, the second dimension interval, and the third dimension interval, the corresponding handling response level is determined in a preset handling response level lookup table.

[0019] Furthermore, the response levels include a first level and a second level; wherein, the response actions corresponding to the first level include sending abnormal alarm notifications and evidence summary information to relevant personnel via SMS or instant messaging applications; the response actions corresponding to the second level include initiating a voice call intervention request to relevant personnel through the platform to promptly stop the illegal behavior from continuing.

[0020] Furthermore, it also includes: taking static images of the cargo box area at fixed time intervals during the entire logistics transportation process by the transport vehicle, and synthesizing the static images into a full-process visual retrospective video in chronological order after the transportation task is completed, and storing it in association with the structured abnormal event evidence to provide two-layer traceability evidence for post-event auditing.

[0021] Another aspect of the present invention provides a cargo transportation anomaly monitoring system based on load sensing and visual recognition for implementing the above-described method, comprising:

[0022] The positioning module is used to obtain the location information of the transport vehicles in real time.

[0023] The load sensing module is used to collect real-time data on the weight of goods transported by vehicles.

[0024] The visual acquisition module is used to acquire image data of the interior of the cargo box or the unloading area, and transmit the acquired image data to the vehicle-mounted edge computing terminal; the visual acquisition module also includes an infrared supplementary lighting unit for providing auxiliary lighting under low light conditions.

[0025] The vehicle-mounted edge computing terminal is connected to the load sensing module, positioning module, and visual acquisition module, respectively. It is used to determine whether the current weight data change of the transport vehicle occurs outside the planned operation area, send a trigger signal to the visual acquisition module to perform visual recognition analysis, push the structured abnormal event evidence to the cloud monitoring platform through the wireless communication network, and also to associate and package video clips with event metadata to generate structured abnormal event evidence.

[0026] The cloud-based monitoring platform communicates with the vehicle-mounted edge computing terminal via a wireless communication network. It is used to receive structured abnormal event evidence, determine the corresponding handling response level based on the structured abnormal event evidence and the preset abnormal level mapping relationship, and execute the handling action corresponding to the handling response level. It is also used to send the planned operation area to the vehicle-mounted edge computing terminal when dispatching logistics orders.

[0027] Furthermore, the vehicle-mounted edge computing terminal is also equipped with a lightweight target detection model, which is a convolutional neural network model that has undergone pruning and quantization processing. This model is used to perform visual recognition analysis on the received image data to determine whether there are any abnormal behavioral features.

[0028] Furthermore, the cloud-based monitoring platform also includes an evidence storage module and an event report generation module. The evidence storage module is used to store structured evidence of abnormal events and full-process visual retrospective video; the event report generation module is used to generate a comprehensive report that includes transportation trajectory, a list of abnormal events, and handling records, supporting access from multiple terminals.

[0029] The beneficial effects of the cargo transportation abnormal event monitoring method and system based on load sensing and visual recognition of the present invention are as follows: (1) The present invention does not simply collect load sensors and cameras in parallel, but establishes a time-series linkage logic that first uses load abnormality as a trigger condition and then uses AI vision as a verification means. Only when the load data shows a sudden change that meets the preset conditions in an unplanned area will the AI ​​vision module be awakened for targeted identification. The causal relationship verification based on the dual-modal data on the time axis effectively overcomes the problem of high false alarm rate of single sensor, and solves the pain point of large computing power consumption and inability to process in real time in the whole process video analysis; (2) After the dual-modal cross-verification confirms the abnormality, the system automatically extracts key video segments and automatically packages them with timestamps, location coordinates, and weight change data to form a structured evidence chain and uploads it in real time, changing the inefficient mode of video and weight data being separated in the existing solution and requiring manual post-event correlation. The legal validity of the evidence is further guaranteed by end-to-end encryption and digital signature mechanism; (3) Cargo box photos are taken at a fixed frequency throughout the transportation process, and timeline videos are automatically synthesized after the task is completed, providing complete visual evidence for post-event auditing. Combined with precise evidence fragments of abnormal events, a two-layer traceability system is formed; (4) Differentiated handling is carried out according to the severity of abnormal events. General abnormalities are notified by SMS / instant messaging to form the first level of handling, and serious abnormalities trigger human voice intervention to form the second level of handling, avoiding one-size-fits-all excessive alarms and improving handling efficiency and user experience. Attached Figure Description

[0030] Figure 1 This is a schematic block diagram of the cargo transportation anomaly monitoring system based on load sensing and visual recognition according to the present invention. Figure 2 This is a flowchart illustrating the abnormal event monitoring method for cargo transportation based on load sensing and visual recognition according to the present invention. Detailed Implementation

[0031] To provide a further understanding of the purpose, structure, features, and functions of the present invention, detailed descriptions are provided below with reference to specific embodiments.

[0032] Example 1: As Figure 1 As shown, this invention proposes a cargo transportation anomaly event monitoring system based on load sensing and visual recognition, comprising:

[0033] The positioning module is used to obtain the location information of the transport vehicle in real time; the positioning module includes a GPS module and a Beidou satellite positioning module.

[0034] The load sensing module is used to collect real-time cargo weight data of the transport vehicle; the load sensing module includes strain gauge sensors or hydraulic sensors, which are installed on the vehicle frame or suspension system.

[0035] The visual acquisition module is used to acquire image data of the interior of the cargo box or the unloading area and transmit the acquired image data to the vehicle-mounted edge computing terminal; the visual acquisition module also includes an infrared supplementary lighting unit for providing auxiliary lighting under low light conditions.

[0036] The vehicle-mounted edge computing terminal is connected to the load sensing module, positioning module, and vision acquisition module, respectively. The vehicle-mounted edge computing terminal also deploys a lightweight target detection model that has undergone model pruning and INT8 quantization processing, used to perform visual recognition analysis on the received image data to determine whether abnormal behavioral characteristics exist. The vehicle-mounted edge computing terminal is connected to the load sensing module via a CAN bus or RS485 interface, and to the vision acquisition module via a MIPI or USB interface.

[0037] The vehicle-mounted edge computing terminal is also used to determine whether the change in the current weight data of the transport vehicle occurs outside the planned operation area, send a trigger signal to the visual acquisition module to perform visual recognition analysis, associate and package video clips with event metadata to generate structured abnormal event evidence, and push the structured abnormal event evidence to the cloud monitoring platform through wireless communication networks such as 4G / 5G.

[0038] The cloud-based monitoring platform communicates with the vehicle-mounted edge computing terminal via a wireless communication network. It receives structured abnormal event evidence, determines the corresponding response level based on the mapping relationship between the structured abnormal event evidence and preset abnormality levels, and executes the corresponding response actions. It also distributes planned work areas to the vehicle-mounted edge computing terminal during logistics order dispatch. The cloud-based monitoring platform further includes an evidence storage module and an event report generation module. The evidence storage module stores structured abnormal event evidence and full-process visual retrospective video; the event report generation module generates a comprehensive report including transportation trajectory, abnormal event list, and handling records, supporting access from multiple terminals including PCs and mobile devices.

[0039] like Figure 2 As shown, this invention proposes a method for monitoring abnormal events in cargo transportation based on load sensing and visual recognition, including the following steps:

[0040] S1: Construct an electronic fence in the planned operation area of ​​the transportation task and obtain the location information of the transportation vehicles in real time.

[0041] When dispatching logistics orders, the monitoring platform sends the planned operation area information to the vehicle-mounted edge computing terminal and stores it locally. The planned operation area includes loading points, unloading points, and key areas along the route.

[0042] S2: Collects cargo weight data from transport vehicles in real time and continuously monitors changes in weight data.

[0043] The load sensing module collects cargo weight data at a sampling rate of 10Hz.

[0044] S3: When the change in weight data is detected to meet the preset conditions, the location information of the transport vehicle and the electronic fence are used to determine whether the current weight data change occurs outside the planned operation area.

[0045] Set a weight change threshold W and a time window T. If a transport vehicle in an area outside the planned operation area experiences a weight change greater than W and a change duration less than T, it is considered a possible abnormal unloading event.

[0046] S4: If S3 determines that the current weight data change occurs outside the planned work area, then trigger and execute visual recognition analysis; if S3 determines that the current weight data change occurs within the planned work area, then return to S2 to continue monitoring.

[0047] Triggering and executing visual recognition analysis specifically includes: the vehicle-mounted edge computing terminal triggers the visual acquisition module, which collects image data of the cargo box or unloading area and transmits the collected image data to the vehicle-mounted edge computing terminal; the lightweight target detection model in the vehicle-mounted edge computing terminal performs inference frame by frame to initially confirm abnormal behavior characteristics.

[0048] S5: If the visual recognition analysis confirms the existence of abnormal behavior characteristics, a video segment of a preset recording duration is automatically recorded, and the video segment is associated with and packaged with event metadata to generate structured abnormal event evidence; if the visual recognition analysis confirms that there are no abnormal behavior characteristics, the process returns to S2 to continue monitoring.

[0049] After confirming the presence of abnormal behavior characteristics, video clips of 8 seconds before and after the trigger time are extracted, encrypted using AES-256 and digitally signed, and packaged together with timestamps, GPS coordinates of the transport vehicle, changes in weight data, types of abnormal behavior characteristics, vehicle IDs and driver information to generate structured abnormal event evidence, which is then uploaded to the cloud monitoring platform via the MQTT protocol.

[0050] S6: Push the structured abnormal event evidence to the cloud. After receiving the evidence, the cloud determines the corresponding handling response level based on the mapping relationship between the structured abnormal event evidence and the preset abnormal level, and executes the handling action corresponding to the handling response level.

[0051] The cloud-based monitoring platform extracts the following key information from structured anomaly evidence: the amount of change in weight data. The types of abnormal behavior characteristics and the current location information of the transport vehicle.

[0052] calculate The ratio of the vehicle's rated load capacity W_max The first dimension level is determined based on the R value: R<10%: Low grade; 10%≤R<25%: Mid-range; R≥25%: High-end.

[0053] The second dimension level is determined based on the type of abnormal behavioral characteristics confirmed by visual recognition analysis: Only one of the following was found: unauthorized intrusion or unauthorized opening: Low-end; Simultaneous occurrence of unauthorized intrusion and unauthorized opening: Mid-range; Loading and unloading equipment operation: High-end.

[0054] Calculate the distance D between the current location of the transport vehicle and the nearest planned work area, and determine the third dimension level based on the value of D: D<5 km: Low-end; 5 km ≤ D < 20 km: Mid-range; D≥20 km: High-end.

[0055] The system has a pre-set response level lookup table as shown in Table 1. The response level is determined by looking up the table based on the combination of the three dimensions mentioned above: Table 1: Response Level Lookup Table First Dimension Second dimension Third dimension Response Level low-end low-end low-end / low-end low-end Mid- to high-end First level low-end Mid- to high-end any First level Mid-range any any First level upscale any any Second level any upscale any Second level Finally, actions corresponding to the response level are executed. The actions corresponding to the first level include sending abnormal alarm notifications and evidence summary information to relevant personnel via SMS or instant messaging applications; the actions corresponding to the second level include initiating a voice call intervention request to relevant personnel through the platform to promptly stop the illegal behavior from continuing.

[0056] Example 2: This example adds a full-process visual backtracking function to Example 1. Every 5 minutes, the transport vehicle takes a static image of the cargo box and uploads it to the cloud monitoring platform. After the transport task is completed, the cloud monitoring platform synthesizes the static images into a visual backtracking video in chronological order. Abnormal events are marked with markers; clicking on a marker allows viewing the corresponding evidence segment. The visual backtracking video is then associated and stored with the structured abnormal event evidence.

[0057] The present invention has been described in the above-described embodiments; however, these embodiments are merely examples for implementing the present invention. It must be noted that the disclosed embodiments do not limit the scope of the present invention. Conversely, any modifications and refinements made without departing from the spirit and scope of the present invention are within the scope of patent protection of the present invention.

[0058] The contents of this invention not described in detail are existing technologies known to those skilled in the art.

Claims

1. A method for monitoring abnormal events in cargo transportation based on load sensing and visual recognition, characterized in that, Includes the following steps; S1: Construct an electronic fence in the planned operation area of ​​the transportation task and obtain the location information of the transportation vehicles in real time; The planned operation area is issued when the logistics order is dispatched, including loading points, unloading points, and key areas along the route; S2: Real-time collection of cargo weight data from transport vehicles and continuous monitoring of weight data changes; S3: When the change in weight data is detected to meet the preset conditions, the location information of the transport vehicle and the electronic fence are used to determine whether the current weight data change occurs outside the planned operation area. S4: If S3 determines that the current weight data change occurs outside the planned work area, then trigger and execute visual recognition analysis; if S3 determines that the current weight data change occurs within the planned work area, then return to S2 to continue monitoring. S5: If the visual recognition analysis confirms the existence of abnormal behavior characteristics, a video segment of a preset recording duration is automatically recorded, and the video segment is associated with and packaged with event metadata to generate structured abnormal event evidence; If the visual recognition analysis confirms that there are no abnormal behavioral characteristics, then return to S2 to continue monitoring; The event metadata includes the timestamp corresponding to the video clip, the location information of the transport vehicle, the type of abnormal behavior characteristics, the amount of change in weight data, the vehicle ID and driver information; S6: Push the structured abnormal event evidence to the cloud. After receiving the evidence, the cloud determines the corresponding handling response level based on the mapping relationship between the structured abnormal event evidence and the preset abnormal level, and executes the handling action corresponding to the handling response level.

2. The method according to claim 1, characterized in that, The visual recognition analysis specifically includes: scanning and collecting image data of the interior of the cargo container or the unloading area, performing visual recognition analysis on the image data, and determining whether there are any abnormal behavioral characteristics. The types of abnormal behavioral characteristics include personnel intrusion, unauthorized opening of the container, and operation of loading and unloading equipment.

3. The method according to claim 1, characterized in that, The preset conditions in step S3 are as follows: set a weight change threshold W and a time window T. If the transport vehicle is located outside the planned operation area, and the weight data change is greater than W and the duration of the change is less than T, it is determined to be a possible abnormal unloading event.

4. The method according to claim 1, characterized in that, The preset anomaly level mapping relationship includes: The change in weight data is mapped to the first dimension interval, the type of abnormal behavior characteristics is mapped to the second dimension interval, and the location information of the current transport vehicle and the distance to the nearest planned operation area are mapped to the third dimension interval. Based on the mapping results of the structured abnormal event evidence in the first dimension interval, the second dimension interval, and the third dimension interval, the corresponding handling response level is determined in a preset handling response level lookup table.

5. The method according to claim 4, characterized in that, The response levels are divided into a first level and a second level. The first level corresponds to actions such as sending abnormal alarm notifications and evidence summary information to relevant personnel via SMS or instant messaging applications. The second level corresponds to actions such as initiating voice call intervention requests to relevant personnel through the platform to promptly stop the illegal behavior from continuing.

6. The method according to claim 1, characterized in that, Also includes: During the entire logistics transportation process, the transport vehicle takes static images of the cargo box area at fixed time intervals. After the transportation task is completed, the static images are synthesized in chronological order into a full-process visual retrospective video, which is then associated with and stored with the structured abnormal event evidence to provide two-layer traceability evidence for post-event auditing.

7. A cargo transportation anomaly monitoring system based on load sensing and visual recognition, characterized in that, include: The positioning module is used to obtain the location information of the transport vehicle in real time; The load sensing module is used to collect real-time data on the weight of goods transported by the vehicle. The visual acquisition module is used to acquire image data of the interior of the cargo box or the unloading area, and transmit the acquired image data to the vehicle-mounted edge computing terminal; the visual acquisition module also includes an infrared supplementary lighting unit for providing auxiliary lighting under low light conditions. The vehicle-mounted edge computing terminal is connected to the load sensing module, the positioning module, and the vision acquisition module, respectively. It is used to determine whether the current weight data change of the transport vehicle occurs outside the planned operation area, send a trigger signal to the vision acquisition module to perform visual recognition analysis, push the structured abnormal event evidence to the cloud monitoring platform through the wireless communication network, and also to associate and package video clips with event metadata to generate structured abnormal event evidence. The cloud-based monitoring platform communicates with the vehicle-mounted edge computing terminal via a wireless communication network. It is used to receive structured abnormal event evidence and perform graded handling responses according to the severity of the abnormal events. It is also used to send planned operation areas to the vehicle-mounted edge computing terminal when dispatching logistics orders.

8. The system according to claim 7, characterized in that, The vehicle-mounted edge computing terminal is also equipped with a lightweight target detection model, which is a convolutional neural network model that has undergone pruning and quantization processing. This model is used to perform visual recognition analysis on the received image data to determine whether there are any abnormal behavioral features.

9. The system according to claim 7, characterized in that, The cloud-based monitoring platform also includes an evidence storage module and an event report generation module. The evidence storage module is used to store structured evidence of abnormal events; the event report generation module is used to generate comprehensive reports that include transportation trajectories, lists of abnormal events, and handling records, and supports access from multiple terminals.

Images