Vehicle loading and unloading operation monitoring method and device, electronic equipment and storage medium

Abstract

The embodiment of the application relates to the technical field of computer vision, and discloses a vehicle loading and unloading operation monitoring method and device, electronic equipment and a storage medium, comprising: acquiring a detection frame of a target vehicle in continuous video frames; dividing the detection frame into a chassis area and a loading area; detecting a horizontal direction displacement amount of the chassis area between adjacent video frames, and determining that the target vehicle is in a physical static state when the horizontal direction displacement amount is less than a first threshold value and continuously satisfies a first preset frame number; detecting a vertical direction motion vector of the loading area between adjacent video frames, and determining that a loading and unloading action has been completed when an amplitude of the vertical direction motion vector is less than a second threshold value and continuously satisfies a second preset frame number; and determining that the target vehicle is in a loading and unloading operation static state when the target vehicle simultaneously satisfies the physical static state and the loading and unloading action has been completed. This method can accurately identify the state of the vehicle completing loading and unloading, and further improve the accuracy of the loading and unloading time length and adapt to complex working conditions in actual loading and unloading operations.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method, device, electronic device, and storage medium for monitoring vehicle loading and unloading operations. Background Technology

[0002] In ports, docks, logistics parks, and other similar settings, tracking the operation time of loading and unloading vehicles is a crucial means of measuring operational efficiency and optimizing scheduling management. As the primary transport vehicle for ton bag loading and unloading operations, the accurate calculation of loading and unloading times for container trucks directly impacts the accuracy of port operation efficiency assessments and billing settlements. Furthermore, accurate calculation of loading and unloading times relies on accurately monitoring the vehicle's status from the start to the completion of loading and unloading.

[0003] In related technologies, the technical approach of automatically detecting vehicle status and calculating operation time using camera equipment and image processing algorithms includes: using frame difference method or background modeling method to detect moving areas in video images to determine whether the vehicle is in motion; or using target detection algorithm to identify vehicle position, judging the vehicle's movement / stationary state by the change in the position of the detection box, and then using the time point when the vehicle's movement / stationary state changes as the start and end time points of loading and unloading to calculate the loading and unloading time.

[0004] However, simply judging the vehicle's movement / stationary state by the change in the position of the detection box and using the time point of state switching as the start and end time points for calculating the loading and unloading time cannot accurately identify the vehicle's completed loading and unloading state. This results in low accuracy of the calculated loading and unloading time and makes it difficult to adapt to the complex working conditions in actual loading and unloading operations. Summary of the Invention

[0005] The purpose of this application is to provide at least one method for monitoring vehicle loading and unloading operations, which can at least solve the problems in related technologies where the loading and unloading time is not accurately identified due to the inability to accurately identify the state of vehicle loading and unloading completion, thus making it difficult to adapt to the complex working conditions in actual loading and unloading operations. This method can at least accurately identify the state of vehicle loading and unloading completion, thereby improving the accuracy of loading and unloading time and adapting to the complex working conditions in actual loading and unloading operations.

[0006] To address the aforementioned technical problems, at least one embodiment of this application provides a method for monitoring vehicle loading and unloading operations, comprising: acquiring a detection frame of a target vehicle in consecutive video frames; dividing the detection frame into a chassis area and a loading area; detecting the horizontal displacement of the chassis area between adjacent video frames, and determining that the target vehicle is physically stationary when the horizontal displacement is less than a first threshold and continuously satisfies a first preset frame number; detecting the vertical motion vector of the loading area between adjacent video frames, and determining that the loading and unloading operation has been completed when the amplitude of the vertical motion vector is less than a second threshold and continuously satisfies a second preset frame number; and determining that the target vehicle is in a loading and unloading operation stationary state when the target vehicle simultaneously satisfies both the physical stationary state and the loading and unloading operation has been completed.

[0007] Actual loading and unloading operations are complex. A vehicle being physically stationary and the completion of loading / unloading are two distinct business states. A vehicle may be stationary while still waiting for a crane to lift the ton bag (loading / unloading not yet complete), while a vehicle may not have moved after the crane has lifted the bag (loading / unloading complete but vehicle not yet moved). By dividing the detection area into a chassis zone and a loading zone, and determining whether the target vehicle is physically stationary and whether loading / unloading has been completed based on the horizontal displacement of the chassis zone and the vertical movement vector of the loading zone, the actual state of the vehicle can be more accurately identified. This method distinguishes between completed loading / unloading and simple physical stationary states, accurately identifying the vehicle's completed loading / unloading status and adapting to complex loading and unloading conditions. When the target vehicle is physically stationary and loading / unloading has been completed, a combined determination of the target vehicle being in a loading / unloading stationary state facilitates the calculation of loading / unloading time, allowing for accurate calculation of the vehicle's loading / unloading duration.

[0008] Furthermore, by detecting the horizontal displacement of the chassis area and combining it with the continuous determination of multiple consecutive frames (the first preset number of frames), it is determined to be physically stationary only when the displacement is continuously less than the first threshold, which effectively eliminates high-frequency, small-amplitude idling vibration interference and avoids misjudgment of the state.

[0009] Furthermore, by detecting the vertical motion vector of the loading area and combining it with the continuous frame count determination (second preset frame count), the loading and unloading action is determined to be completed only when the amplitude of the vertical motion is continuously less than the second threshold, thus eliminating false alarms such as crane swaying under no-load conditions and wind interference, and accurately identifying that the loading and unloading action has been completed.

[0010] In some optional embodiments, the method further includes: recording the start time when the target vehicle changes from a moving state to a stationary state during the loading and unloading operation, and extracting the ReID (Re-Identification) feature of the target vehicle as an anchor feature; extracting the current ReID feature when the target vehicle changes from a stationary state to a moving state during the loading and unloading operation, and calculating the similarity between the current ReID feature and the anchor feature; recording the end time when the similarity is less than a third threshold, and calculating the loading and unloading duration based on the end time and the start time.

[0011] Using the stationary state of the loading and unloading operation as the trigger condition for starting / ending the timing ensures that the timing time truly matches the start and end times of the actual loading and unloading operation, solving the problem of low accuracy in calculating loading and unloading time based solely on the time point of change from static to dynamic state. By extracting and saving the ReID features of the target vehicle as anchor features, feature comparison is used at the end of the timing to confirm that it is the same vehicle, avoiding the incorrect accumulation of operation times for different vehicles (such as new vehicles entering the same berth).

[0012] In some optional embodiments, the method further includes: during the period when the target vehicle is in the stationary state of the loading and unloading operation, if the horizontal displacement of the chassis area is greater than or equal to the first threshold, then extracting the ReID features of the target vehicle after displacement; calculating the similarity between the ReID features after displacement and the anchor point features; if the similarity is greater than a fourth threshold and the amplitude of the vertical motion vector of the loading area is less than the second threshold, then determining that the target vehicle has undergone fine-tuning and moving, continuing the timing, and updating the anchor point features to the ReID features after displacement.

[0013] In actual loading and unloading operations, vehicles may need to make minor adjustments to their position within the berth to align with the crane. Current technologies often incorrectly terminate timing when a vehicle shifts, immediately classifying it as movement. By using ReID feature comparison, it can be identified that the shifted vehicle is still the same vehicle (similarity greater than the fourth threshold), and there is no loading or unloading activity in the loading area. This is then classified as a minor adjustment, and timing continues, ensuring continuity. Furthermore, after identifying a minor adjustment, the anchor point features are updated to the ReID features after the shift, ensuring that subsequent similarity comparisons always use the latest vehicle appearance as the standard, avoiding misjudgments caused by feature drift due to changes in lighting or angle.

[0014] In some optional embodiments, the method further includes: if the target vehicle is empty at the start of the timing and fully loaded at the end of the timing, then it is determined to be a loading operation; if the target vehicle is fully loaded at the start of the timing and empty at the end of the timing, then it is determined to be an unloading operation.

[0015] In some optional embodiments, the method further includes: acquiring a spatiotemporal topology map of the port area camera equipment, the spatiotemporal topology map including the shortest travel time window and the longest travel time window of a vehicle between any two camera equipment, and road connectivity information between the camera equipment; when the first camera equipment detects the disappearance of the target vehicle, extracting the first ReID feature and disappearance time of the target vehicle; when the second camera equipment detects a candidate vehicle, extracting the second ReID feature and appearance time of the candidate vehicle; calculating a comprehensive matching score, the comprehensive matching score being determined based on at least one of the following: the similarity between the first ReID feature and the second ReID feature, the spatiotemporal transition probability, and the spatial reachability determined based on the road connectivity information between the first camera equipment and the second camera equipment, the spatiotemporal transition probability being determined based on a first time difference, the shortest travel time window and the longest travel time window between the first camera equipment and the second camera equipment, the first time difference being the time difference between the disappearance time and the appearance time; if the comprehensive matching score is greater than a fifth threshold, then assigning the global identity of the target vehicle in the first camera equipment to the candidate vehicle in the second camera equipment.

[0016] In real-world scenarios, many vehicles have identical paint schemes and similar appearances, making pure ReID feature comparison prone to false matches. By introducing spatiotemporal transition probabilities and spatial reachability as constraints, even if two vehicles look similar, they cannot be matched if the time difference does not conform to the physical travel time window or there is no road connection between the camera devices, thus reducing the false match rate. The spatiotemporal transition probability eliminates erroneous trajectory matching that violates physical laws, such as instantaneous movement. By assigning a unified global identity to vehicles under different camera devices, the system supports full trajectory tracking of vehicles from entering the work area to completing the task and leaving, providing data support for scheduling management and post-event traceability.

[0017] In some optional embodiments, the determination of the spatiotemporal transition probability includes: if the first time difference is between the shortest travel time window and the longest travel time window, then the spatiotemporal transition probability is a first value; if the first time difference is less than the shortest travel time window, then the spatiotemporal transition probability is a second value; if the first time difference is greater than the longest travel time window, then the spatiotemporal transition probability decreases as the second time difference increases based on the first value, where the second time difference is the difference between the first time difference and the longest travel time window.

[0018] At least one embodiment of this application also provides a vehicle loading and unloading operation monitoring device, comprising: an acquisition module for acquiring a detection frame of a target vehicle in consecutive video frames; a division module for dividing the detection frame into a chassis area and a loading area; and a detection and determination module for detecting the horizontal displacement of the chassis area between adjacent video frames, and determining that the target vehicle is physically stationary when the horizontal displacement is less than a first threshold and continuously satisfies a first preset frame number; detecting the vertical motion vector of the loading area between adjacent video frames, and determining that the loading and unloading operation has been completed when the amplitude of the vertical motion vector is less than a second threshold and continuously satisfies a second preset frame number; and determining that the target vehicle is in a loading and unloading operation stationary state when the target vehicle simultaneously satisfies the physical stationary state and the loading and unloading operation has been completed.

[0019] In some optional embodiments, the detection and determination module is configured to record the start time when the target vehicle changes from a moving state to a stationary state during the loading and unloading operation, and extract the ReID features of the target vehicle as anchor features; when the target vehicle changes from a stationary state to a moving state during the loading and unloading operation, extract the current ReID features and calculate the similarity between the current ReID features and the anchor features; when the similarity is less than a third threshold, record the end time, and calculate the loading and unloading duration based on the end time and the start time.

[0020] In some optional embodiments, the detection and determination module is further configured to: extract the ReID features of the target vehicle after displacement if the horizontal displacement of the chassis area is greater than or equal to the first threshold while the target vehicle is in a stationary state during the loading and unloading operation; calculate the similarity between the ReID features after displacement and the anchor point features; if the similarity is greater than a fourth threshold and the amplitude of the vertical motion vector of the loading area is less than the second threshold, determine that the target vehicle has undergone fine-tuning and repositioning, continue timing, and update the anchor point features to the ReID features after displacement.

[0021] In some optional embodiments, the detection and determination module is further configured to determine a loading operation if the target vehicle is empty at the start of the timing and fully loaded at the end of the timing; and to determine an unloading operation if the target vehicle is fully loaded at the start of the timing and empty at the end of the timing.

[0022] In some optional embodiments, the detection and determination module is further configured to acquire a spatiotemporal topology map of the port area camera equipment, the spatiotemporal topology map including the shortest travel time window and the longest travel time window between any two camera equipment, as well as road connectivity information between camera equipment; when the first camera equipment detects the disappearance of the target vehicle, it extracts the first ReID feature and disappearance time of the target vehicle; when the second camera equipment detects a candidate vehicle, it extracts the second ReID feature and appearance time of the candidate vehicle; calculates a comprehensive matching score, the comprehensive matching score being determined based on at least one of the following: the similarity between the first ReID feature and the second ReID feature, the spatiotemporal transition probability, and the spatial reachability determined based on the road connectivity information between the first camera equipment and the second camera equipment, the spatiotemporal transition probability being determined based on a first time difference, the shortest travel time window and the longest travel time window between the first camera equipment and the second camera equipment, the first time difference being the time difference between the disappearance time and the appearance time; if the comprehensive matching score is greater than a fifth threshold, the global identity of the target vehicle in the first camera equipment is assigned to the candidate vehicle in the second camera equipment.

[0023] At least one embodiment of this application also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described vehicle loading and unloading operation monitoring method.

[0024] At least one embodiment of this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described vehicle loading and unloading operation monitoring method.

[0025] At least one embodiment of this application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described vehicle loading and unloading operation monitoring method. Attached Figure Description

[0026] One or more embodiments are illustrated by way of example with reference to the accompanying drawings, and these illustrative descriptions do not constitute a limitation on the embodiments.

[0027] Figure 1 This is a flowchart of a vehicle loading and unloading operation monitoring method provided in one embodiment of this application; Figure 2 This is a schematic diagram of an unloaded vehicle provided in one embodiment of this application; Figure 3This is a schematic diagram of a fully loaded vehicle provided in one embodiment of this application; Figure 4 This is a schematic diagram of a vehicle loading area and chassis area provided in one embodiment of this application; Figure 5 This is a schematic diagram of a vehicle loading and unloading operation monitoring device provided in another embodiment of this application. Detailed Implementation

[0028] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0029] Furthermore, the accompanying drawings are merely illustrative of this application and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0030] To facilitate understanding of the embodiments of this application, relevant content regarding vehicle loading and unloading operation monitoring will be introduced first.

[0031] In ports, docks, logistics parks, and other similar settings, tracking the operation time of loading and unloading vehicles is a crucial means of measuring operational efficiency and optimizing scheduling management. As the primary transport vehicle for ton bag loading and unloading operations, the accurate tracking of loading and unloading time of vehicles (container trucks) directly impacts the accuracy of port operation efficiency assessments and billing settlements. Furthermore, the accurate calculation of loading and unloading time relies on accurately monitoring the vehicle's status from the start to the completion of loading and unloading.

[0032] Currently, the statistics on loading and unloading operation time mainly rely on the following two methods: (a) Manual timing. The on-site dispatcher or tally clerk manually records the start time when the vehicle enters the work area and the end time when the vehicle finishes loading and unloading and leaves. This method has problems such as strong subjectivity, recording lag, easy omissions, inability to work continuously 24 / 7, and high labor costs, making it difficult to meet the needs of modern port's refined management.

[0033] (II) Automatic Detection Methods Based on Video Surveillance. With the development of computer vision technology, some ports have begun to try to automatically detect vehicle status and calculate operation time using camera equipment and image processing algorithms. The technical approach includes: using frame difference method or background modeling method to detect moving areas in video images to determine whether the vehicle is in motion; or using target detection algorithm to identify vehicle position, judging the vehicle's movement / stationary state by the change in the position of the detection box, and then using the time point when the vehicle's movement / stationary state changes as the start and end time points of loading and unloading to calculate the loading and unloading time.

[0034] However, the above-mentioned automatic detection methods still have the following technical problems in actual port scenarios: First, vehicle idling vibration leads to misjudgment of vehicle status. When vehicles are waiting to be loaded or unloaded or are queuing, their engines are usually idling, continuously generating high-frequency, small-amplitude mechanical vibrations. Traditional frame difference methods or background modeling methods cannot distinguish these minute vibrations from the actual physical movement of the vehicle, easily misjudging vehicles with idling vibrations as "moving," leading to frequent interruptions in loading / unloading timing or failure to start normally, resulting in severely distorted statistical results.

[0035] Second, interference from the movement of cranes and loading / unloading equipment. During loading and unloading operations, the crane boom and ton bags frequently rise and fall, creating significant areas of motion in the image. Traditional methods easily misinterpret crane movement as vehicle movement, or misinterpret the swaying of the crane that has not completely stopped as the loading / unloading action not yet being completed, leading to inaccurate judgments of the completion time of the loading / unloading operation.

[0036] Third, there is a lack of effective distinction between the physical stationary state of the vehicle and the completed state of the operation. A vehicle coming to a complete stop (physical stationary) and the completion of loading and unloading are two distinct operational states. After a vehicle comes to a complete stop, it may still be waiting for the crane to lift the ton bag (loading and unloading not yet complete), while after the crane has lifted the bag, the vehicle may still be waiting for instructions and has not yet moved (loading and unloading completed, but the vehicle has not moved). Traditional methods rely solely on whether the vehicle is moving or whether there is a moving area in the image to determine the loading and unloading status, failing to distinguish between these two scenarios. This leads to ambiguity in the timing start / end trigger logic, making it impossible to accurately identify the completed loading and unloading state. Consequently, the calculated loading and unloading time is inaccurate and difficult to adapt to the complex working conditions in actual loading and unloading operations.

[0037] To address the aforementioned problems, this invention provides a method for monitoring vehicle loading and unloading operations. By dividing the vehicle detection frame into a chassis area and a loading area for independent detection and joint judgment, the method achieves accurate identification of the static state of loading and unloading operations, thereby providing accurate and reliable triggering conditions for loading and unloading time statistics. The implementation details of this embodiment's vehicle loading and unloading operation monitoring method are described below. These details are provided for ease of understanding and are not essential for implementing this solution.

[0038] Example 1: The vehicle loading and unloading operation monitoring method of this embodiment can be applied to electronic devices with communication, computing, and data storage capabilities. Its specific process can be as follows: Figure 1 As shown, it includes: S101-S104.

[0039] S101, Obtain the detection bounding box of the target vehicle in consecutive video frames.

[0040] Among them, vehicle loading and unloading operations can take place in scenarios such as ports, docks, and logistics parks.

[0041] In vehicle loading and unloading operations, multiple camera devices are deployed. These devices can capture images of different areas within the operation, creating a video stream (continuous video frames) for each device. These cameras are connected to an electronic device, server, or other device executing this method, allowing the real-time video stream to be transmitted. Alternatively, if not directly connected to the electronic device, the device can connect to a server or other device connected to multiple cameras to obtain the video stream from each camera. For example, the camera device could be a webcam.

[0042] In a vehicle loading and unloading operation scenario, there may be one or more vehicles, usually multiple, and the same monitoring method is used for monitoring the loading and unloading operations of each vehicle.

[0043] Taking the monitoring of target vehicles as an example, when a target vehicle enters the vehicle loading and unloading operation scenario, the electronic equipment identifies and assigns a global identity identifier to the target vehicle to uniquely identify the target vehicle, so as to summarize the vehicle's trajectory, loading and unloading operations, etc. based on the global identity identifier.

[0044] In some examples, obtaining the detection box of a target vehicle in consecutive video frames may include: using a target detection model to perform target detection on each video frame that includes the target vehicle, thereby identifying the target vehicle and generating a detection box containing the target vehicle.

[0045] The specific model used for object detection is not limited in the embodiments of this application. Any model that can monitor target objects in video frames in real time can be applied here, such as YOLOv8 (You Only Look Once Version 8).

[0046] In some examples, the object detection model applied in this application includes a feature extraction module and a backbone network module. The feature extraction module is a multi-level feature pyramid used to extract multi-level features from the image, thereby facilitating better object detection. The backbone network module incorporates a GAM (Global Attention Mechanism) module to assign higher attention weights to small-scale targets (vehicles) in video frames and suppress background interference.

[0047] The embodiments of this application do not limit the specific number of feature extraction layers included in the multi-level feature pyramid. For example, the multi-level feature pyramid includes three feature extraction layers, which is a three-level downsampling architecture. For example, a three-level downsampling architecture of 160×160, 80×80, and 40×40 is adopted, wherein: the 160×160 level retains rich detail information to facilitate the detection of small-scale vehicles at long distances; the 80×80 level balances detail and semantic information; and the 40×40 level extracts high-level semantic features to facilitate the identification of fully loaded / empty states.

[0048] In some examples, the object detection model is YOLOv8, and the GAM module is inserted between YOLOv8's CSPN (CrossStage Partial Network) and SPPF (Spatial Pyramid Pooling – Fast). The GAM module extracts the global context vector, calculates the relevance score between each pixel feature and the global context, and enhances the target features in occluded regions (such as a carriage obscured by a crane) based on the relevance score.

[0049] In some examples, using object detection models to process video frames can output not only vehicle detection bounding boxes but also the vehicle's occupancy status (full load / empty). Specifically, an empty vehicle can be identified as follows: Figure 2 As shown, the record has a state: empty (empty). Vehicles in a fully loaded state can be... Figure 3 As shown, the record has a state: full (fully loaded).

[0050] Furthermore, the embodiments of this application do not limit how the object detection model is trained. In some examples, the samples used to train the object detection model can be obtained by collecting video streams of vehicle loading and unloading operations at different times (day / night / dusk) and under different weather conditions (e.g., sunny / rainy / foggy). Data augmentation is then performed on the collected video streams: random flipping, lighting simulation (e.g., brightness / contrast adjustment ±20%), blur simulation (simulating rainy / foggy weather), and occlusion simulation (randomly adding rectangular occlusion blocks to simulate crane occlusion), ultimately obtaining samples for training the object detection model.

[0051] In some cases, during the training of the target detection model using samples, the training method can adopt scenario-based adversarial training based on local occlusion adversarial masks.

[0052] To address the issue of frequent obstruction of ton bags during vehicle loading and unloading operations, a local obstruction adversarial mask training method is proposed: (1) Constructing a local occlusion adversarial mask: During training, simulated crane booms or ton bags are randomly generated to cover some features of the vehicle (e.g., occlusion ratio of 10%-40%). The occlusion areas are preferentially covered: the upper part of the truck bed (ton bag loading area), the side of the cab, and the wheel area.

[0053] (2) Anti-occlusion feature learning: This forces the model to reduce its reliance on the easily occluded ton bags themselves and instead learn to extract the following occlusion-resistant robust features: vehicle chassis structural features (not easily occluded), tire quantity and arrangement pattern, fixed edge contours of the cargo box, and longitudinal beam texture of the chassis.

[0054] (3) Adversarial example generation: The normalized pixel matrix of the video frame is used as a parameter, and the pixel matrix is ​​updated by gradient through the loss function; the pixel values ​​of the target object box in the original video frame are replaced with the pixel values ​​of the corresponding region of the adversarial sample; this process is repeated multiple times (e.g., 3-5 times) to generate diverse adversarial training samples.

[0055] Adding a certain proportion (e.g., 30%) of adversarial examples to the original training samples can reduce the risk of model overfitting.

[0056] In some examples, continuous tracking of the target vehicle is also included. The embodiments of this application do not limit how the continuous tracking of the target vehicle is performed.

[0057] In some examples, the process also includes: acquiring a spatiotemporal topology map of the port area's camera equipment, which includes the shortest and longest travel time windows between any two camera equipments, as well as road connectivity information between the camera equipments; when the first camera equipment detects the disappearance of a target vehicle, extracting the target vehicle's first ReID feature and disappearance time; when the second camera equipment detects a candidate vehicle, extracting the candidate vehicle's second ReID feature and appearance time; calculating a comprehensive matching score, which is determined based on at least one of the following: the similarity between the first and second ReID features, the spatiotemporal transition probability, and the spatial reachability determined based on the road connectivity information between the first and second camera equipments, where the spatiotemporal transition probability is determined based on a first time difference, the shortest and longest travel time windows between the first and second camera equipments, and the first time difference is the time difference between the disappearance time and the appearance time; if the comprehensive matching score is greater than a fifth threshold, then assigning the global identity of the target vehicle in the first camera equipment to the candidate vehicle in the second camera equipment.

[0058] Among them, the spatiotemporal topology graph can be pre-built and retrieved directly from memory when it is used.

[0059] In some cases, after the target vehicle disappears, the extracted first ReID feature of the target vehicle and the disappearance time are associated and stored in the feature dormancy pool. The feature dormancy pool retains the stored vehicle features (ReID feature and disappearance time) for a preset time, after which the vehicle features are deleted.

[0060] After detecting a newly emerging candidate vehicle, the comprehensive matching score between the candidate vehicle and each vehicle in the feature dormancy pool is first calculated, and then the candidate vehicle is determined to be a vehicle in the feature dormancy pool based on the comprehensive matching score.

[0061] If the candidate vehicle is determined to be a vehicle in the feature dormancy pool based on the comprehensive matching score, then the global identity of the vehicle in the feature dormancy pool will be assigned to the candidate vehicle.

[0062] If the candidate vehicle is determined not to be a vehicle in the feature dormancy pool based on the comprehensive matching score, then a new global identity identifier is assigned to the candidate vehicle.

[0063] The similarity between the first ReID feature and the second ReID feature can be calculated using any method that can calculate the similarity between features, such as cosine similarity, L1 similarity, etc.

[0064] Spatial accessibility is categorized as reachable or inaccessible. Based on the road connectivity information between camera devices in the spatiotemporal topology map, the spatial reachability between any two camera devices can be determined. If the road connectivity information indicates that the road between any two camera devices is blocked, then the spatial reachability between them is inaccessible. If the road connectivity information indicates that the road between any two camera devices is connected, then the spatial reachability between them is reachable.

[0065] In some examples, the overall matching score S: .

[0066] in, , , All are weighting coefficients. ; The similarity of the ReID features of the two vehicles; For spatiotemporal transition probability; The value is determined based on the spatial accessibility between the camera devices corresponding to the two vehicles.

[0067] about , , The specific values ​​for each are not limited in the embodiments of this application, and can be set and adjusted based on experience. For example, , , The values ​​are 0.5, 0.3, and 0.2 respectively.

[0068] In real-world scenarios, many vehicles have identical paint schemes and similar appearances, making pure ReID feature comparison prone to false matches. By introducing spatiotemporal transition probabilities and spatial reachability as constraints, even if two vehicles look similar, they cannot be matched if the time difference does not conform to the physical travel time window or there is no road connection between the camera devices, thus reducing the false match rate. The spatiotemporal transition probability eliminates erroneous trajectory matching that violates physical laws, such as instantaneous movement. By assigning a unified global identity to vehicles under different camera devices, the system supports full trajectory tracking of vehicles from entering the work area to completing the task and leaving, providing data support for scheduling management and post-event traceability.

[0069] In some examples, the spatiotemporal transition probability is determined as follows: if the first time difference is between the shortest travel time window and the longest travel time window, the spatiotemporal transition probability is a first value; if the first time difference is less than the shortest travel time window, the spatiotemporal transition probability is a second value; if the time difference is greater than the longest travel time window, the spatiotemporal transition probability decreases as the second time difference increases based on the first value, where the second time difference is the difference between the first time difference and the longest travel time window.

[0070] The first value can be 1, and the second value can be 0. Regarding the decrease in the spatiotemporal transition probability based on the first data as the second time difference increases, the specific manner of this decrease is not limited in the embodiments of this application. For example:

[0071] in, For the first time difference, For the longest travel time window, This is the attenuation coefficient (e.g., 5 minutes).

[0072] In some cases, the value is determined based on the spatial reachability between the camera devices corresponding to the two vehicles. If spatial reachability is reachable, then The third value is determined by spatial accessibility; if the accessibility is inaccessible (e.g., in a port setting, two cameras belong to different docks and there is no road access), then... The fourth value is 1. In some examples, the third value is 1 and the fourth value is 0.

[0073] In some examples, features of vehicles in the feature dormancy pool are prioritized when participating in matching. The more successful matches a vehicle corresponding to a global identity has, the higher its feature priority is when participating in matching. The embodiments of this application do not limit how to configure the priority of features of vehicles in the feature dormancy pool when participating in matching. Where the more successful matches a vehicle corresponding to a global identity has, the higher its feature priority is when participating in matching, the specific priority mechanism can be set based on experience.

[0074] By deleting vehicle features from the feature dormancy pool after a timeout and configuring the priority of vehicle features for matching, memory resources can be saved and computing resources can be reduced.

[0075] S102 divides the detection area into a chassis area and a loading area.

[0076] In some examples, dividing the detection box into chassis and loading areas means dividing the detection box in each video frame into chassis and loading areas; alternatively, it can be done after the detection box as a whole has not moved (from a moving state to a stationary state), and then dividing the detection box in subsequent stationary video frames into chassis and loading areas. This is because the vehicle is usually stationary during loading and unloading operations, and it is only at this time that it is necessary to determine whether the loading and unloading operation has been completed.

[0077] In some cases, the specific determination that the detection box as a whole does not move can be made by comparing the displacement of the detection box in adjacent frames. If the displacement is less than a set threshold, and multiple consecutive frames meet the condition of being less than the set threshold, then it can be determined that the detection box as a whole does not move. Then, the detection boxes in subsequent video frames are divided into chassis area and loading area.

[0078] In some cases, once the loading and unloading of the vehicle is determined to be complete, the detection area can be further divided into chassis and loading areas. For details on determining whether the loading and unloading of the vehicle is complete, please refer to S103 below, which will not be elaborated here.

[0079] In some examples, such as Figure 4 As shown (it should be noted that, Figure 4 The loading area in the text refers to the loading area. Dividing the detection frame into a chassis area and a loading area can include: dividing the detection frame into upper and lower parts by a horizontal dotted line, with the lower part being the chassis area and the upper part being the loading area, and the height ratio of the upper and lower parts being a preset ratio. For example, the lower part accounts for 30% of the total height and the upper part accounts for 70% of the total height.

[0080] S103, detect the horizontal displacement of the chassis area between adjacent video frames, and determine that the target vehicle is physically stationary when the horizontal displacement is less than the first threshold and continues to meet the first preset frame number; detect the vertical motion vector of the loading area between adjacent video frames, and determine that the loading and unloading action has been completed when the amplitude of the vertical motion vector is less than the second threshold and continues to meet the second preset frame number.

[0081] The embodiments of this application do not limit the methods for detecting the horizontal displacement of the chassis area between adjacent video frames. In some examples, detecting the horizontal displacement of the chassis area between adjacent video frames may include: detecting the horizontal displacement of the chassis area between adjacent video frames based on sparse feature point matching (e.g., ORB (Oriented FAST and Rotated BRIEF)) or based on dense optical flow fields.

[0082] The embodiments of this application do not limit how the vertical motion vector of the loading area is detected between adjacent video frames. In some examples, detecting the vertical motion vector of the loading area between adjacent video frames may include: extracting a foreground mask for the loading area using background modeling (e.g., using the MOG2 (Mixture of Gaussians version 2) algorithm); calculating the vertical motion vector of the foreground mask of adjacent video frames, and using the vertical motion vector of the foreground mask as the vertical motion vector between adjacent video frames.

[0083] The embodiments of this application do not limit the specific values ​​of the first threshold and the first preset frame number, the second threshold and the second preset frame number. For example, the first threshold is 2 pixels, the first preset frame number is 10 frames, the second threshold is 1 pixel, and the second preset frame number is (all frames within 5 seconds).

[0084] S104. When the target vehicle simultaneously meets the conditions of being physically stationary and having completed the loading and unloading operation, the target vehicle is determined to be in a stationary state during the loading and unloading operation.

[0085] Actual loading and unloading operations are complex. A vehicle being physically stationary and the completion of loading / unloading are two distinct business states. A vehicle may be stationary while still waiting for a crane to lift the ton bag (loading / unloading not yet complete), while a vehicle may not have moved after the crane has lifted the bag (loading / unloading complete but vehicle not yet moved). By dividing the detection area into a chassis zone and a loading zone, and determining whether the target vehicle is physically stationary and whether loading / unloading has been completed based on the horizontal displacement of the chassis zone and the vertical movement vector of the loading zone, the actual state of the vehicle can be more accurately identified. This method distinguishes between completed loading / unloading and simple physical stationary states, accurately identifying the vehicle's completed loading / unloading status and adapting to complex loading and unloading conditions. When the target vehicle is physically stationary and loading / unloading has been completed, a combined determination of the target vehicle being in a loading / unloading stationary state facilitates the calculation of loading / unloading time, allowing for accurate calculation of the vehicle's loading / unloading duration.

[0086] Furthermore, by detecting the horizontal displacement of the chassis area and combining it with the continuous determination of multiple consecutive frames (the first preset number of frames), it is determined to be physically stationary only when the displacement is continuously less than the first threshold, which effectively eliminates high-frequency, small-amplitude idling vibration interference and avoids misjudgment of the state.

[0087] Furthermore, by detecting the vertical motion vector of the loading area and combining it with the continuous frame count determination (second preset frame count), the loading and unloading action is determined to be completed only when the amplitude of the vertical motion is continuously less than the second threshold, thus eliminating false alarms such as crane swaying under no-load conditions and wind interference, and accurately identifying that the loading and unloading action has been completed.

[0088] In some examples, the start time is recorded when the target vehicle changes from a moving state to a stationary state during loading and unloading operations, and the ReID feature of the target vehicle is extracted as the anchor feature; when the target vehicle changes from a stationary state to a moving state during loading and unloading operations, the current ReID feature is extracted, and the similarity between the current ReID feature and the anchor feature is calculated; when the similarity is less than a third threshold, the end time is recorded, and the loading and unloading time is calculated based on the end time and the start time.

[0089] Using the stationary state of the loading and unloading operation as the trigger condition for starting / ending the timing ensures that the timing time truly matches the start and end times of the actual loading and unloading operation, solving the problem of low accuracy in calculating loading and unloading time based solely on the time point of change from static to dynamic state. By extracting and saving the ReID features of the target vehicle as anchor features, feature comparison is used at the end of the timing to confirm that it is the same vehicle, avoiding the incorrect accumulation of operation times for different vehicles (such as new vehicles entering the same berth).

[0090] In some cases, the start time is recorded when the target vehicle changes from a moving state to a stationary state during loading and unloading operations, and the end time is recorded when the target vehicle changes from a stationary state to a moving state during loading and unloading operations. The loading and unloading time is then calculated based on the end time and the start time.

[0091] In some examples, a target vehicle can be considered to be in motion if it is determined to be in a non-physically stationary state. In other examples, the horizontal displacement of the chassis area between adjacent video frames is detected, and the target vehicle is determined to be in motion if the horizontal displacement is greater than a sixth threshold and continues to satisfy a third preset number of frames.

[0092] The sixth threshold is not less than the first threshold, and the third preset frame number can be set based on experience. For example, the third preset frame number can be all video frames within five seconds.

[0093] In some examples, the process also includes: if the horizontal displacement of the chassis area is greater than or equal to the first threshold while the target vehicle is stationary during loading and unloading operations, then extract the ReID features of the target vehicle after displacement; calculate the similarity between the ReID features after displacement and the anchor point features; if the similarity is greater than the fourth threshold and the amplitude of the vertical motion vector of the loading area is less than the second threshold, then determine that the target vehicle has undergone minor adjustments, continue timing, and update the anchor point features to the ReID features after displacement.

[0094] In actual loading and unloading operations, vehicles may need to make minor adjustments to their position within the berth to align with the crane. Current technologies often incorrectly terminate timing when a vehicle shifts, immediately classifying it as movement. By using ReID feature comparison, it can be identified that the shifted vehicle is still the same vehicle (similarity greater than the fourth threshold), and there is no loading or unloading activity in the loading area. This is then classified as a minor adjustment, and timing continues, ensuring continuity. Furthermore, after identifying a minor adjustment, the anchor point features are updated to the ReID features after the shift, ensuring that subsequent similarity comparisons always use the latest vehicle appearance as the standard, avoiding misjudgments caused by feature drift due to changes in lighting or angle.

[0095] In some examples, the object detection model also outputs the full-load / empty status of the target vehicle in each video frame. The method also includes: if the target vehicle is empty at the start of the timer and full-load at the end of the timer, it is determined to be a loading operation; if the target vehicle is full-load at the start of the timer and empty at the end of the timer, it is determined to be an unloading operation.

[0096] In some cases, by assigning the same global identity to the same vehicle, all trajectories under the same global identity can be merged.

[0097] In some examples, the process also includes: aggregating local trajectory segments with the same global identity from different camera devices based on a global identity to form a first local trajectory set for the vehicle; filtering the first local trajectory set to obtain an optimized second local trajectory set; constructing a cost matrix for trajectory segments from different camera devices in the second local trajectory set, solving for minimum cost matching based on the cost matrix to ensure that each local trajectory is uniquely assigned to the global trajectory; then using a hierarchical clustering algorithm to cluster the local trajectories, and connecting the local trajectories within the same cluster to generate a complete global motion trajectory.

[0098] In some cases, filtering the first local set of trajectories can be based on one of the following constraints: Time Constraint: Verify whether the time difference between adjacent local trajectories (trajectory segments) is within the physical travel time window. For example, local trajectory 1 disappears at camera A at time t1, and local trajectory 2 appears at camera B at time t2. If t2-t1 [ , If ], then the association between local trajectory 1 and local trajectory 2 is eliminated; Spatial constraints: Based on the spatiotemporal topology map, verify the spatiotemporal reachability between trajectory segments; if there is no road connection between the camera devices corresponding to two local trajectories, then the association between local trajectory 1 and local trajectory 2 is eliminated; and if the ratio of the distance between the beginning and end of two local trajectories to the time difference between the beginning and end (the time difference between the time when the vehicle disappears in the previous trajectory and the time when the vehicle appears in the next trajectory) is greater than the set speed, then the association between local trajectory 1 and local trajectory 2 is eliminated (the speed is unreasonable). Motion consistency constraint: Verify the continuity of velocity and direction changes, calculate the average velocities v1 and v2 of adjacent local trajectories. If |v2-v1| / v1>the abrupt change threshold (velocity abrupt change exceeds the abrupt change threshold, for example, 0.5), the association between the two adjacent local trajectories is eliminated. If the change in the direction angle of the two adjacent local trajectories>the set degree, and the time difference between the beginning and end is<the time threshold, the association is eliminated (sharp turns are unreasonable).

[0099] In some examples, the cost matrix is ,in, This represents the combined matching score for the vehicles in the two corresponding local trajectories. In some cases, the Hungarian algorithm can be used to solve for the minimum cost matching, ensuring that each local trajectory is uniquely assigned to the global trajectory.

[0100] In some examples, the distance metric in the hierarchical clustering described above is... The clustering threshold is a spatiotemporal distance threshold (which can be set based on experience).

[0101] The temporal distance is the time difference between the beginning and end of the trajectory, and the spatial distance is the distance between the vanishing point of the vehicle in the previous local trajectory and the appearing point of the vehicle in the next local trajectory. The time of the vehicle's first appearance in a local trajectory can be used to distinguish which of the two local trajectories is the previous one and which is the next.

[0102] Example 2: Another embodiment of this application relates to a vehicle loading and unloading operation monitoring device. The implementation details of this embodiment's vehicle loading and unloading operation monitoring device are described below. The following implementation details are provided for ease of understanding and are not essential for implementing this solution. A schematic diagram of this embodiment's vehicle loading and unloading operation monitoring device can be seen as follows: Figure 5 As shown, the system includes: an acquisition module 51, used to acquire the detection box of the target vehicle in consecutive video frames; a division module 52, used to divide the detection box into a chassis area and a loading area; and a detection and determination module 53, used to detect the horizontal displacement of the chassis area between adjacent video frames, and determine that the target vehicle is physically stationary when the horizontal displacement is less than a first threshold and continuously meets a first preset frame number; detect the vertical motion vector of the loading area between adjacent video frames, and determine that the loading and unloading action has been completed when the amplitude of the vertical motion vector is less than a second threshold and continuously meets a second preset frame number; and determine that the target vehicle is in a loading and unloading operation stationary state when the target vehicle simultaneously meets the conditions of being physically stationary and the loading and unloading action has been completed.

[0103] In some optional embodiments, the detection and determination module 53 is used to record the start time when the target vehicle changes from a moving state to a stationary state during loading and unloading operations, and extract the ReID features of the target vehicle as anchor features; when the target vehicle changes from a stationary state during loading and unloading operations to a moving state, it extracts the current ReID features and calculates the similarity between the current ReID features and the anchor features; when the similarity is less than a third threshold, it records the end time and calculates the loading and unloading duration based on the end time and the start time.

[0104] In some optional embodiments, the detection and determination module 53 is further configured to, while the target vehicle is in a stationary state during loading and unloading operations, if the horizontal displacement of the chassis area is greater than or equal to a first threshold, extract the ReID features of the target vehicle after displacement; calculate the similarity between the ReID features after displacement and the anchor point features; if the similarity is greater than a fourth threshold and the amplitude of the vertical motion vector of the loading area is less than a second threshold, determine that the target vehicle has undergone fine-tuning and repositioning, continue timing, and update the anchor point features to the ReID features after displacement.

[0105] In some optional embodiments, the detection and determination module 53 is further configured to determine that if the target vehicle is empty at the start of the timing and fully loaded at the end of the timing, it is a loading operation; if the target vehicle is fully loaded at the start of the timing and empty at the end of the timing, it is a unloading operation.

[0106] In some optional embodiments, the detection and determination module 53 is further configured to acquire a spatiotemporal topology map of the port area camera equipment, the spatiotemporal topology map including the shortest and longest travel time windows of a vehicle between any two camera equipment, and road connectivity information between camera equipment; when the first camera equipment detects the disappearance of a target vehicle, it extracts the first ReID feature and disappearance time of the target vehicle; when the second camera equipment detects a candidate vehicle, it extracts the second ReID feature and appearance time of the candidate vehicle; calculates a comprehensive matching score, the comprehensive matching score being determined based on at least one of the following: the similarity between the first ReID feature and the second ReID feature, the spatiotemporal transition probability, and the spatial reachability determined based on the road connectivity information between the first and second camera equipment, the spatiotemporal transition probability being determined based on a first time difference, the shortest and longest travel time windows between the first and second camera equipment, the first time difference being the time difference between the disappearance time and the appearance time; if the comprehensive matching score is greater than a fifth threshold, the global identity of the target vehicle in the first camera equipment is assigned to the candidate vehicle in the second camera equipment.

[0107] Actual loading and unloading operations are complex. A vehicle being physically stationary and the completion of loading / unloading are two distinct business states. A vehicle may be stationary while still waiting for a crane to lift the ton bag (loading / unloading not yet complete), while a vehicle may not have moved after the crane has lifted the bag (loading / unloading complete but vehicle not yet moved). By dividing the detection area into a chassis zone and a loading zone, and determining whether the target vehicle is physically stationary and whether loading / unloading has been completed based on the horizontal displacement of the chassis zone and the vertical movement vector of the loading zone, the actual state of the vehicle can be more accurately identified. This method distinguishes between completed loading / unloading and simple physical stationary states, accurately identifying the vehicle's completed loading / unloading status and adapting to complex loading and unloading conditions. When the target vehicle is physically stationary and loading / unloading has been completed, a combined determination of the target vehicle being in a loading / unloading stationary state facilitates the calculation of loading / unloading time, allowing for accurate calculation of the vehicle's loading / unloading duration.

[0108] Furthermore, by detecting the horizontal displacement of the chassis area and combining it with the continuous determination of multiple consecutive frames (the first preset number of frames), it is determined to be physically stationary only when the displacement is continuously less than the first threshold, which effectively eliminates high-frequency, small-amplitude idling vibration interference and avoids misjudgment of the state.

[0109] Furthermore, by detecting the vertical motion vector of the loading area and combining it with the continuous frame count determination (second preset frame count), the loading and unloading action is determined to be completed only when the amplitude of the vertical motion is continuously less than the second threshold, thus eliminating false alarms such as crane swaying under no-load conditions and wind interference, and accurately identifying that the loading and unloading action has been completed.

[0110] It is worth mentioning that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed in this application; however, this does not mean that other units are absent in this embodiment.

[0111] Example 3: Another embodiment of this application relates to an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the vehicle loading and unloading operation monitoring method in the above embodiments.

[0112] The memory and processor are connected via a bus, which can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and memories. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0113] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0114] Example 4: Another embodiment of this application relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method embodiments described above.

[0115] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0116] Example 5: Another embodiment of this application relates to a computer program product, including a computer program. When executed by a processor, the computer program implements the method embodiments described above.

[0117] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0118] Furthermore, although the steps of the method in this application are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0119] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware.

[0120] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing this application, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of this application.

Claims

1. A method for monitoring vehicle loading and unloading operations, characterized in that, include: Obtain the bounding box of the target vehicle in consecutive video frames; The detection frame is divided into a chassis area and a loading area; The horizontal displacement of the chassis area between adjacent video frames is detected. When the horizontal displacement is less than a first threshold and continues to meet a first preset frame number, the target vehicle is determined to be physically stationary. The vertical motion vector of the loading area between adjacent video frames is detected. When the amplitude of the vertical motion vector is less than a second threshold and continues to meet a second preset frame number, the loading and unloading action is determined to be completed. When the target vehicle simultaneously meets the conditions of being physically stationary and the loading / unloading action being completed, the target vehicle is determined to be in a stationary state during loading / unloading operations.

2. The method according to claim 1, characterized in that, Also includes: When the target vehicle changes from a moving state to a stationary state during the loading and unloading operation, the start time of timing is recorded, and the ReID feature of the target vehicle is extracted as the anchor point feature. When the target vehicle changes from a static state to a moving state during the loading and unloading operation, the current ReID feature is extracted, and the similarity between the current ReID feature and the anchor point feature is calculated. When the similarity is less than the third threshold, the end time of the timing is recorded, and the loading and unloading time is calculated based on the end time of the timing and the start time of the timing.

3. The method according to claim 2, characterized in that, Also includes: If the horizontal displacement of the chassis area is greater than or equal to the first threshold during the period when the target vehicle is stationary during the loading and unloading operation, the ReID features of the target vehicle after displacement are extracted. Calculate the similarity between the ReID features after displacement and the anchor point features; If the similarity is greater than the fourth threshold and the amplitude of the vertical motion vector of the loading area is less than the second threshold, then it is determined that the target vehicle has undergone minor adjustment and relocation, the timing continues, and the anchor point feature is updated to the ReID feature after the displacement.

4. The method according to claim 2, characterized in that, Also includes: If the target vehicle is empty at the start of the timing and fully loaded at the end of the timing, it is determined to be a loading operation. If the target vehicle is fully loaded at the start of the timer and empty at the end of the timer, it is determined to be an unloading operation.

5. The method according to any one of claims 1-4, characterized in that, Also includes: Obtain a spatiotemporal topology map of the port area camera equipment, the spatiotemporal topology map including the shortest and longest travel time windows of vehicles between any two camera equipment, as well as road connectivity information between camera equipment; When the first camera device detects that the target vehicle has disappeared, it extracts the first ReID feature and disappearance time of the target vehicle; When the second camera device detects a candidate vehicle, it extracts the second ReID feature and the time of appearance of the candidate vehicle; Calculate a comprehensive matching score, which is determined based on at least one of the following: the similarity between the first ReID feature and the second ReID feature, the spatiotemporal transition probability, and the spatial reachability determined based on the road connectivity information between the first camera device and the second camera device. The spatiotemporal transition probability is determined based on a first time difference, the shortest travel time window and the longest travel time window between the first camera device and the second camera device. The first time difference is the time difference between the disappearance time and the appearance time. If the overall matching score is greater than the fifth threshold, then the global identity of the target vehicle in the first camera device is assigned to the candidate vehicle in the second camera device.

6. The method according to claim 5, characterized in that, The methods for determining the spatiotemporal transition probability include: If the first time difference is between the shortest travel time window and the longest travel time window, then the spatiotemporal transition probability is a first value; If the first time difference is less than the shortest travel time window, then the spatiotemporal transition probability is the second value; If the first time difference is greater than the longest travel time window, the spatiotemporal transition probability decreases as the second time difference increases, based on the first value. The second time difference is the difference between the first time difference and the longest travel time window.

7. A vehicle loading and unloading operation monitoring device, characterized in that, include: The acquisition module is used to acquire the detection bounding boxes of target vehicles in consecutive video frames; A partitioning module is used to divide the detection frame into a chassis area and a loading area; The detection and determination module is used to detect the horizontal displacement of the chassis area between adjacent video frames. When the horizontal displacement is less than a first threshold and continuously meets a first preset frame number, the target vehicle is determined to be in a physically stationary state. The module also detects the vertical motion vector of the loading area between adjacent video frames. When the amplitude of the vertical motion vector is less than a second threshold and continuously meets a second preset frame number, the loading and unloading action is determined to be completed. When the target vehicle simultaneously meets the conditions of being physically stationary and the loading and unloading action being completed, the target vehicle is determined to be in a loading and unloading operation stationary state.

8. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the vehicle loading and unloading operation monitoring method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the vehicle loading and unloading operation monitoring method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the vehicle loading and unloading operation monitoring method according to any one of claims 1 to 6.

Images