Truck loading violation behavior detection method, device and equipment and storage medium

Abstract

The invention relates to the technical field of intelligent logistics, in particular to a loading violation behavior detection method, device and equipment and a storage medium, and the method comprises the steps: firstly carrying out the labeling processing of an obtained original video sample set, obtaining a labeled video sample set, then carrying out the data enhancement processing of the labeled video sample set, obtaining a target enhanced video sample set, and carrying out the data enhancement processing of the target enhanced video sample set; backbone network optimization processing and attention mechanism introduction processing are carried out on the constructed initial illegal behavior detection model to obtain an optimized illegal behavior detection model, and then model interface parameters of the optimized illegal behavior detection model are set; and performing multiple rounds of iterative training on the optimized violation behavior detection model based on the model interface parameters and the target enhanced video sample set to obtain a target violation behavior detection model, finally obtaining a to-be-identified loading scene video, inputting the to-be-identified loading scene video into the target violation behavior detection model to perform violation behavior detection, and obtaining a detection result. The objective of the invention is to improve the efficiency of logistics loading safety management.

Description

Technical Field

[0001] This invention relates to the field of intelligent logistics technology, and in particular to a method, device, equipment, and storage medium for detecting loading violations. Background Technology

[0002] As a core hub for commodity circulation, the loading process in the logistics industry is a crucial link between cargo storage and transportation. Its operational safety directly impacts the health and safety of personnel, the integrity of goods, and the overall efficiency of the supply chain. In practice, negligence or violations by some staff during loading can easily lead to accidents such as cargo tipping and collisions, causing direct economic losses. Furthermore, it can seriously threaten the lives of on-site personnel, disrupt the normal order of logistics transportation, and trigger a series of chain-reaction operational problems.

[0003] For a long time, loading safety inspections have mainly relied on manual monitoring. Staff members identify violations through visual observation and on-site inspections. This model is limited by labor costs and manpower, and has inherent drawbacks such as low inspection efficiency and high missed detection rates, making it difficult to provide real-time and comprehensive coverage of the entire loading process. With the continuous expansion of the logistics industry, the substantial increase in freight volume, and the increasing complexity of loading scenarios, the application of various new types of goods and loading / unloading equipment has further increased the difficulty of manual inspections. Traditional manual inspection methods can no longer meet the safety management needs of modern logistics systems. Summary of the Invention

[0004] In order to overcome the shortcomings of the prior art, the present invention aims to provide a method, device, equipment and storage medium for detecting loading violations, which aims to reduce the reliance on manual monitoring and improve the efficiency of logistics loading safety management.

[0005] The first aspect of this invention provides a method for detecting loading violations, comprising: acquiring an original video sample set; labeling the original video sample set to obtain a labeled video sample set; performing data augmentation on the labeled video sample set to obtain a target augmented video sample set; constructing an initial violation detection model; performing backbone network optimization and attention mechanism introduction on the initial violation detection model to obtain an optimized violation detection model; setting model interface parameters of the optimized violation detection model; performing multi-round iterative training on the optimized violation detection model based on the model interface parameters and the target augmented video sample set to obtain a target violation detection model; acquiring a loading scene video to be identified; inputting the loading scene video to be identified into the target violation detection model for violation detection to obtain a detection result.

[0006] Optionally, in a first implementation of the first aspect of the present invention, the original video sample set includes multiple video samples, and each video sample includes multiple video frames; the step of annotating the original video sample set to obtain an annotated video sample set includes: using an annotation tool to annotate the forklift position of each video frame to obtain multiple initial annotated video samples, each initial annotated video sample including a rectangle corresponding to each video frame; determining the vehicle moving speed and vehicle moving direction corresponding to each initial annotated video sample based on the multiple rectangles; determining the forklift behavior category corresponding to each initial annotated video sample based on the vehicle moving speed and vehicle moving direction; and using the annotation tool to perform behavior category annotation processing on each initial annotated video sample based on the multiple forklift behavior categories to obtain the annotated video sample set.

[0007] Optionally, in a second implementation of the first aspect of the present invention, the step of performing data augmentation processing on the labeled video sample set to obtain a target augmented video sample set includes: performing random cropping processing on the labeled video sample set using a random region cropping algorithm to obtain a first augmented video sample set; performing horizontal flipping processing on the first augmented video sample set using a video geometric transformation algorithm to obtain a second augmented video sample set; performing brightness and contrast adjustment processing on the second augmented video sample set using a pixel adjustment algorithm to obtain a third augmented video sample set; performing Gaussian noise addition processing on the third augmented video sample set using a Gaussian noise generation algorithm to obtain a fourth augmented video sample set; and performing scale scaling processing on the fourth augmented video sample set using a video scale transformation algorithm to obtain the target augmented video sample set.

[0008] Optionally, in a third implementation of the first aspect of the present invention, the initial violation detection model includes an initial backbone network. The step of optimizing the initial violation detection model by removing fully connected layers from the initial backbone network using a model optimization algorithm to obtain a lightweight violation detection model. The lightweight violation detection model includes an optimized backbone network, which includes multiple key feature layers. An attention mechanism is then introduced into the lightweight violation detection model based on these multiple key feature layers to obtain the optimized violation detection model.

[0009] Optionally, in a fourth implementation of the first aspect of the present invention, the step of introducing an attention mechanism into the lightweight violation detection model based on multiple key feature layers to obtain the optimized violation detection model includes: using a response analysis algorithm to perform vehicle entry detection correlation filtering on the multiple key feature layers to obtain a key feature layer to be enhanced, and introducing an attention mechanism into the key feature layer to be enhanced to obtain a feature layer to be pooled; using a pooling algorithm to perform global average pooling on the feature layer to be pooled to obtain a global average pooled feature vector; calculating weight coefficients for each feature channel of the global average pooled feature vector based on the Sigmoid activation function to obtain weight coefficients corresponding to each feature channel; performing element-wise multiplication on each feature channel based on multiple weight coefficients to obtain an optimized feature layer; and using a feature fusion algorithm to fuse the optimized feature layer into the lightweight violation detection model to obtain the optimized violation detection model.

[0010] Optionally, in a fifth implementation of the first aspect of the present invention, setting the model interface parameters of the optimized violation detection model includes: performing resolution statistical processing on the labeled video sample set using a statistical algorithm to obtain sample resolution data; determining and setting the input size parameters of the optimized violation detection model based on the sample resolution data; acquiring vehicle appearance morphology features based on the labeled video sample set, and determining and setting the output anchor frame size parameters of the optimized violation detection model based on the vehicle appearance morphology features.

[0011] Optionally, in a sixth implementation of the first aspect of the present invention, the step of performing multiple rounds of iterative training on the optimized violation detection model based on the model interface parameters and the target enhanced video sample set to obtain the target violation detection model includes: obtaining actual detection results corresponding to the target enhanced video sample set; inputting the target enhanced video sample set into the optimized violation detection model based on the model interface parameters to perform violation detection and obtain predicted detection results; calculating model loss based on the predicted detection results and the actual detection results; iteratively updating the weight parameters of the optimized violation detection model using a backpropagation algorithm based on the model loss; and outputting the target violation detection model when a preset iteration stopping condition is met.

[0012] A second aspect of the present invention provides a device for detecting loading violations, comprising: a data acquisition module for acquiring an original video sample set and labeling the original video sample set to obtain a labeled video sample set; a data augmentation module for performing data augmentation on the labeled video sample set to obtain a target augmented video sample set; a model building module for constructing an initial violation detection model and performing backbone network optimization and attention mechanism introduction on the initial violation detection model to obtain an optimized violation detection model; a parameter setting module for setting the model interface parameters of the optimized violation detection model; a model training module for performing multiple rounds of iterative training on the optimized violation detection model based on the model interface parameters and the target augmented video sample set to obtain a target violation detection model; and a detection module for acquiring a loading scene video to be identified, inputting the loading scene video to be identified into the target violation detection model for violation detection, and obtaining a detection result.

[0013] A third aspect of the present invention provides a vehicle loading violation detection device, the vehicle loading violation detection device comprising: a memory and at least one processor, the memory storing instructions; the at least one processor calling the instructions in the memory to cause the vehicle loading violation detection device to perform each step of the vehicle loading violation detection method described above.

[0014] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the steps of the loading violation detection method described in any of the preceding claims.

[0015] In the technical solution of this invention, firstly, an original video sample set is acquired, and then labeled to obtain a labeled video sample set. Next, data augmentation processing is performed on the labeled video sample set to obtain a target augmented video sample set. An initial violation detection model is constructed, and the initial violation detection model undergoes backbone network optimization and attention mechanism introduction to obtain an optimized violation detection model. Then, model interface parameters of the optimized violation detection model are set, and the optimized violation detection model is iteratively trained multiple times based on the model interface parameters and the target augmented video sample set to obtain a target violation detection model. Finally, a video of the loading scene to be identified is acquired, and the video of the loading scene to be identified is input into the target violation detection model for violation detection to obtain the detection result. This approach aims to reduce reliance on manual monitoring, lower labor costs and monitoring workload, improve the efficiency of logistics loading safety management, and enable real-time detection to promptly detect and handle forklift violations entering the truck compartment, effectively maintaining the safe operation of logistics companies and ensuring the safety of staff and the integrity of goods. Attached Figure Description

[0016] Figure 1 This is a first flowchart of a method for detecting loading violations provided in an embodiment of the present invention; Figure 2 This is a second flowchart of the loading violation detection method provided in the embodiments of the present invention; Figure 3 This is a third flowchart of the loading violation detection method provided in the embodiments of the present invention; Figure 4 This is a fourth flowchart of the loading violation detection method provided in the embodiments of the present invention; Figure 5 This is a fifth flowchart of the method for detecting loading violations provided in this embodiment of the invention; Figure 6 This is a sixth flowchart of the method for detecting loading violations provided in this embodiment of the invention; Figure 7 This is the seventh flowchart of the method for detecting loading violations provided in this embodiment of the invention; Figure 8 This is a schematic diagram of the structure of the vehicle loading violation detection device provided in an embodiment of the present invention; Figure 9 This is a schematic diagram of the structure of the vehicle loading violation detection equipment provided in an embodiment of the present invention. Detailed Implementation

[0017] This invention provides a method, apparatus, device, and storage medium for detecting loading violations. In this invention, the terms "first," "second," "third," "fourth," etc. (if applicable) in the specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0018] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 One embodiment of the loading violation detection method of the present invention includes: 101. Obtain the original video sample set, and perform annotation processing on the original video sample set to obtain an annotated video sample set; In this embodiment, the original video sample set is collected from collection points covering the truck entrance area, forklift operation aisles, and cargo stacking areas. Monitoring equipment is deployed at these collection points, and a continuous collection cycle is set to fully cover loading scenarios under different time periods, weather conditions, and operational intensities, ensuring scene diversity in the samples. Each video sample completely contains the entire loading operation process. Subsequently, invalid segments that make key targets unrecognizable due to blurry images, camera shake, or abnormal lighting are further removed, retaining valid samples with stable images and clear target features. This ultimately constitutes a high-quality original video sample set, providing a reliable foundation for annotation work. The diverse scene coverage effectively supports the model's adaptive training in complex environments. The annotation processing of the original video sample set combines the core requirements of loading safety detection, annotating the forklift movement direction and corresponding behavior category, clearly distinguishing between normal loading operations and abnormal behaviors such as forklifts illegally entering the truck compartment. This ultimately forms an annotated video sample set. The accurate annotation information provides rich and effective supervision signals for the subsequent training of the violation detection model, significantly improving the model's recognition accuracy of key targets such as forklifts and truck entrances, as well as the efficiency and accuracy of violation judgment.

[0019] 102. Perform data augmentation processing on the labeled video sample set to obtain the target augmented video sample set; In this embodiment, multi-dimensional enhancement operations are simultaneously implemented for the valid video segments and corresponding annotation information in the labeled video sample set. Specifically, a random cropping strategy is used to randomly select sub-regions within the valid area of ​​the video frame to ensure the complete preservation of key targets such as forklift and van entrances, while updating the bounding box annotation coordinates of the corresponding targets; horizontal flipping is used to simulate different orientations during forklift operation, improving the model's adaptability to changes in target orientation; brightness and contrast adjustments are implemented to simulate image features under different lighting conditions (such as backlighting in the morning, insufficient lighting at night, etc.); a Gaussian noise addition strategy is introduced to simulate environmental interference during the imaging process of monitoring equipment, enhancing the model's robustness to noisy data; and a ruler is used... The scaling process enlarges or reduces video frames according to a preset ratio, covering the size differences of forklifts at different operating distances. Simultaneously, it calibrates the target position and size parameters in the annotation information, ultimately forming a target augmentation video sample set with sufficient sample quantity and comprehensive scene coverage. Through a combination of multiple strategies for data augmentation, the total number of samples is significantly expanded, effectively alleviating the overfitting problem caused by insufficient samples during model training. In addition, the data augmentation process covers various complex situations that may exist in logistics loading scenarios, such as changes in lighting, differences in target orientation, changes in distance and size, and environmental interference. This greatly improves the scene diversity and feature richness of the target augmentation video sample set, helping the model learn more generalized target feature representations.

[0020] 103. Construct an initial violation detection model, and perform backbone network optimization and attention mechanism introduction on the initial violation detection model to obtain an optimized violation detection model; In this embodiment, the initial violation detection model is constructed using VGG16 (Visual Geometry Group 16-layer network) or MobileNet as the basic backbone network. The model's basic architecture is built upon the mature feature extraction capabilities of these networks, providing core feature extraction support for subsequent violation detection. Then, by removing the fully connected layers originally used for image classification in the basic backbone network, and retaining the convolutional and pooling layers with strong feature extraction capabilities, the drawback of fully connected layers disrupting the feature map spatial structure is avoided. This also effectively simplifies the model parameter scale, reduces computational overhead, and adapts to the real-time detection requirements of logistics loading scenarios. Based on this, an attention mechanism is introduced. Combining the core objective of loading safety detection, key feature layers highly correlated with forklift and van entrance detection in the model are accurately selected as attention enhancement objects. By introducing the Squeeze-and-Excitation module in SENet (Squeeze Excitation Network), feature optimization is achieved, ultimately resulting in an optimized violation detection model. The introduction of the attention mechanism strengthens the feature expression of the core detection target through targeted enhancement of key feature layers, effectively improving the model's recognition accuracy of key targets such as forklift and van entrances, and reducing the interference of complex backgrounds on the detection results.

[0021] 104. Set the model interface parameters of the optimized violation detection model; In this embodiment, the input image size parameters are standardized based on the resolution differences of the actual acquired videos, taking into account the interface parameters of the model input layer. For example, they are uniformly set to 300×300 pixels or 512×512 pixels. This parameter configuration ensures the consistency of the data format of the input model and achieves a dynamic balance between detection accuracy and computing speed. The 300×300 pixel size meets the requirements of high real-time detection, while the 512×512 pixel size is suitable for scenarios with higher detection accuracy requirements. For the model output layer interface parameters, based on the common size range of forklifts and the specifications of van entrances in logistics scenarios, output anchor frame size parameters with various aspect ratios such as 1:1, 1:2, and 2:1 are configured. Through multi-dimensional anchor frame size parameter settings, it is ensured that the model can comprehensively cover the differences in the appearance of forklifts under different operating distances and placement angles, improving the adaptability to core detection targets.

[0022] 105. Based on the model interface parameters and the target enhanced video sample set, the optimized violation detection model is trained in multiple rounds of iterations to obtain the target violation detection model; In this embodiment, when conducting multi-round iterative training of the optimized violation detection model based on model interface parameters and a target-enhanced video sample set, a systematic configuration of training hyperparameters is required to ensure the stability and effectiveness of the training process. Specifically, stochastic gradient descent is selected as the model optimizer. The initial learning rate is set to 0.001, the momentum coefficient is configured to 0.9 to accelerate the convergence of the gradient descent process, and the weight decay coefficient is set to 0.0005 to suppress overfitting of model parameters. The batch size is flexibly configured to 8 or 16 based on actual computing resources, and the number of training rounds is set to 100 to ensure the model fully learns the sample features. An adaptive strategy is adopted for learning rate adjustment. Every 20 training rounds, the learning rate is decayed to one-tenth of its original value. Simultaneously, the validation set loss is monitored in real time. If the validation set loss does not decrease significantly for five consecutive training rounds, the learning rate decay is terminated. This dynamic adjustment mechanism balances the model convergence speed with the risk of overfitting. Furthermore, to address the imbalance in the distribution of positive and negative samples during training, a positive-negative sample balancing mechanism and a difficult sample mining strategy are introduced. In each batch of training samples, the top 25% of difficult negative samples with the highest predicted loss are selected. These are then used to construct training sample pairs with a 1:3 ratio of positive samples to difficult negative samples, ensuring that the model focuses on learning the features of samples that are difficult to distinguish, thus improving its ability to detect borderline violations. The loss function is a weighted sum of classification and regression losses, with both losses set to a weight of 1. The total loss gradient is propagated backward to each layer of the model using a backpropagation algorithm, enabling iterative updates of the model parameters. Through multiple rounds of iterative training and periodic evaluation and optimization, the target violation detection model is finally obtained.

[0023] 106. Obtain the video of the loading scene to be identified, input the video of the loading scene to be identified into the target violation detection model to detect violations, and obtain the detection results.

[0024] In this embodiment, after acquiring the video of the loading scene to be identified, targeted preprocessing is required to ensure compatibility with the target violation detection model. Specifically, video images are extracted frame by frame, and the size of each frame is adjusted to match the model's input size. Simultaneously, normalization is performed to eliminate the interference of pixel value differences on detection accuracy. After preprocessing, each frame is sequentially input into the target violation detection model. Through the model's forward propagation operation, the core information of all detected targets in each frame is output, covering the target category, confidence level, and the coordinates of the target's bounding box, thus forming the initial detection results. To improve the reliability of the detection results, the initial detection results are filtered based on preset rules, removing low-confidence detection information with a confidence level below 0.7 and retaining only high-confidence target data, effectively filtering out low-confidence detection information. Simultaneously, based on the pre-calibrated coordinates of the van entrance area, the overlap area between the forklift target's circumscribed rectangle and the van entrance area is calculated. If the overlap area exceeds 50% of the forklift target's own area, the image frame is determined to contain a forklift illegally entering the van. The pre-calibrated van entrance area coordinates refer to the rectangular boundary pixel coordinates obtained by digitally calibrating the physical area of ​​the van entrance at the van's parking position based on the pixel coordinate system of the monitoring screen after the logistics loading site monitoring equipment is installed and debugged. In addition, to avoid false alarms caused by misdetection of single-frame images, a multi-frame verification mechanism is further set up. If the same forklift illegally entering the van is detected in three consecutive frames, the violation event is confirmed. At this time, the final detection result includes information such as the type of violation, the time of occurrence, the location of occurrence, the target characteristics, and a real-time screenshot, which simultaneously triggers the system's early warning response. Through the dual methods of audible and visual alarms and monitoring center prompts, real-time feedback on the violation event is achieved, helping supervisors to quickly handle the situation and effectively improving the timeliness and effectiveness of safety supervision in the logistics loading process, providing strong protection for on-site operational safety.

[0025] Please see Figure 2 In this embodiment of the invention, the original video sample set includes multiple video samples, and each video sample includes multiple video frames; the annotation process of the original video sample set to obtain an annotated video sample set includes: 201. Use a labeling tool to label the forklift position in each of the video frames to obtain multiple initial labeled video samples. Each initial labeled video sample includes a rectangle corresponding to each of the video frames. 202. Determine the vehicle speed and vehicle direction of motion corresponding to each of the initial labeled video samples based on the multiple bounding boxes; 203. Determine the forklift behavior category corresponding to each of the initially labeled video samples based on the vehicle's moving speed and direction of movement; 204. Using the annotation tool, perform behavior category annotation processing on each of the initial annotated video samples based on multiple forklift behavior categories to obtain the annotated video sample set.

[0026] In this embodiment, when labeling each video frame with a labeling tool, the labeling requirements for violation detection in logistics loading scenarios are considered. Detailed labeling is performed using frame-by-frame labeling or keyframe-interval labeling. The spatial position of the forklift in each video frame is precisely defined by drawing a bounding rectangle around the forklift, resulting in multiple initial labeled video samples. Each initial labeled video sample contains forklift position rectangle information corresponding to a specific video frame. When determining the vehicle's speed and direction of movement based on these rectangles, the coordinate data of the forklift rectangles in adjacent video frames is extracted, and the vehicle's direction of movement is determined based on the coordinate change trend. Simultaneously, the vehicle's speed is calculated by combining the video frame interval time with the displacement distance of the forklift rectangles between adjacent frames, using the ratio of displacement to time. Based on this, and combined with the pre-calibrated coordinates of the van entrance area, the spatial relationship between the forklift rectangle and the van entrance area is determined. By comprehensively considering the vehicle's moving speed, direction of movement, and relative position of the forklift and the van entrance, the forklift behavior category corresponding to each initially labeled video sample is accurately determined, clearly distinguishing between normal loading behavior and forklift unauthorized entry into the van. The pre-calibrated coordinates of the van entrance area refer to the rectangular boundary pixel coordinates obtained by digitally calibrating the physical area of ​​the van entrance at the van parking position based on the pixel coordinate system of the monitoring screen after the installation and debugging of the monitoring equipment at the logistics loading site. After determining the behavior category, an annotation tool is used to associate the determination result with the initially annotated video samples. Behavior category annotation is performed on each initially annotated video sample, and a corresponding behavior category label is added to each sample. After all annotations are completed, a cross-validation mechanism is introduced. Multiple professional annotators review the same batch of initially annotated video samples and their corresponding behavior category annotation results to correct issues such as bounding box position deviations, vehicle speed calculation errors, vehicle movement direction determination deviations, and inconsistencies in behavior category annotations. This ensures the accuracy, consistency, and completeness of the annotation information, reduces the interference of annotation noise on model training, and makes the final annotated video sample set highly reliable. It can provide high-quality data support for the training of the target violation behavior detection model, helping to improve the model's recognition accuracy and generalization ability for forklift position, movement status, and violation behavior. Finally, an annotated video sample set containing forklift position bounding boxes, vehicle speed, vehicle movement direction, and behavior category labels is obtained.

[0027] Please see Figure 3 In this embodiment of the invention, the step of performing data augmentation processing on the labeled video sample set to obtain the target augmented video sample set includes: 301. The labeled video sample set is randomly cropped using a random region cropping algorithm to obtain the first enhanced video sample set; 302. The first enhanced video sample set is horizontally flipped using a video geometric transformation algorithm to obtain the second enhanced video sample set; 303. The second enhanced video sample set is processed by a pixel adjustment algorithm to adjust the brightness and contrast, resulting in a third enhanced video sample set; 304. The third enhanced video sample set is processed by adding Gaussian noise using a Gaussian noise generation algorithm to obtain the fourth enhanced video sample set; 305. The fourth enhanced video sample set is scaled using a video scale transformation algorithm to obtain the target enhanced video sample set.

[0028] In this embodiment, when using a random region cropping algorithm to randomly crop the labeled video sample set, the cropping area must be limited to the effective range of the video frames to ensure that core detection targets such as forklift and van entrances are completely preserved. Simultaneously, for changes in target positions in the cropped image, the coordinate parameters of the corresponding bounding boxes in the labeled information are updated synchronously to avoid misalignment between the labels and target features due to image cropping, thus obtaining the first enhanced video sample set. Based on the first enhanced video sample set, a video geometric transformation algorithm is used to perform horizontal flipping processing. Mirror transformation simulates the different orientations of the forklift in a loading scenario. During the flipping process, the horizontal mirror mapping of the target bounding box coordinates in the labeled information is adjusted synchronously to ensure accurate matching between the labeled information and image features, generating the second enhanced video sample set. For the second enhanced video sample set, a pixel adjustment algorithm is used to perform brightness and contrast adjustment processing. By dynamically adjusting the range of pixel grayscale values, image features under different lighting conditions such as morning backlight, nighttime lighting, and rainy weather are simulated. This process must ensure a one-to-one correspondence between the labeled content and the adjusted image frames, resulting in the third enhanced video sample set. Based on the third enhanced video sample set, a Gaussian noise generation algorithm is used to add an appropriate amount of Gaussian noise to simulate scenarios such as electronic noise and environmental interference that may exist during the imaging process of monitoring equipment. The integrity and stability of the annotation information are maintained during the noise addition process, forming the fourth enhanced video sample set. Finally, a video scaling algorithm is used to scale the fourth enhanced video sample set, enlarging or reducing video frames proportionally according to a preset ratio to cover the size differences of the forklift at different operating distances. Simultaneously, the position coordinates and size parameters of the target rectangles in the annotation information are precisely calibrated according to the scaling ratio to ensure that the annotation information completely matches the target features of the scaled image. The final target enhanced video sample set is obtained. Through a multi-algorithm cascade enhancement strategy, the number of samples is systematically expanded, effectively alleviating the overfitting problem caused by insufficient sample size during model training and providing sufficient data support for the model to fully learn the features.

[0029] Please see Figure 4 In this embodiment of the invention, the initial violation detection model includes an initial backbone network. The process of optimizing the initial violation detection model by performing backbone network optimization and introducing an attention mechanism to obtain an optimized violation detection model includes: 401. The initial backbone network is subjected to fully connected layer removal processing using a model optimization algorithm to obtain a lightweight violation detection model. The lightweight violation detection model includes an optimized backbone network, which includes multiple key feature layers. 402. Based on the multiple key feature layers, an attention mechanism is introduced into the lightweight violation detection model to obtain the optimized violation detection model.

[0030] In this embodiment, VGG16 or MobileNet, which has feature extraction capabilities, is first used as the initial backbone network architecture. A model optimization algorithm is then used to precisely remove the fully connected layers used for image classification tasks from the original network, retaining the core convolutional and pooling layers to construct an optimized backbone network, thus obtaining a lightweight violation detection model. The optimized backbone network includes multiple key feature layers that achieve multi-scale feature extraction through hierarchical operations of convolutional and pooling layers. Shallow key feature layers capture detailed features of small forklift targets at close range, while deep key feature layers extract global features of large forklift targets at long distances. Simultaneously, detection branches are set at different levels of the feature extraction network to construct classification and regression sub-networks. For each default bounding box, the classification branch outputs three probabilities: background, normal forklift, and illegal forklift, while the regression branch outputs the positional offset of the default bounding box relative to the actual target bounding box. This ensures that the lightweight violation detection model possesses both lightweight characteristics to adapt to real-time detection requirements and complete multi-scale detection capabilities. When introducing an attention mechanism into a lightweight violation detection model based on multi-layer key feature layers, the first step is to select feature layers highly correlated with forklift and van entrance detection as the core enhancement objects. The attention mechanism is then integrated by introducing the Squeeze-and-Excitation module from SENet, ultimately resulting in an optimized violation detection model. The introduction of the attention mechanism enhances the core feature channels and suppresses background interference, enabling the optimized violation detection model to focus on key detection targets. This significantly improves the feature perception capability of core targets such as forklift and van entrances, reduces the interference of complex backgrounds on detection results, and enhances the accuracy of violation detection.

[0031] Please see Figure 5In this embodiment of the invention, the step of introducing an attention mechanism into the lightweight violation detection model based on multiple key feature layers to obtain the optimized violation detection model includes: 501. The response analysis algorithm is used to filter the vehicle entrance detection correlation of the multiple key feature layers to obtain the key feature layer to be enhanced, and the attention mechanism is introduced into the key feature layer to be enhanced to obtain the feature layer to be pooled. 502. The feature layer to be pooled is subjected to global average pooling using a pooling algorithm to obtain a global average pooled feature vector; 503. Based on the Sigmoid activation function, calculate the weight coefficients for each feature channel of the global average pooling feature vector to obtain the weight coefficients corresponding to each feature channel; 504. Element-wise multiplication is performed on each of the feature channels based on the multiple weight coefficients to obtain an optimized feature layer; 505. The optimized feature layer is fused into the lightweight violation detection model using a feature fusion algorithm to obtain the optimized violation detection model.

[0032] In this embodiment, when using the response analysis algorithm to screen the correlation between vehicle entrance detection and multiple key feature layers, feature layers highly correlated with forklift and van entrance detection are selected as core enhancement objects, resulting in the key feature layers to be enhanced. This avoids ineffective enhancement of redundant feature layers, significantly improves the targeting and efficiency of the attention mechanism introduction, and reduces the model's computation time. Subsequently, a squeezing excitation module is introduced to implement the attention mechanism introduction process on the key feature layers to be enhanced, completing the initial integration of the module and the feature layers, resulting in the feature layers to be pooled. A pooling algorithm is then used to perform global average pooling on the feature layers to be pooled. This process corresponds to the compression stage of the squeezing excitation module. By calculating the global mean of the two-dimensional spatial information of each feature channel, it is compressed into a one-dimensional value that can represent the global response of the channel, forming a global average pooled feature vector, realizing the transformation of feature information from the spatial dimension to the channel dimension.

[0033] Next, for the globally average pooled feature vector, a bottleneck structure containing two fully connected layers is constructed to perform feature dimension transformation and nonlinear enhancement processing. The first fully connected layer reduces the feature dimension to one-sixteenth of the original dimension and enhances the nonlinear expressive power of the features using the ReLU activation function. The second fully connected layer restores the feature dimension to the initial dimension, and then operates on the transformed feature vector based on the Sigmoid activation function to obtain weight coefficients corresponding to each feature channel. The magnitude of the weight coefficients directly reflects the contribution of the corresponding feature channel to the detection of forklift violation into the van. Subsequently, element-wise multiplication is performed on each feature channel based on the multiple weight coefficients. By performing element-wise multiplication of the weight coefficients with the features of each channel of the original key feature layer to be enhanced, adaptive enhancement of key feature channels such as forklift outline and van entrance edge is achieved, while suppressing interference from irrelevant feature channels such as background, thus obtaining an optimized feature layer with significantly improved feature expressive power. Finally, a feature fusion algorithm is used to fuse the optimized feature layer into the lightweight violation detection model, completing the integration of the attention mechanism and the lightweight model architecture, and finally obtaining the optimized violation detection model.

[0034] Please see Figure 6 In this embodiment of the invention, setting the model interface parameters of the optimized violation detection model includes: 601. Perform resolution statistical processing on the labeled video sample set using statistical algorithms to obtain sample resolution data; 602. Determine and set the input size parameters of the optimized violation detection model based on the sample resolution data; 603. Obtain vehicle appearance morphology features based on the labeled video sample set, and determine and set the output anchor frame size parameters of the optimized violation detection model based on the vehicle appearance morphology features.

[0035] In this embodiment, when performing resolution statistical processing on the labeled video sample set using statistical algorithms, the resolution information of all video frames in the sample set is systematically statistically analyzed to identify the distribution characteristics, proportions, and core resolution ranges of different resolutions, thereby obtaining sample resolution data that reflects the characteristics of the sample resolution. When determining and optimizing the input size parameters of the violation detection model based on the sample resolution data, the core objective is to adapt to the real-time detection needs of logistics loading scenarios. Combining the statistically obtained core resolution ranges, a standardized size that balances detection accuracy and computational speed is selected as the input size parameter, typically set to 300×300 pixels or 512×512 pixels. This parameter setting ensures that video frames of different resolutions have a unified format after size adjustment, enabling efficient adaptation to the model's feature extraction network and avoiding the impact of data format differences on the smoothness of the detection process. When acquiring vehicle appearance features based on labeled video sample sets, the size data and shape features of forklifts under different working distances and placement angles are extracted from the labeled information of the samples. At the same time, combined with the specification parameters of the van entrance, the core variation rules and size range of the vehicle appearance are sorted out. Then, based on the vehicle appearance features, the output anchor frame size parameters of the violation detection model are determined and set to optimize. At the same time, multiple width-to-height ratio parameters such as 1:1, 1:2, and 2:1 are set to ensure that the output anchor frame can fully cover various appearance shapes of vehicles and improve the matching degree between the anchor frame and the actual vehicle target.

[0036] Please see Figure 7 In this embodiment of the invention, the step of performing multiple rounds of iterative training on the optimized violation detection model based on the model interface parameters and the target enhanced video sample set to obtain the target violation detection model includes: 701. Obtain the actual detection results corresponding to the target enhanced video sample set; 702. Based on the model interface parameters, input the target enhanced video sample set into the optimized violation detection model to detect violations and obtain the predicted detection results; 703. Calculate the model loss based on the predicted detection results and the actual detection results; 704. Based on the model loss, the backpropagation algorithm is used to iteratively update the weight parameters of the optimized violation detection model; 705. When the preset iteration stopping condition is met, output the target violation detection model.

[0037] In this embodiment, obtaining the actual detection results corresponding to the target augmented video sample set essentially involves extracting the core information already labeled in the target augmented video sample set, including precisely labeled data such as the spatial location of the forklift, behavior category, and vehicle appearance. This data serves as the true benchmark for model training, providing a reference for subsequent loss calculation and parameter optimization. Based on the model interface parameters, when the target augmented video sample set is input into the optimized violation detection model according to a preset batch size to obtain the predicted detection results, the batch size is configured to be 8 or 16 depending on the actual computing power resources. The model will rely on the input size, output anchor box, and other configurations set in the interface parameters to perform feature extraction and multi-scale detection on the input samples, and output the predicted detection results containing the target prediction category, confidence level, and location coordinates. When calculating the model loss based on the predicted and actual detection results, the weighted sum of classification and regression losses is used as the total loss, with both classification and regression losses set to a weight of 1. Simultaneously, a positive-negative sample balancing mechanism is employed. A difficult sample mining strategy selects the top 25% of the most difficult negative samples from each batch, and pairs them with positive samples at a 1:3 ratio to participate in the loss calculation. This ensures that the loss value accurately reflects the model's recognition error for difficult-to-distinguish samples, thus obtaining a model loss that truly characterizes the model's prediction bias. Based on this model loss, the backpropagation algorithm is used to iteratively update the weight parameters of the violation detection model. Stochastic gradient descent is selected as the optimizer, driven by an initial learning rate of 0.001, momentum coefficient of 0.9, and weight decay coefficient of 0.0005. An adaptive learning rate adjustment strategy is also employed, decaying the learning rate to one-tenth of its original value every 20 iterations. If the validation set loss does not decrease significantly after 5 consecutive iterations, the learning rate decay stops, accelerating parameter convergence while suppressing overfitting. When the preset iteration stopping conditions are met, the target violation detection model is output. The iteration stopping conditions include the completion of 100 preset training rounds or the validation set loss continuously not decreasing and reaching a set threshold. This avoids the impact of insufficient or excessive training on model performance. The final output target violation detection model has both high recognition accuracy and strong generalization ability, which can accurately adapt to the actual violation detection needs of logistics loading scenarios and ensure the effectiveness of on-site operation safety supervision.

[0038] The above describes the method for detecting loading violations in the embodiments of the present invention. The following describes the device for detecting loading violations in the embodiments of the present invention. Please refer to [link / reference]. Figure 8 One embodiment of the vehicle loading violation detection device of the present invention includes: Data acquisition module 801: used to acquire the original video sample set, and to perform annotation processing on the original video sample set to obtain an annotated video sample set; Data augmentation module 802: used to perform data augmentation processing on the labeled video sample set to obtain the target augmented video sample set; Model building module 803: used to build an initial violation detection model, and to perform backbone network optimization and attention mechanism introduction on the initial violation detection model to obtain an optimized violation detection model; Parameter setting module 804: Used to set the model interface parameters of the optimized violation detection model; Model training module 805: used to perform multiple rounds of iterative training on the optimized violation detection model based on the model interface parameters and the target enhanced video sample set, so as to obtain the target violation detection model; Detection module 806: Used to acquire a video of the loading scene to be identified, input the video of the loading scene to be identified into the target violation detection model to detect violations, and obtain the detection result.

[0039] Based on the same ideas as the methods in the above embodiments, the apparatus provided in this application can implement the methods in the above embodiments.

[0040] above Figure 8 The vehicle loading violation detection device in this embodiment of the invention is described in detail from the perspective of modular functional entities. The vehicle loading violation detection device in this embodiment of the invention is described in detail below from the perspective of hardware processing.

[0041] Figure 9 This is a schematic diagram of the structure of a vehicle loading violation detection device 900 provided in an embodiment of the present invention. The vehicle loading violation detection device 900 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 910 (e.g., one or more processors) and a memory 920, and one or more storage media 930 (e.g., one or more mass storage devices) storing application programs 933 or data 932. The memory 920 and storage media 930 can be temporary or persistent storage. The program stored in the storage media 930 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the vehicle loading violation detection device 900. Furthermore, the processor 910 may be configured to communicate with the storage media 930 and execute the series of instruction operations in the storage media 930 on the vehicle loading violation detection device 900 to implement the steps of the vehicle loading violation detection method provided in the above-described method embodiments.

[0042] The vehicle loading violation detection device 900 may also include one or more power supplies 940, one or more wired or wireless network interfaces 950, one or more input / output interfaces 960, and / or one or more operating systems 931, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 9 The illustrated structure of the loading violation detection equipment does not constitute a limitation on the loading violation detection equipment. It may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0043] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the loading violation detection method.

[0044] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system, device, or unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0045] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0046] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting loading violations, characterized in that, include: Obtain the original video sample set, and perform annotation processing on the original video sample set to obtain the annotated video sample set; Data augmentation processing is performed on the labeled video sample set to obtain the target augmented video sample set; An initial violation detection model is constructed, and the initial violation detection model is optimized by backbone network processing and attention mechanism introduction processing to obtain an optimized violation detection model. Set the model interface parameters of the optimized violation detection model; The optimized violation detection model is trained iteratively in multiple rounds based on the model interface parameters and the target enhanced video sample set to obtain the target violation detection model. A video of the loading scene to be identified is acquired, and the video of the loading scene to be identified is input into the target violation detection model to detect violations and obtain the detection results.

2. The method for detecting loading violations according to claim 1, characterized in that, The original video sample set includes multiple video samples, and each video sample includes multiple video frames; the annotation process of the original video sample set to obtain an annotated video sample set includes: Each video frame is labeled with a forklift position using a labeling tool to obtain multiple initial labeled video samples. Each initial labeled video sample includes a rectangular box corresponding to each video frame. Based on the multiple bounding boxes, determine the vehicle speed and vehicle direction of motion corresponding to each of the initial labeled video samples; Based on the vehicle's moving speed and direction of movement, determine the forklift behavior category corresponding to each of the initial labeled video samples; Using the aforementioned annotation tool, behavior category annotation processing is performed on each of the initial annotated video samples based on multiple forklift behavior categories to obtain the annotated video sample set.

3. The method for detecting loading violations according to claim 1, characterized in that, The step of performing data augmentation processing on the labeled video sample set to obtain the target augmented video sample set includes: The labeled video sample set is randomly cropped using a random region cropping algorithm to obtain the first enhanced video sample set. The first enhanced video sample set is horizontally flipped using a video geometric transformation algorithm to obtain the second enhanced video sample set; A pixel adjustment algorithm is used to adjust the brightness and contrast of the second enhanced video sample set to obtain a third enhanced video sample set. The third enhanced video sample set is processed by adding Gaussian noise using a Gaussian noise generation algorithm to obtain the fourth enhanced video sample set. The fourth enhanced video sample set is scaled using a video scale transformation algorithm to obtain the target enhanced video sample set.

4. The method for detecting loading violations according to claim 1, characterized in that, The initial violation detection model includes an initial backbone network. The process of optimizing the initial violation detection model by adding an attention mechanism to the backbone network and obtaining an optimized violation detection model includes: A model optimization algorithm is used to remove fully connected layers from the initial backbone network to obtain a lightweight violation detection model. The lightweight violation detection model includes an optimized backbone network, which includes multiple key feature layers. The lightweight violation detection model is processed by introducing an attention mechanism based on the multiple key feature layers to obtain the optimized violation detection model.

5. The method for detecting loading violations according to claim 4, characterized in that, The optimized violation detection model is obtained by introducing an attention mechanism into the lightweight violation detection model based on multiple key feature layers, including: The response analysis algorithm is used to filter the vehicle entrance detection correlation of the multiple key feature layers to obtain the key feature layer to be enhanced, and the attention mechanism is introduced into the key feature layer to be enhanced to obtain the feature layer to be pooled. A pooling algorithm is used to perform global average pooling on the feature layer to be pooled, resulting in a global average pooled feature vector. Based on the Sigmoid activation function, the weight coefficients of each feature channel of the global average pooling feature vector are calculated to obtain the weight coefficients corresponding to each feature channel. Based on the multiple weight coefficients, each feature channel is element-wise multiplied to obtain an optimized feature layer; The optimized feature layer is fused into the lightweight violation detection model using a feature fusion algorithm to obtain the optimized violation detection model.

6. The method for detecting loading violations according to claim 1, characterized in that, The setting of the model interface parameters for the optimized violation detection model includes: Statistical algorithms were used to perform resolution statistical processing on the labeled video sample set to obtain sample resolution data. The input size parameters of the optimized violation detection model are determined and set based on the sample resolution data. Vehicle appearance features are obtained based on the labeled video sample set, and the output anchor frame size parameters of the optimized violation detection model are determined and set based on the vehicle appearance features.

7. The method for detecting loading violations according to claim 1, characterized in that, The process of performing multiple rounds of iterative training on the optimized violation detection model based on the model interface parameters and the target enhanced video sample set to obtain the target violation detection model includes: Obtain the actual detection results corresponding to the target enhanced video sample set; Based on the model interface parameters, the target enhanced video sample set is input into the optimized violation detection model to detect violations and obtain the predicted detection results; The model loss is calculated based on the predicted detection results and the actual detection results. Based on the model loss, the backpropagation algorithm is used to iteratively update the weight parameters of the optimized violation detection model; When the preset iteration stopping condition is met, the target violation detection model is output.

8. A device for detecting loading violations, characterized in that, include: Data acquisition module: used to acquire the original video sample set, and to perform annotation processing on the original video sample set to obtain an annotated video sample set; Data augmentation module: used to perform data augmentation processing on the labeled video sample set to obtain the target augmented video sample set; Model building module: used to build an initial violation detection model, and to perform backbone network optimization and attention mechanism introduction on the initial violation detection model to obtain an optimized violation detection model; Parameter setting module: Used to set the model interface parameters of the optimized violation detection model; Model training module: used to perform multiple rounds of iterative training on the optimized violation detection model based on the model interface parameters and the target enhanced video sample set, so as to obtain the target violation detection model; Detection module: used to acquire video of the loading scene to be identified, input the video of the loading scene to be identified into the target violation detection model to detect violations, and obtain the detection results.

9. A device for detecting loading violations, characterized in that, The vehicle loading violation detection device includes: a memory and at least one processor, wherein the memory stores instructions; At least one of the processors invokes the instructions in the memory to cause the vehicle loading violation detection device to perform the steps of the vehicle loading violation detection method as described in any one of claims 1-7.

10. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement each step of the loading violation detection method as described in any one of claims 1-7.

Images