Factory vehicle violation intelligent detection method and device based on dynamic trend sensing and guiding

Abstract

The invention discloses an intelligent factory vehicle violation detection method and device based on dynamic trend perception and guidance. The method comprises the following steps: performing data preprocessing and enhancement on an obtained factory loading and unloading area monitoring image; performing multi-scale feature extraction on the preprocessed data set by using a RetinaNet framework, sensing a loss function through a dynamic trend, and adaptively adjusting a loss weight according to the dynamic change of sample difficulty so as to optimize the learning process of the model; a multi-scale boundary guidance module is adopted to optimize edge information of a target vehicle, and a graph convolutional network (GCN) is introduced to realize accurate detection of vehicle violation behaviors under a complex background. Compared with the prior art, the method has the advantages that the robustness and the real-time performance of the detection system are effectively improved by dynamically adjusting the loss weight and performing multi-scale optimization on the vehicle boundary information, and the method is suitable for vehicle management and violation detection in complex environments such as factory loading and unloading areas.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent detection and industrial safety management, and in particular relates to a method and device for intelligent detection of factory vehicle violations based on dynamic trend perception and guidance. Background Art

[0002] In the operation of modern factories, the loading and unloading area is a key link in the material inflow and outflow and logistics distribution, and efficient and standardized management is crucial. With the improvement of factory automation and intelligence, the irregular operation of vehicles in the loading and unloading area may lead to a series of problems, such as safety accidents, reduced logistics efficiency, equipment damage, etc.

[0003] Traditional management methods mainly rely on manual monitoring and recording, which is not only time-consuming and labor-intensive, but also has the following significant disadvantages: 1. Limitations of manual monitoring: The traditional method of relying on manual monitoring not only requires a lot of manpower input, but is also easily affected by factors such as fatigue and subjective judgment of the monitor, resulting in missed detection or misjudgment from time to time, and cannot guarantee all-weather, high-precision monitoring effects. For example, some factory reports show that the missed detection rate of manual monitoring is as high as 20%; 2. Challenges of complex backgrounds: In the loading and unloading area of ​​the factory, there are various types of vehicles and complex backgrounds, which may include various equipment, personnel and material stacking. These complex background information makes it difficult for traditional detection methods based on rules or fixed algorithms to effectively distinguish and identify vehicle violations; 3. High real-time requirements: In the actual operation of the loading and unloading area, timely detection and handling of vehicle violations is the key to ensuring safety and efficiency. However, existing technologies are difficult to take into account both real-time and accuracy in complex scenarios, and cannot meet the factory's needs for immediate response.

[0004] Based on the above shortcomings, the application effect of existing intelligent detection technology in complex industrial environments, especially in factory loading and unloading areas, is not satisfactory. There is a need for a method that can realize efficient, accurate and real-time vehicle violation detection in complex factory loading and unloading areas, so as to improve the level of intelligent factory management and ensure production safety and logistics efficiency. Summary of the invention

[0005] Purpose of the invention: In view of the above problems, the present invention provides a method and device for intelligent detection of factory vehicle violations based on dynamic trend perception and guidance. Based on dynamic trend perception and multi-scale boundary guidance, by introducing the dynamic trend perception loss function, the learning process of the detection model is adaptively optimized, thereby improving the model's sensitivity to vehicle violations in complex scenes. At the same time, combined with the multi-scale boundary guidance technology, the edge information of the target vehicle is accurately extracted and processed, thereby enhancing the detection accuracy of the system in complex backgrounds. The invention can effectively solve many challenges in the detection of vehicle violations in factory loading and unloading areas, and realize real-time, accurate and efficient intelligent detection.

[0006] Technical solution: The present invention discloses an intelligent detection method for factory vehicle violations based on dynamic trend perception and guidance, comprising the following steps:

[0007] Step 1: Preprocess the pre-acquired dataset D1 of vehicle violations in the factory loading and unloading area to obtain the initial dataset D2;

[0008] Step 2: Use the RetinaNet detection model on the preprocessed dataset D2 and extract feature maps of multiple scales through the feature pyramid network FPN;

[0009] Step 3: Apply the dynamic trend-aware loss function, dynamically adjust the loss weight according to the sample difficulty during training, adaptively optimize the learning process of the RetinaNet detection model, obtain the optimized RetinaNet detection model, and generate the enhanced feature map;

[0010] Step 4: Based on the dynamically optimized feature map, the multi-scale boundary guidance module is used to accurately extract the edge information of the target vehicle, and the graph convolutional network GCN is introduced to process the edge information, enhance the spatial correlation of the edge features, and generate the final optimized edge feature map;

[0011] Step 5: Perform vehicle violation detection and bounding box regression on the edge-optimized feature map and output the vehicle violation detection results.

[0012] Furthermore, the specific method of step 1 is:

[0013] Step 1.1: Preprocess the images in the acquired data set D1 by translation and scaling, flip transformation, and image enhancement;

[0014] Step 1.2: Use the label annotation tool LabelImg software to annotate the images in dataset D2, marking the location, type, and corresponding violation of the vehicle.

[0015] Furthermore, the specific method of step 2 is:

[0016] Step 2.1: Load and initialize the RetinaNet detection model and set the basic parameters of the model;

[0017] Step 2.2: Configure the feature pyramid network FPN and define the number of input channels and output channels of each level of feature map;

[0018] Step 2.3: Input the preprocessed images in the dataset D2 into the RetinaNet detection model, and after the forward propagation of the ResNet network, the images are respectively from C 3 , C 4 , C5 Layer extracts feature maps;

[0019] Step 2.4: Use FPN to process the input image and generate feature maps P of multiple scales 1 , P 2 , P 3 .

[0020] Furthermore, the specific method of dynamically adjusting the loss weight in step 3 to adaptively optimize the learning process of the RetinaNet detection model is:

[0021] Step 3.1: Extract three multi-scale feature maps P from the FPN of the RetinaNet detection model 1 , P 2 , P 3 , set the loss trend change threshold ε 1 , the threshold of the total loss after dynamic adjustment ε 2 ;

[0022] Step 3.2: For each extracted feature map P 1 , P 2 , P 3 , using the cross entropy function Calculate the classification loss value and Smooth L1 loss function Calculate the regression loss value, where y i represents the i-th true label, represents the i-th predicted label, N is the total number of samples, SmoothL1 represents the Smooth L1 loss function, and b i represents the parameters of the i-th ground-truth bounding box, represents the parameters of the i-th predicted bounding box;

[0023] Step 3.3: Calculate the loss change between the current training batch and the previous batch, and calculate the loss trend change. The change calculation formula is: ΔL trend =L current -L previous , L current Represents the total loss of the current batch, L previous Indicates the total loss of the previous batch;

[0024] Step 3.4: If ΔL trend >ε 1 , then execute step 3.5, otherwise jump to step 3.6;

[0025] Step 3.5: Use the dynamic trend perception loss function to optimize the RetinaNet detection model to obtain the optimized RetinaNet detection model, and dynamically adjust the weights of the classification and regression losses of the feature graph. The specific formula is: Among them, L dynamic represents the total loss after dynamic adjustment, α(t), β(t), γ(t) represent dynamic weight coefficients, λ and μ represent regularization coefficients, and Represents the square of the gradient of classification and regression loss with respect to the model parameter k respectively;

[0026] Step 3.6: If L dynamic ≤ε 2 , indicating that the optimization of the RetinaNet detection model has achieved the expected effect, then execute 3.7, otherwise jump to step 3.5;

[0027] Step 3.7: Use the dynamically adjusted loss function for backpropagation, calculate the gradient, update the model parameters, and generate the optimized feature map.

[0028] Furthermore, in step 4, the specific method of using the multi-scale boundary guidance module to perform edge optimization and introduce the graph convolutional network GCN is:

[0029] Step 4.1: Input the optimized feature map set {P 1 ',P 2 ',P 3 '}, set the edge extraction threshold θ e , gradient threshold θ g , the number of graph convolutional network layers L, the number of iterations T, the learning rate η, the activation function α, the feature difference threshold ξ, the feature vector x of nodes i and j i and j , the bandwidth parameter σ of the node;

[0030] Step 4.2: For each feature map P i ', set the high and low thresholds of the Canny algorithm according to different feature map levels, use the Canny edge detection algorithm to extract edge features, and obtain the corresponding edge map E i ;

[0031] Step 4.3: Construct the initial graph structure and define the graph structure G = (V, E), where the node set V is composed of pixels in the edge graph, the edge set E is established based on the Euclidean distance and similarity between pixels, and the weight ω between each pair of adjacent nodes is calculated. ij , the formula is:

[0032] Step 4.4: Define the loop variable t and assign an initial value of 1;

[0033] Step 4.5: If t≤L, execute steps 4.6-4.8, otherwise jump to step 4.9;

[0034] Step 4.6: For each node v i , calculate the feature representation of the tth layer, the formula is: Where N(i) represents the node v i The neighbor node set of b (t) is the bias term;

[0035] Step 4.7: For each node v i Feature representation of the tth layer Normalization is performed, and the formula is: in, Represents node v i The feature representation of the tth layer is Represents node v i The new normalized feature representation after the tth layer iteration;

[0036] Step 4.8: Update the loop variable, increase the value of t by 1, and return to step 4.5 to continue iterating;

[0037] Step 4.9: Calculate feature differences: If Δh≤ξ, execute step 4.10, otherwise jump to step 4.6, where N represents the total number of nodes participating in the calculation;

[0038] Step 4.10: The node features {h i (L)}Map back to the corresponding spatial position to form the aggregated edge feature map E a pt;

[0039] Step 4.11: E a Each pixel value in pt is thresholded: if the pixel value e i ≥θ e , then retain the pixel if the pixel value e i <θ e , set the pixel value to 0;

[0040] Step 4.12: For each original feature map P i ', perform fusion operation: P″ i =P i ′+υ*E a pt,υ is the fusion coefficient, which is dynamically adjusted according to the importance of different scale layers to control the weight of edge features in the fusion result;

[0041] Step 4.13: For the fused feature map P″i Perform Gaussian smoothing to obtain the smoothed feature map P i ”', which is the edge feature map;

[0042] Step 4.14: Calculate the smoothed feature map P i ”' Gradient

[0043] Step 4.15: If Then execute step 4.16, otherwise jump to step 4.12;

[0044] Step 4.16: For each smoothed feature map P i ”'Perform regularization and map pixel values ​​to the interval [0,1];

[0045] Step 4.17: Define loop variable n and assign an initial value of 1;

[0046] Step 4.18: If n≤T, execute steps 4.19-4.21, otherwise jump to step 4.22;

[0047] Step 4.19: Calculate the loss function L for the current iteration (n) , based on the smoothed feature map P i ”'The difference between the target value and the target truth value;

[0048] Step 4.20: Update the model parameters using gradient descent: Among them, k (n) represents the model parameters before the nth iteration, k (n+1) represents the model parameters after the n+1th iteration, Represents the gradient of the loss function L with respect to the model parameter k;

[0049] Step 4.21: Increase the value of n by 1 and return to step 4.18 to continue iterating;

[0050] Step 4.22: Use the optimized edge feature map set {P 1 ”',P 2 ”',P 3 ”'}, the final vehicle violation detection results are generated through the object detector.

[0051] Furthermore, the specific method of step 5 is:

[0052] Step 5.1: Input the optimized edge feature map set {P 1 ”',P 2 ”',P 3 ”'} and the optimized RetinaNet model;

[0053] Step 5.2: Set the optimized edge feature map {P 1 ”',P 2 ”',P 3 ”'} is directly applied to the optimized RetinaNet model for target detection, and performs preliminary feature processing and convolution operations to generate a preliminary candidate region set R 0 ;

[0054] Step 5.3: For the candidate region R 0 Perform bounding box regression to obtain the initial predicted bounding box set B 0 ;

[0055] Step 5.4: For B 0 Perform target detection and classification operations on each bounding box in and generate a confidence score S corresponding to each bounding box 0 and category label C 0 ;

[0056] Step 5.5: If S 0 >θ s ,θ s If it is the confidence threshold, execute steps 5.6-5.8, otherwise jump to step 5.9;

[0057] Step 5.6: For all predicted bounding boxes B 0 Calculate the overlap IoU;

[0058] Step 5.7: If IoU>θ o ,θ o If it is the overlap threshold, execute step 5.8, otherwise execute step 5.9;

[0059] Step 5.8: Keep the bounding box with the highest confidence and discard the remaining bounding boxes with high overlap;

[0060] Step 5.9: Generate the bounding box set and category label after NMS processing as the final detection result output.

[0061] The present invention also discloses an intelligent detection device for vehicle violations in a factory loading and unloading area, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, the intelligent detection method for vehicle violations in a factory based on dynamic trend perception and guidance is implemented.

[0062] Beneficial effects:

[0063] 1. The present invention can adaptively adjust the learning process of the model in complex factory loading and unloading area scenarios by introducing a dynamic trend perception loss function, thereby improving the accuracy and robustness of vehicle violation detection. The loss function dynamically allocates weights according to sample difficulty during training, so that the model can still maintain a high detection accuracy when facing diverse backgrounds, changes in lighting conditions and vehicle types.

[0064] 2. The present invention improves the RetinaNet detection model, combines low-level detail features with high-level semantic features using multi-scale boundary guidance technology, and improves the ability to accurately extract the edge of the target vehicle by optimizing edge information layer by layer. This boundary guidance technology can effectively solve the edge blur problem in complex backgrounds and significantly improve the accuracy of detection.

[0065] 3. In the edge optimization process, the present invention introduces a graph convolutional network (GCN) to enhance the spatial correlation of edge features. This improvement allows edge information to remain coherent in high-dimensional space, ensuring accurate detection of vehicle contours in complex scenarios and reducing false detections caused by loss of edge features. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is the overall flow chart of the present invention;

[0067] Figure 2 Flowchart for multi-scale feature extraction;

[0068] Figure 3 Optimize the flow chart for dynamic trend-aware loss function;

[0069] Figure 4 Flowchart for multi-scale boundary guidance and edge optimization;

[0070] Figure 5 Flowchart for vehicle violation detection and bounding box regression. DETAILED DESCRIPTION

[0071] The present invention is further explained below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0072] The present invention discloses a method and device for intelligent detection of factory vehicle violations based on dynamic trend perception and guidance, aiming to solve the accuracy and real-time problems of vehicle violation detection in complex backgrounds in factory loading and unloading areas. The method optimizes the detection model by introducing a dynamic trend perception loss function, and combines the multi-scale boundary guidance technology for edge optimization, thereby improving the accuracy and robustness of vehicle violation detection. First, the vehicle operation video or image data in the factory loading and unloading area is collected to form an original data set D1. Necessary preprocessing operations such as image cropping, rotation and brightness adjustment are performed on D1 to meet subsequent processing requirements, and an optimized data set D2 is obtained. The RetinaNet framework is used in combination with a feature pyramid network (FPN) to conduct an in-depth analysis of D2, extract key visual features from different levels, and generate multi-scale feature maps. These feature maps can capture multi-level information from details to abstractions, providing a basis for subsequent dynamic optimization. On the basis of feature extraction, a dynamic trend perception loss function is introduced to adjust the learning process of the RetinaNet detection model to obtain an optimized RetinaNet detection model. The loss weight is dynamically adjusted according to real-time training feedback to optimize the responsiveness and sensitivity of the algorithm. In this way, the model's ability to identify vehicle behavior in complex factory scenarios is enhanced. Based on the dynamically optimized feature map, a multi-scale boundary guidance module is used to accurately extract the edge information of the target vehicle. Through the graph convolutional network (GCN), the spatial correlation of edge features is enhanced, significantly improving the accuracy and robustness of edge detection. Finally, the edge-optimized feature map is input into the optimized RetinaNet detection model to analyze vehicle violations at the terminal. The model performs bounding box regression and classification on each detected vehicle to ensure that violations are accurately marked and reported.

[0073] Step 1: Preprocess the pre-acquired dataset D1 of vehicle violations in the factory loading and unloading area to obtain the initial dataset D2.

[0074] Step 1.1: The images in the acquired data set D1 are preprocessed by translation and scaling, flip transformation, and image enhancement.

[0075] Step 1.2: Use the label annotation tool LabelImg software to annotate the images in dataset D2, marking the location, type, and corresponding violation of the vehicle.

[0076] Step 2: Use the RetinaNet framework on the dataset D2 to extract feature maps of multiple scales through the feature pyramid network (FPN), as follows: Figure 2 As shown:

[0077] Step 2.1: Load and initialize the RetinaNet model and set the basic parameters of the model.

[0078] Step 2.2: Configure the Feature Pyramid Network (FPN) and define the number of input channels and output channels of each level of feature map.

[0079] Step 2.3: Input the preprocessed images in the dataset D2 into the RetinaNet model, and after the forward propagation of the ResNet model, 3 , C 4 , C 5 Layer extracts feature maps.

[0080] Step 2.4: Use FPN to process the input image and generate feature maps P of multiple scales 1 , P 2 , P 3 .

[0081] Step 3: Apply the dynamic trend-aware loss function, dynamically adjust the loss weight according to the sample difficulty during training, adaptively optimize the learning process of the RetinaNet detection model, and generate enhanced feature maps. Figure 3 As shown:

[0082] Step 3.1: Extract three multi-scale feature maps P from the FPN of the RetinaNet detection model 1 , P 2 , P 3 , set the loss trend change threshold ε 1 , the threshold of the total loss after dynamic adjustment ε 2 .

[0083] Step 3.2: For each extracted feature map P 1 , P 2 , P 3 , using the cross entropy function Calculate the classification loss value and Smooth L1 loss function Calculate the regression loss value, where y i represents the i-th true label, represents the i-th predicted label, N is the total number of samples, SmoothL1 represents the Smooth L1 loss function, and b i represents the parameters of the i-th ground-truth bounding box, represents the parameters of the i-th predicted bounding box.

[0084] Step 3.3: Calculate the loss change between the current training batch and the previous batch, and calculate the loss trend change. The change calculation formula is: ΔL trend =L current -L previous .

[0085] Step 3.4: If ΔL trend >ε 1 , then execute step 3.5, otherwise jump to step 3.6.

[0086] Step 3.5: Use the dynamic trend perception loss function to optimize the RetinaNet detection model to obtain the optimized RetinaNet detection model, and dynamically adjust the weights of the classification and regression losses of the feature graph. The specific formula is: In , α(t), β(t), and γ(t) represent dynamic weight coefficients, and λ and μ represent regularization coefficients. and They represent the square of the gradient of the classification and regression loss with respect to the model parameter k, respectively.

[0087] Step 3.6: If L dynamic ≤ε 2 , then execute 3.7, otherwise jump to step 3.5.

[0088] Step 3.7: Use the dynamically adjusted loss function for backpropagation, calculate the gradient, update the model parameters, and generate the optimized feature map.

[0089] Step 4: Based on the dynamically optimized feature map, the multi-scale boundary guidance module is used to accurately extract the edge information of the target vehicle, and the graph convolutional network GCN is introduced to process the edge information, enhance the spatial correlation of the edge features, and generate the final optimized edge feature map. Figure 4 As shown:

[0090] Step 4.1: Input the optimized feature map set {P 1 ',P 2 ',P 3 '}, set the edge extraction threshold θ e , gradient threshold θ g , graph convolutional network layer number L, number of iterations T, learning rate η, activation function α, feature difference threshold ξ, node feature vector x i and j , the bandwidth parameter σ of the node.

[0091] Step 4.2: For each feature map P i '(i=1,2,3), set the high and low thresholds of the Canny algorithm according to different feature map levels, use the Canny edge detection algorithm to extract edge features, and obtain the corresponding edge map E i (i=1,2,3).

[0092] Step 4.3: Construct the initial graph structure and define the graph structure G = (V, E), where the node set V is composed of pixels in the edge graph, and the edge set E is established based on the Euclidean distance and similarity between pixels. Calculate the weight w between each pair of adjacent nodes ij , the formula is: Among them, x i and j represent the feature vectors of nodes i and j respectively, and σ represents the bandwidth parameter.

[0093] Step 4.4: Define the loop variable t and assign it an initial value of 1.

[0094] Step 4.5: If t≤L, execute steps 4.6-4.8, otherwise jump to step 4.9.

[0095] Step 4.6: For each node vi, calculate the feature representation of the tth layer, the formula is: Where N(i) represents the node v i The set of neighbor nodes, b (t) is the bias term.

[0096] Step 4.7: For each node feature Normalization is performed, the formula is: in, Represents node v i The feature representation of the tth layer, Represents node v i The new normalized feature representation after the tth layer iteration.

[0097] Step 4.8: Update the loop variable and increase the value of t by 1, that is, t=t+1, and return to step 4.5 to continue iterating.

[0098] Step 4.9: Calculate feature differences: If Δh≤ξ, execute step 4.10, otherwise jump to step 4.6, where N represents the total number of nodes participating in the calculation.

[0099] Step 4.10: The node features {h i (L)}Map back to the corresponding spatial position to form the aggregated edge feature map E a pt;

[0100] Step 4.11: E a Each pixel value in pt is thresholded: if the pixel value e i ≥θ e , then retain the pixel if the pixel value e i <θ e, set the pixel value to 0;

[0101] Step 4.12: For each feature map P i ', perform fusion operation: P″ i =P i ′+υ*E a pt, υ is the fusion coefficient, which is dynamically adjusted according to the importance of different scale layers to control the weight of edge features in the fusion result.

[0102] Step 4.13: For the fused feature map P″ i Perform Gaussian smoothing to obtain the smoothed feature map P i ”', which is the edge feature map;

[0103] Step 4.14: Calculate the smoothed feature map P i ”' Gradient

[0104] Step 4.15: If Then execute step 4.16, otherwise jump to step 4.12;

[0105] Step 4.16: For each smoothed feature map P i ”'Perform regularization and map pixel values ​​to the interval [0,1];

[0106] Step 4.17: Define loop variable n and assign an initial value of 1;

[0107] Step 4.18: If n≤T, execute steps 4.19-4.21, otherwise jump to step 4.22;

[0108] Step 4.19: Calculate the loss function L for the current iteration (n) , based on the smoothed feature map P i ”'The difference between the target value and the target truth value;

[0109] Step 4.20: Update the model parameters using gradient descent: Among them, k (n) represents the model parameters before the nth iteration, k (n+1) represents the model parameters after the n+1th iteration, Represents the gradient of the loss function L with respect to the model parameter k;

[0110] Step 4.21: Increase the value of n by 1 and return to step 4.18 to continue iterating;

[0111] Step 4.22: Use the optimized edge feature map set {P 1 ”',P 2 ”',P 3”'}, the final vehicle violation detection results are generated through the object detector.

[0112] Step 5: Perform vehicle violation detection on the edge-optimized feature map, use the RetinaNet detection head to perform target detection and bounding box regression, and output the vehicle violation detection results. Figure 5 As shown:

[0113] Step 5.1: Input the optimized edge feature map set {P 1 ”',P 2 ”',P 3 ”'} and the optimized RetinaNet model.

[0114] Step 5.2: Set the optimized edge feature map {P 1 ”',P 2 ”',P 3 ”'} is directly applied to the optimized RetinaNet model for target detection, and performs preliminary feature processing and convolution operations to generate a preliminary candidate region set R 0 .

[0115] Step 5.3: For the candidate region R 0 Perform bounding box regression to obtain the initial predicted bounding box set B 0 ;

[0116] Step 5.4: For B 0 Perform target detection and classification operations on each bounding box in and generate a confidence score S corresponding to each bounding box 0 and category label C 0 ;

[0117] Step 5.5: If S 0 >θ s , then execute steps 5.6-5.8, otherwise jump to step 5.9

[0118] Step 5.6: For all predicted bounding boxes B 0 Perform overlap (IoU) calculation;

[0119] Step 5.7: If IoU(b i ,b j )>θ o , then execute step 5.8, otherwise execute step 5.9;

[0120] Step 5.8: Keep the bounding box with the highest confidence and discard the remaining bounding boxes with high overlap;

[0121] Step 5.9: Generate the bounding box set B after NMS processing 1 and category label C1 As the final detection result, output.

[0122] The following is an explanation of the relevant variables involved in the material:

[0123]

[0124]

[0125]

[0126]

[0127] The present invention can be combined with a computer system to form an intelligent detection device for vehicle violations in a factory loading and unloading area based on dynamic trend perception and multi-scale boundary guidance. The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, it can implement the above-mentioned intelligent detection method for factory vehicle violations based on dynamic trend perception and guidance.

[0128] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent transformation or modification made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for intelligently detecting factory vehicle violations based on dynamic trend perception and guidance, characterized in that: The steps include: Step 1: Preprocess the pre-acquired dataset D1 of vehicle violations in the factory loading and unloading area to obtain the initial dataset D2; Step 2: Use the RetinaNet detection model on the preprocessed dataset D2 and extract feature maps of multiple scales through the feature pyramid network FPN; Step 3: Apply the dynamic trend-aware loss function, dynamically adjust the loss weight according to the sample difficulty during training, adaptively optimize the learning process of the RetinaNet detection model, obtain the optimized RetinaNet detection model, and generate the enhanced feature map; Step 4: Based on the dynamically optimized feature map, the multi-scale boundary guidance module is used to accurately extract the edge information of the target vehicle, and the graph convolutional network GCN is introduced to process the edge information, enhance the spatial correlation of the edge features, and generate the final optimized edge feature map set; Step 5: Perform vehicle violation detection and bounding box regression on the optimized edge feature map set, and output the vehicle violation detection results.

2. The method for intelligently detecting factory vehicle violations based on dynamic trend perception and guidance according to claim 1 is characterized in that: The specific method of step 1 is: Step 1.1: Preprocess the images in the acquired data set D1 by translation and scaling, flip transformation, and image enhancement; Step 1.2: Use the label annotation tool LabelImg software to annotate the images in dataset D2, marking the location, type, and corresponding violation of the vehicle.

3. The method for intelligently detecting factory vehicle violations based on dynamic trend perception and guidance according to claim 1 is characterized in that: The specific method of step 2 is: Step 2.1: Load and initialize the RetinaNet detection model and set the basic parameters of the model; Step 2.2: Configure the feature pyramid network FPN and define the number of input channels and output channels of each level of feature map; Step 2.3: Input the preprocessed images in the dataset D2 into the RetinaNet detection model, and extract feature maps from the C3, C4, and C5 layers respectively after forward propagation of the ResNet network; Step 2.4: Use FPN to process the input image and generate feature maps P1, P2, and P3 of multiple scales.

4. The method for intelligently detecting factory vehicle violations based on dynamic trend perception and guidance according to claim 3 is characterized in that: The specific method of dynamically adjusting the loss weight in step 3 to adaptively optimize the learning process of the RetinaNet detection model is: Step 3.1: Input three multi-scale feature maps P1, P2, and P3 extracted from the FPN of the RetinaNet detection model, set the loss trend change threshold ε1, and dynamically adjust the total loss threshold ε2; Step 3.2: For each extracted feature map P1, P2, P3, use the cross entropy function Calculate the classification loss value and Smooth L1 loss function Calculate the regression loss value, where y i represents the i-th true label, represents the i-th predicted label, N is the total number of samples, SmoothL1 represents the Smooth L1 loss function, and b i represents the parameters of the i-th ground-truth bounding box, represents the parameters of the i-th predicted bounding box; Step 3.3: Calculate the loss change between the current training batch and the previous batch, and calculate the loss trend change. The change calculation formula is: ΔL trend =L current -L previous , L current Represents the total loss of the current batch, L predvious Indicates the total loss of the previous batch; Step 3.4: If ΔL trend >ε1, execute step 3.5, otherwise jump to step 3.6; Step 3.5: Use the dynamic trend perception loss function to optimize the RetinaNet detection model to obtain the optimized RetinaNet detection model, and dynamically adjust the weights of the classification and regression losses of the feature graph. The specific formula is: Among them, L dynamic represents the total loss after dynamic adjustment, α(t), β(t), γ(t) represent dynamic weight coefficients, λ and μ represent regularization coefficients, and Represents the square of the gradient of classification and regression loss with respect to the model parameter k respectively; Step 3.6: If L dynamic ≤ε2, indicating that the optimization of the RetinaNet detection model has achieved the expected effect, then execute 3.7, otherwise jump to step 3.5; Step 3.7: Use the dynamically adjusted loss function for backpropagation, calculate the gradient, update the model parameters, and generate the optimized feature map.

5. The method for intelligently detecting factory vehicle violations based on dynamic trend perception and guidance according to claim 1 is characterized in that: In step 4, the specific method of using the multi-scale boundary guidance module to perform edge optimization and introduce the graph convolutional network GCN is as follows: Step 4.1: Input the optimized feature map set {P1', P2', P3'} and set the edge extraction threshold θ e , gradient threshold θ g , the number of graph convolutional network layers L, the number of iterations T, the learning rate η, the activation function α, the feature difference threshold ξ, the feature vector x of nodes i and j i and j , the bandwidth parameter σ of the node; Step 4.2: For each feature map P i ', set the high and low thresholds of the Canny algorithm according to different feature map levels, use the Canny edge detection algorithm to extract edge features, and obtain the corresponding edge map E i ; Step 4.3: Construct the initial graph structure and define the graph structure G = (V, E), where the node set V is composed of pixels in the edge graph, the edge set E is established based on the Euclidean distance and similarity between pixels, and the weight ω between each pair of adjacent nodes is calculated. ij , the formula is: Step 4.4: Define the loop variable t and assign an initial value of 1; Step 4.5: If t≤L, execute steps 4.6-4.8, otherwise jump to step 4.9; Step 4.6: For each node v i , calculate the feature representation of the tth layer, the formula is: Where N(i) represents the node v i The set of neighbor nodes, b (t) is the bias term; Step 4.7: For each node v i Feature representation of the tth layer Normalization is performed, and the formula is: in, Represents node v i The feature representation of the tth layer is Represents node v i The new normalized feature representation after the tth layer iteration; Step 4.8: Update the loop variable, increase the value of t by 1, and return to step 4.5 to continue iterating; Step 4.9: Calculate feature differences: If Δh≤ξ, execute step 4.10, otherwise jump to step 4.6, where N represents the total number of nodes participating in the calculation; Step 4.10: The node features {h i (L) }Map back to the corresponding spatial position to form the aggregated edge feature map E a pt; Step 4.11: E a Each pixel value in pt is thresholded: if the pixel value e i ≥θ e , then retain the pixel if the pixel value e i <θ e , set the pixel value to 0; Step 4.12: For each feature map P i ', perform fusion operation: P″ i =P i ′+v*E a pt,υ is the fusion coefficient, which is dynamically adjusted according to the importance of different scale layers to control the weight of edge features in the fusion result; Step 4.13: For the fused feature map P″ i Perform Gaussian smoothing to obtain the smoothed feature map P i ”', which is the edge feature map; Step 4.14: Calculate the smoothed feature map P i ”' Gradient Step 4.15: If Then execute step 4.16, otherwise jump to step 4.12; Step 4.16: For each smoothed feature map P i ”'Perform regularization and map pixel values ​​to the interval [0,1]; Step 4.17: Define loop variable n and assign an initial value of 1; Step 4.18: If n≤T, execute steps 4.19-4.21, otherwise jump to step 4.22; Step 4.19: Calculate the loss function L for the current iteration (n) , based on the smoothed feature map P i ”'The difference between the target value and the target truth value; Step 4.20: Update the model parameters using gradient descent: Among them, k (n) represents the model parameters before the nth iteration, k (n+1) represents the model parameters after the n+1th iteration, Represents the gradient of the loss function L with respect to the model parameter k; Step 4.21: Increase the value of n by 1 and return to step 4.18 to continue iterating; Step 4.22: Use the optimized edge feature map set {P1"', P2"', P3"'} to generate the final vehicle violation detection result through the object detector.

6. The method for intelligently detecting factory vehicle violations based on dynamic trend perception and guidance according to claim 1 is characterized in that: The specific method of step 5 is: Step 5.1: Input the optimized edge feature map set {P1"', P2"', P3"'} and the optimized RetinaNet model; Step 5.2: Apply the optimized edge feature map set {P1', P2', P3'} directly to the optimized RetinaNet model for target detection, perform preliminary feature processing and convolution operations, and generate a preliminary candidate region set R0; Step 5.3: Perform bounding box regression on the candidate region R0 to obtain a preliminary predicted bounding box set B0; Step 5.4: Perform target detection and classification operations on each bounding box in B0 to generate the confidence score S0 and category label C0 corresponding to each bounding box; Step 5.5: If S0>θ s ,θ s If it is the confidence threshold, execute steps 5.6-5.8, otherwise jump to step 5.9; Step 5.6: Calculate the overlap IoU for all predicted bounding boxes B0; Step 5.7: If IoU>θ o ,θ o If it is the overlap threshold, execute step 5.8, otherwise execute step 5.9; Step 5.8: Keep the bounding box with the highest confidence and discard the remaining bounding boxes with high overlap; Step 5.9: Generate the bounding box set and category label after NMS processing as the final detection result output.

7. An intelligent detection device for vehicle violations in a factory loading and unloading area, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is loaded into the processor, the method for intelligently detecting factory vehicle violations based on dynamic trend perception and guidance according to any one of claims 1 to 6 is implemented.

Images