Method for detecting violation of engineering vehicle in high-risk area

Abstract

The invention relates to the technical field of vehicle detection, and discloses a high-risk area engineering vehicle violation detection method comprising the following steps: collecting a high-risk area RGB image; a CBAM attention mechanism and an improved GAM attention mechanism are introduced, an improved spatial pyramid pooling convolutional neural network and a DySample dynamic up-sampler are introduced to enter a YOLOv7 network model, and an improved YOLOv7 network model is constructed for identifying high-risk area engineering vehicles and categories; inputting the RGB image of the high-risk area into the improved YOLOv7 network model for training to obtain a trained improved YOLOv7 network model; the RGB image of the high-risk area is obtained again and input into the trained improved YOLOv7 network model for detection, and the engineering vehicle and the category of the high-risk area are obtained; the method is used for detecting illegal parking of the engineering vehicle in the high-risk area, the false detection rate is reduced, the detection precision is improved, and meanwhile the accuracy and robustness of overall detection are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle detection, and in particular to a method for detecting violations of engineering vehicles in high-risk areas. Background Art

[0002] On modern construction sites, due to the complex environment and dense personnel and equipment, the problem of illegal parking of construction vehicles in high-risk areas may cause serious safety accidents and reduce production efficiency. Therefore, automated monitoring and detection technology has become necessary. With the development of technologies such as the Internet of Things (IoT), artificial intelligence (AI), and computer vision (CV), intelligent monitoring systems have gradually become a key tool to solve this problem. These technologies can monitor the situation on the construction site in real time, automatically detect and identify abnormal behaviors of vehicles, and provide an efficient risk warning mechanism. For the detection of illegal parking of construction vehicles in high-risk areas of construction sites, the current technical methods mainly include computer vision technology based on video surveillance, sensor network technology, and vehicle identification and behavior analysis technology based on deep learning. These methods use cameras, lidar, infrared sensors and other equipment, combined with advanced image processing and machine learning algorithms such as convolutional neural networks and YOLO (You Only Look Once), which can accurately identify the parking location of the vehicle and determine whether it is located in a high-risk area.

[0003] However, existing research on the detection methods for illegal parking of engineering vehicles in high-risk areas usually has problems such as insufficient detection accuracy, poor real-time performance, and high false positive rate in complex environments. For example, most of the detection methods for illegal parking of engineering vehicles in high-risk areas use YOLOv7 for detection. In the detection of illegal parking of engineering vehicles in high-risk areas, although YOLOv7 has shown good performance, its false positive rate is high and the detection accuracy of small targets is still not ideal, that is, there are still problems such as insufficient detection accuracy, poor real-time performance, and high false positive rate in complex environments. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a method for detecting violations of engineering vehicles in high-risk areas, which is used to solve the problems of insufficient detection accuracy, poor real-time performance and high false detection rate in complex environments existing in the prior art.

[0005] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is: A method for detecting violations of engineering vehicles in high-risk areas, comprising the following steps: S1, collect RGB images of high-risk areas; S2. Introduce the CBAM attention mechanism and the improved GAM attention mechanism, and introduce the improved spatial pyramid pooling convolutional neural network and DySample dynamic upsampler into the YOLOv7 network model to build an improved YOLOv7 network model for identifying engineering vehicles and categories in high-risk areas; S3, inputting the RGB image of the high-risk area into the improved YOLOv7 network model for training, to obtain a trained improved YOLOv7 network model; S4. Re-acquire the RGB image of the high-risk area and input it into the trained improved YOLOv7 network model for detection to obtain the engineering vehicles and categories in the high-risk area.

[0006] Furthermore, the improved YOLOv7 network model in step S2 includes an input end, a backbone network end, a head network end, and an output end; The backbone network includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a first partial residual network, a first maximum pooling module, a second partial residual network, a second maximum pooling module, a third partial residual network, a third maximum pooling module, and a fourth partial residual network; The head network end includes the first convolution block attention module, the second convolution block attention module, the fifth convolution layer, the sixth convolution layer, the improved spatial pyramid pooling convolutional neural network, the seventh convolution layer, the first DySample dynamic upsampler, the first feature map splicing module, the fifth partial residual network, the eighth convolution layer, the second DySample dynamic upsampler, the second feature map splicing module, the sixth partial residual network, the first parameter convolution, the first improved maximum pooling module, the seventh partial residual network, the second parameter convolution, the second improved maximum pooling module, the eighth partial residual network, and the third parameter convolution.

[0007] Furthermore, the first convolutional block attention module and the second convolutional block attention module are both CBAM structures, wherein the CBAM structure includes a channel attention module and a spatial attention module, and is used to achieve accurate feature extraction.

[0008] Furthermore, the improved spatial pyramid pooling convolutional neural network includes a first convolution layer, a second convolution layer, a third convolution layer, a fourth maximum pooling operation module, a fifth feature map splicing module, a sixth convolution layer, a seventh convolution layer, an eighth heavy parameter convolution, a ninth feature map splicing module, and a tenth convolution layer; The fourth layer maximum pooling operation module includes a first maximum pooling operation module, a second maximum pooling operation module, and a third maximum pooling operation module; Among them, the stride of the eighth layer of heavy parameter convolution is 1 and the size of the convolution kernel is 3×3; The convolution kernel sizes of the first maximum pooling operation module, the second maximum pooling operation module, and the third maximum pooling operation module are all 9×9.

[0009] Furthermore, the first DySample dynamic upsampler and the second DySample dynamic upsampler both include a sampling point generator and a network sample module.

[0010] Furthermore, the first improved maximum pooling module and the second improved maximum pooling module both include a maximum pooling operation module, a global attention mechanism module, a ninth convolutional layer, a tenth convolutional layer and a third feature map splicing module.

[0011] Furthermore, the global attention mechanism module introduces an improved GAM attention mechanism to realize key area feature extraction; Among them, the improved GAM attention mechanism includes a channel attention mechanism and a spatial attention mechanism, and the ReLU activation function in the channel attention mechanism and the spatial attention mechanism is set to a SiLU activation function.

[0012] Furthermore, step S3 specifically includes: S31, inputting the RGB image of the high-risk area into the input end of the improved YOLOv7 network model for standardization to obtain a standardized RGB image; Among them, the RGB image of the high-risk area includes three channels: red, green, and blue, and the size of each channel is the same fixed pixel; S32, inputting the standardized RGB image into the first convolution layer, the second convolution layer, the third convolution layer, and the fourth convolution layer of the backbone network in sequence to perform convolution operations and extract low-level features, and then inputting the first partial residual network, the first maximum pooling module, and the second partial residual network in sequence to perform deep convolution operations to extract deep-level features, thereby obtaining first deep features; The first deep feature is sequentially input into the second maximum pooling module and the third partial residual network to perform deep convolution operation and extract deep-level features to obtain a second deep feature; Input the second deep feature into the third maximum pooling module and the fourth partial residual network to perform a deep convolution operation and extract deep-level features to obtain a third deep feature; S33, inputting the third deep feature into the improved spatial pyramid pooling convolutional neural network for feature fusion to obtain a first fused feature, inputting the first fused feature into the seventh convolutional layer for convolution operation, and then inputting the first DySample dynamic upsampler for upsampling operation to obtain a first upsampled feature; At the same time, the second depth feature is input into the second convolutional block attention module, and the CBAM attention mechanism is used to perform attention weighting of the channel and spatial dimensions, and then input into the sixth convolutional layer for convolution operation to obtain the first output feature; Inputting the first output feature and the first up-sampled feature into a first feature map splicing module for splicing to obtain a first splicing feature; The first concatenated feature is sequentially input into the fifth partial residual network and the eighth convolutional layer for convolution operation, and then input into the second DySample dynamic upsampler for upsampling operation to obtain the second upsampled feature; At the same time, the first depth feature is input into the first convolutional block attention module, and the CBAM attention mechanism is used to perform attention weighting of the channel and spatial dimensions, and then input into the fifth convolutional layer for convolution operation to obtain the second output feature; Inputting the second output feature and the second up-sampled feature into a second feature map splicing module for splicing to obtain a second splicing feature; The second concatenated feature is sequentially input into the sixth partial residual network and the first parameter convolution for performing the parameter convolution to obtain the first parameter convolution feature; The second splicing feature is input into the first improved maximum pooling module after passing through the sixth partial residual network, and the first splicing feature is input into the first improved maximum pooling module after passing through the fifth partial residual network, and the second splicing feature and the first splicing feature are fused by using the improved GAM attention mechanism to obtain the first weighted feature; The first weighted feature is sequentially input into the seventh partial residual network and the second parameter convolution for performing the parameter convolution to obtain the second parameter convolution feature; The first weighted feature is input into the second improved maximum pooling module after passing through the seventh part of the residual network, and the first fused feature is input into the second improved maximum pooling module at the same time, and the first weighted feature and the first fused feature are fused by using the improved GAM attention mechanism to obtain the second weighted feature; The second weighted feature is sequentially input into the eighth partial residual network and the third parameter convolution for performing the parameter convolution to obtain the third parameter convolution feature; S34, inputting the first parameter convolution feature, the second parameter convolution feature, and the third parameter convolution feature to the output end for decoding operation, obtaining a predicted detection frame, completing the training of the improved YOLOv7 network model, and obtaining a trained improved YOLOv7 network model; The predicted detection box includes the predicted detection box coordinates, confidence, and classification information.

[0013] Furthermore, in step S33, the first fusion feature is input into the seventh convolutional layer for convolution operation and then input into the first DySample dynamic upsampler for upsampling operation, and the specific process of obtaining the first upsampling feature is as follows: First, the first fused feature is input into the seventh convolutional layer for convolution operation to obtain the output feature, and the output feature is input into the sampling point generator of the first DySample dynamic upsampler to generate a sample set, and the sample set and the output feature are input into the network sample for dynamic interpolation operation, that is, according to the dynamic sampling points in the sample set, fine feature information is extracted from the output feature to obtain the first upsampling feature.

[0014] Furthermore, the sample set consists of the sum of the original sampling grid and the generated offset; Among them, the original sampling grid is generated by using a static scope cell structure through linear operations and pixel shuffling on the output features in the sampling point generator; The generated offset is generated by dynamic changes of the output features using a dynamically acting cell structure.

[0015] The present invention has the following beneficial effects: 1. The invention proposes a method for detecting violations of engineering vehicles in high-risk areas, which introduces the CBAM attention mechanism and the improved GAM attention mechanism into the improved YOLOv7 network model, so that the model focuses more on important features and reduces the interference of irrelevant information, thereby reducing the false detection rate; 2. The improved spatial pyramid pooling convolutional neural network and DySample dynamic upsampler are introduced into the improved YOLOv7 network model, which enhances the model's adaptability to targets of different scales, overcomes the shortcomings of existing technologies in dealing with small targets and complex scenes, and significantly improves the accuracy and reliability of detection; 3. The improved YOLOv7 network model is used to identify engineering vehicles in the divided high-risk areas, realizing the detection of illegal parking of engineering vehicles in high-risk areas, improving the detection accuracy, and enhancing the accuracy and robustness of the overall detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of a flow chart of a method for detecting violations of engineering vehicles in high-risk areas proposed by the present invention; Figure 2 It is a schematic diagram of the structure of the improved YOLOv7 network model in the embodiment; Figure 3 It is a structural schematic diagram of the CBAM structure in the embodiment; Figure 4 Schematic diagram of the structure of the improved spatial pyramid pooling convolutional neural network in the embodiment; Figure 5It is a structural diagram of the DySample dynamic upsampler in the embodiment; Figure 6 It is a schematic diagram of the structure of the improved maximum pooling module in the embodiment; Figure 7 It is a schematic diagram of the static scope cell structure of the sampling point generator; Figure 8 It is a schematic diagram of the dynamic scope cell structure of the sampling point generator; Fig. 9 Schematic diagram for comparing detection results of different models in the embodiments. DETAILED DESCRIPTION

[0017] The specific implementation modes of the present invention are described below to facilitate those skilled in the art to understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0018] like Figure 1 As shown, a method for detecting violations of engineering vehicles in high-risk areas includes the following steps S1-S4: S1. Collect RGB images of high-risk areas.

[0019] In this embodiment, a camera is used to collect RGB images of the high-risk area in the designated high-risk area so as to be used for training the improved YOLOv7 network model in subsequent steps.

[0020] S2. Introduce the CBAM attention mechanism and the improved GAM attention mechanism. At the same time, introduce the improved spatial pyramid pooling convolutional neural network and DySample dynamic upsampler into the YOLOv7 network model to build an improved YOLOv7 network model for identifying engineering vehicles and categories in high-risk areas.

[0021] In this embodiment, the CBAM attention mechanism and the improved GAM attention mechanism are introduced into the YOLOv7 network model, so that the model pays more attention to the key areas or features in the image when processing the image, thereby improving the accuracy of target detection; at the same time, the DySample dynamic upsampler is introduced into the YOLOv7 network model, which can better retain the details, thereby improving the accuracy of target detection; secondly, the improved spatial pyramid pooling convolutional neural network is introduced to improve the efficiency and effect of feature extraction, while maintaining the accuracy of the model, reducing the computational cost and increasing the reasoning speed; therefore, the constructed improved YOLOv7 network model solves the problems of high false detection rate and insufficient detection accuracy of small targets in the original YOLOv7 network model, and realizes high-precision detection and recognition of engineering vehicles.

[0022] In addition, the structure and connection relationship of the constructed improved YOLOv7 network model are as follows Figure 2 As shown, it specifically includes an input end, a backbone network end, a head network end and an output end. The backbone network end includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a first partial residual network, a first maximum pooling module, a second partial residual network, a second maximum pooling module, a third partial residual network, a third maximum pooling module, and a fourth partial residual network. The head network end includes a first convolutional block attention module, a second convolutional block attention module, a fifth convolutional layer, a sixth convolutional layer, an improved spatial pyramid pooling convolutional neural network, a seventh convolutional layer, a first DySample dynamic upsampler, a first feature map splicing module, a fifth partial residual network, an eighth convolutional layer, a second DySample dynamic upsampler, a second feature map splicing module, a sixth partial residual network, a first parameter convolution, a first improved maximum pooling module, a seventh partial residual network, a second parameter convolution, a second improved maximum pooling module, an eighth partial residual network, and a third parameter convolution.

[0023] Among them, the first convolution block attention module and the second convolution block attention module are both CBAM structures, as shown in the following figure. Figure 3As shown in the figure, the CBAM structure includes a channel attention module and a spatial attention module, which are used to extract accurate features. In this embodiment, the CBAM structure is the CBAM attention mechanism, which includes two core parts: a channel attention module and a spatial attention module. The channel attention module uses global average pooling and global maximum pooling to capture global information, generates two different feature descriptions, and generates a channel attention map through a shared fully connected layer and a Sigmoid function, thereby re-weighting the input features in the channel dimension. The spatial attention module generates two types of feature descriptions by performing global average pooling and global maximum pooling in the channel dimension. After the single-channel feature description is subjected to the convolution operation, the spatial attention map is generated through the Sigmoid function, thereby re-weighting the input features in the spatial dimension; therefore, the introduction of the CBAM attention mechanism between the backbone network and the head network helps to improve the flow of information in the entire network, that is, by introducing the CBAM attention mechanism between the backbone network end and the head network end, the flow of information in the entire network can be effectively improved. This mechanism helps to reduce the impact of background noise by focusing on and strengthening important features, so that the network can better utilize the multi-scale features extracted from the backbone network, thereby providing more valuable input for the final detection task.

[0024] Among them, in this embodiment, the spatial pyramid pooling convolutional neural network of the original YOLOv7 network model is improved to obtain an improved spatial pyramid pooling convolutional neural network, whose specific structure and connection relationship are as follows: Figure 4As shown in FIG, it includes the first convolution layer, the second convolution layer, the third convolution layer, the fourth maximum pooling operation module, the fifth feature map splicing module, the sixth convolution layer, the seventh convolution layer, the eighth heavy parameter convolution, the ninth feature map splicing module, and the tenth convolution layer; the fourth maximum pooling operation module includes the first maximum pooling operation module, the second maximum pooling operation module, and the third maximum pooling operation module; wherein, the stride of the eighth heavy parameter convolution is 1, and the size of the convolution kernel is 3×3. The purpose of this setting is to integrate the parameters added during the training process into a more efficient convolution operation. The convolution kernel size of the first, second and third maximum pooling operation modules are all 9×9. The purpose of this setting is to unify the three pooling operation kernel sizes of the spatial pyramid pooling convolutional neural network in the original YOLOv7 network model from 5×5, 9×9 and 13×13 to three 9×9 kernels. This design not only simplifies the pooling operation, but also reduces the complexity and improves the training efficiency. In summary, the improved spatial pyramid pooling convolutional neural network has the following advantages: (1) Enhanced feature extraction capability and unified pooling strategy; using a larger 9×9 maximum pooling layer can capture a wider range of contextual information, enhance the feature extraction capability, and help the model better understand complex scenes and targets; at the same time, it can avoid the error problem caused by different pooling layers to the feature map in certain programming, thereby improving the consistency of the feature map, which is beneficial to subsequent module processing; (2) Reduced computational complexity and improved model inference speed; in the training phase, a more complex structure such as heavy parameter convolution is used to improve the expressiveness of the model. This design reduces the computational complexity of the inference phase while ensuring the expressiveness of the model, thereby improving the inference speed; at the same time, such a design will reduce the number of parameters and the amount of calculation in the inference phase of the model, first improving the inference speed, and at the same time reducing the memory access cost.

[0025] The introduced DySample dynamic upsamplers are the first DySample dynamic upsampler and the second DySample dynamic upsampler, and their specific structures are as follows: Figure 5 As shown, it includes a sampling point generator and a network sample module, wherein the network sample module is used to improve the interpolation accuracy and flexibility during the upsampling process. Specifically, grid sampling can more flexibly adjust the sampling points in the input feature map by generating dynamic sampling positions, thereby enhancing the processing capabilities of targets at different scales. Compared with traditional fixed sampling methods such as nearest neighbor interpolation, grid sampling can dynamically adjust the sampling position according to the local features of the feature map, so that the upsampling result can better retain the detailed information of the original features; in addition, Figure 5 Medium Parameters , , Represents the linear layer of height, width, and input channels, respectively, with parameters , Represent the input features and up-sampled features respectively. represents the upsampling factor, It indicates the number of groups of waveform cores, not the number of waveform cores themselves. is a group number parameter, which means that multiple waveform kernels are combined together to enhance the flexibility and expressiveness of the upsampling process. In fact, it means that the waveform kernels in the sampler are divided into multiple groups, each group uses two sets of waveform kernels, which further enhances the accuracy of feature extraction and dynamic adjustment capabilities. Since traditional upsampling methods usually rely on bilinear interpolation, these methods have inherent limitations and may lead to the loss of key image details; in addition, the traditional kernel-based upsampling process requires a lot of computational and parameter overhead, which is not conducive to the realization of lightweight network architecture; in the construction scene captured, the image of the engineering vehicle is relatively small, and pixel distortion and other problems may occur, resulting in the loss of fine-grained details and difficulty in feature learning in the recognition task; and the DySample dynamic upsampler can adaptively adjust the sampling strategy according to the contextual information of the input feature map to generate more detailed and accurate upsampling results; compared with the nearest neighbor interpolation upsampling module, the DySample dynamic upsampler performs better in retaining details and edge information, reducing information loss and distortion, and handling complex backgrounds and small targets. Therefore, the DySample dynamic upsampler not only has higher accuracy, but also has stronger robustness, which helps to improve the performance of the model in practical applications. Therefore, this embodiment replaces the nearest neighbor interpolation upsampler in the original YOLOv7 network model with the DySample dynamic upsampler. Its main purpose is to adjust the size of the input feature map through the upsampling operation to match the size of the original image, so as to effectively detect objects of various sizes and distances; and to improve the accuracy and fidelity of the upsampling frequency, thereby improving the effect of detecting small targets and reducing the false detection rate.

[0026] Among them, the improved GAM attention mechanism is introduced into the improved maximum pooling module, namely the first improved maximum pooling module and the second improved maximum pooling module, whose structure is as follows Figure 6As shown, it includes a maximum pooling operation module, a global attention mechanism module, a ninth convolutional layer, a tenth convolutional layer, and a third feature map splicing module; wherein, the global attention mechanism module introduces an improved GAM attention mechanism for realizing key area feature extraction; the improved GAM attention mechanism includes a channel attention mechanism and a spatial attention mechanism, and the ReLU activation function in the channel attention mechanism and the spatial attention mechanism is adjusted to a SiLU activation function. In this embodiment, an improved GAM attention mechanism is introduced into the improved YOLOv7 network model to improve the expression ability and performance of the model; at the same time, the model's attention to important features can be enhanced, and the accuracy and robustness of target detection can be improved; and because the SiLU activation function has better continuity and continuity of derivatives, and its nonlinear characteristics can better capture complex feature relationships, the activation function in the original channel attention mechanism and the spatial attention mechanism is adjusted from ReLU to SiLU, so that the model performs more stably when processing targets of different types and sizes, so that the model can learn and extract features more effectively, and improve the accuracy of target detection.

[0027] S3. Input the RGB image of the high-risk area into the improved YOLOv7 network model for training to obtain a trained improved YOLOv7 network model.

[0028] In this embodiment, images of six types of vehicles commonly used in construction sites are selected: dump trucks, road rollers, mixer trucks, forklifts, excavators and loaders, and divided into training set, validation set and test set, where the training set is 7303 images, the validation set is 904 images and the test set is 885 images; the training set is input into the improved YOLOv7 network model for training, and the validation set and test set are used to verify the model performance. In addition, during the specific training, the training environment is set as follows: using the PyTorch deep learning framework, under the Windows 11 Home Chinese Edition 23H2 operating system, using the 12th Gen Intel(R) Core(TM) i7-12700H 2.30 GHz processor for training and testing of custom data sets; the graphics card is NVIDIA GeForce RTX3060, the video memory is 6GB, the system memory is 16GB, the CUDA version number is 11.7, and the PyTorch version number is 2.0.1; the training parameters are set as follows: the input image size is 640×640 pixels, the batch size is 4, and the epoch is set to 100; the specific training process is as follows: Specifically, step S3 specifically includes S31-S34: S31. Input the RGB image of the high-risk area into the input end of the improved YOLOv7 network model for standardization to obtain a standardized RGB image.

[0029] Among them, the RGB image of the high-risk area includes three channels: red, green, and blue, and the size of each channel is the same fixed pixel.

[0030] S32. Input the standardized RGB image into the first convolution layer, the second convolution layer, the third convolution layer, and the fourth convolution layer of the backbone network in sequence for convolution operation and extraction of low-level features, and then input the first partial residual network, the first maximum pooling module, and the second partial residual network in sequence for deep convolution operation to extract deep-level features, and obtain the first deep features.

[0031] The first deep feature is sequentially input into the second maximum pooling module and the third partial residual network for deep convolution operation and deep-level features are extracted to obtain the second deep feature.

[0032] The second deep feature is input into the third maximum pooling module and the fourth partial residual network to perform deep convolution operation and extract deep-level features to obtain the third deep feature.

[0033] S33. Input the third deep feature into the improved spatial pyramid pooling convolutional neural network for feature fusion to obtain a first fused feature. Input the first fused feature into the seventh convolutional layer for convolution operation and then input into the first DySample dynamic upsampler for upsampling operation to obtain a first upsampled feature.

[0034] Specifically, in step S33, the first fusion feature is input into the seventh convolutional layer for convolution operation and then input into the first DySample dynamic upsampler for upsampling operation. The specific process of obtaining the first upsampling feature is as follows: First, the first fused feature is input into the seventh convolutional layer for convolution operation to obtain the output feature, and the output feature is input into the sampling point generator of the first DySample dynamic upsampler to generate a sample set, and the sample set and the output feature are input into the network sample for dynamic interpolation operation, that is, according to the dynamic sampling points in the sample set, fine feature information is extracted from the output feature to obtain the first upsampling feature.

[0035] In this embodiment, in the DySample dynamic upsampler, the sampling point generator is used to generate a set of dynamic sampling points, called a sample set. These sampling points are generated adaptively, and are intended to accurately sample the local changes in the output features of the input. The generated sample set will be input into the network together with the output features for interpolation. Specifically, after the sample set and the output feature map are input into the network, dynamic interpolation will be performed. Through this interpolation operation, the network will use the generated sampling points to extract more refined feature information from the input features. The interpolation process will be flexibly adjusted according to the position of the sampling points to obtain more accurate upsampling results. The core purpose of this process is to utilize the flexibility of dynamic sampling points to optimize the sampling accuracy, so that the final output feature map is more effective on the basis of retaining the original features, avoiding the loss or blurring of information, especially when dealing with complex boundaries or small targets.

[0036] Specifically, the sample set consists of the sum of the original sampling grid and the generated offset; the original sampling grid is generated by performing linear operations and pixel shuffling on the output features in the sampling point generator using a static scope cell structure; the generated offset is generated by the dynamic change of the output features using a dynamic scope cell structure.

[0037] In this embodiment, Figure 7 The static scope cell structure of the sampling point generator is shown in Figure 1. The static scope cell structure is responsible for generating the basic sampling grid. The principle is to determine the position of the sampling point through a fixed offset to ensure global consistency during the upsampling process. This process is similar to the traditional upsampling method. The position of the sampling point will not change with the change of the input feature map, which ensures the spatial stability during upsampling and the regularity of the initial sampling point. Therefore, the static scope provides a basic sampling structure for upsampling, so that the model maintains preliminary global constraints during the sampling process. Figure 8The schematic diagram of the dynamic scope cell structure of the sampling point generator is that the dynamic scope cell structure dynamically generates the offset of the sampling point through the change of the output feature of the input. The principle is to dynamically adjust the position of the sampling grid according to the characteristic response of each pixel point, so that the sampling point can adaptively align with the change of the input feature; through the convolution operation, the dynamic offset is generated according to the local change of the input feature map, so that the sampling point can more accurately capture the local information in the area with rich details or complex targets. This dynamic adjustment method enhances the flexibility of the upsampling process and helps to improve the detection effect of small targets or irregular areas. In addition, linear operations usually refer to linear transformations of the output features of the input, while pixel shuffling involves the rearrangement of the pixel positions in the feature map. This method enables the generated offset to flexibly adapt to different feature distributions; therefore, the combination of linear transformation and pixel shuffling makes the generation of the offset more flexible and can adapt to different input features. In addition, the range of the offset can be determined by static and dynamic factors, that is, the generation of the offset not only depends on a specific calculation formula, but can also be dynamically adjusted according to the change of the input data features. This illustrates the importance of dynamic characteristics in DySample, which enables the model to generate appropriate offsets according to different input information. Among them, Figure 8 Parameters in Represents the Sigmoid activation function.

[0038] At the same time, the second depth feature is input into the second convolutional block attention module, and the CBAM attention mechanism is used to perform attention weighting on the channel and spatial dimensions, and then input into the sixth convolutional layer for convolution operation to obtain the first output feature.

[0039] The first output feature and the first up-sampled feature are input into a first feature map splicing module for splicing to obtain a first splicing feature.

[0040] The first concatenated feature is sequentially input into the fifth partial residual network and the eighth convolutional layer for convolution operation, and then input into the second DySample dynamic upsampler for upsampling operation to obtain the second upsampling feature.

[0041] In this embodiment, the upsampling operation process of the second DySample dynamic upsampler is the same as the operation process of the first DySample dynamic upsampler.

[0042] At the same time, the first depth feature is input into the first convolutional block attention module, and the CBAM attention mechanism is used to perform attention weighting on the channel and spatial dimensions, and then input into the fifth convolutional layer for convolution operation to obtain the second output feature.

[0043] The second output feature and the second up-sampled feature are input into the second feature map splicing module for splicing to obtain a second splicing feature.

[0044] The second concatenated feature is sequentially input into the sixth partial residual network and the first parameter convolution for parameter convolution to obtain the first parameter convolution feature.

[0045] The second splicing feature is passed through the sixth partial residual network and then input into the first improved maximum pooling module. At the same time, the first splicing feature is passed through the fifth partial residual network and then input into the first improved maximum pooling module. The improved GAM attention mechanism is used to fuse the second splicing feature and the first splicing feature to obtain the first weighted feature.

[0046] The first weighted feature is sequentially input into the seventh partial residual network and the second parameter convolution for parameter convolution to obtain the second parameter convolution feature.

[0047] The first weighted feature is passed through the seventh part of the residual network and then input into the second improved maximum pooling module. At the same time, the first fused feature is input into the second improved maximum pooling module. The first weighted feature and the first fused feature are fused using the improved GAM attention mechanism to obtain the second weighted feature.

[0048] The second weighted feature is sequentially input into the eighth partial residual network and the third parameter convolution for re-parameter convolution to obtain the third parameter convolution feature.

[0049] S34, inputting the first parameter convolution feature, the second parameter convolution feature, and the third parameter convolution feature to the output end for decoding operation, obtaining a predicted detection frame, completing the training of the improved YOLOv7 network model, and obtaining a trained improved YOLOv7 network model.

[0050] The predicted detection box includes the predicted detection box coordinates, confidence, and classification information.

[0051] In this embodiment, after the training of the improved YOLOv7 network model is completed, the accuracy is used (Precision)、Recall (Recall)、 and As the main indicator for evaluating model performance. The curve is used as an auxiliary evaluation indicator to evaluate the performance of the trained improved YOLOv7 network model, showing the prediction accuracy of the model under different classification thresholds. and recall The relationship between the average precision and the Represents the average accuracy of all categories, and its value is plotted with recall as the horizontal axis and precision as the vertical axis The area under the curve, where and They represent the average precision when the IoU threshold is 0.5 and the average precision between the IoU thresholds of 0.5 and 0.95 (with a step size of 0.05), which are comprehensive evaluation criteria for the performance of the model under different precision requirements; yes The area under the curve reflects the overall detection ability of the model at different thresholds. Precision, also known as the precision rate, indicates the proportion of all samples predicted to be positive that are actually positive; recall, also known as the recall rate, indicates the proportion of all samples predicted to be positive that are actually positive. , recall rate , Average Precision and the mean average precision The calculation formula is as follows:

[0052] in, is the number of positive samples correctly classified as positive examples, is the number of negative samples that are misclassified as positive examples, is the number of positive samples that are misclassified as negative examples, Indicates the number of data categories in the data set; Indicates the data set The average precision of the data categories.

[0053] S4. Re-acquire the RGB image of the high-risk area and input it into the trained improved YOLOv7 network model for detection to obtain the engineering vehicles and categories in the high-risk area.

[0054] In this embodiment, in order to further verify the effectiveness of the high-risk area engineering vehicle violation detection method proposed in the present invention, the improved YOLOv7 network model proposed in the present invention is compared with YOLOv5, YOLOv7, YOLOv7-tiny and YOLOv8 under the same indicators, and the experimental results are shown in Table 1: Table 1 Comparison of detection results of different models

[0055] The experimental results in Table 1 are plotted as Fig. 9 The visual attachment shown; therefore, from Table 1 and Fig. 9 It can be seen that the method proposed in this invention performs detection tasks on the self-made dataset. and They reached 98.6% and 90.4% respectively, indicating that the average accuracy of the model under different IoU thresholds is excellent; compared with YOLOv7, , , and There are some improvements, which are 1.3 percentage points, 1.4 percentage points, 1.4 percentage points and 3.7 percentage points respectively. Compared with other models, the method proposed in this invention has a certain degree of improvement in recall rate, and All achieved good performance.

[0056] In summary, the present invention proposes a method for detecting violations of engineering vehicles in high-risk areas. By introducing the CBAM attention mechanism and the improved GAM attention mechanism, and introducing the improved spatial pyramid pooling convolutional neural network and the DySample dynamic upsampler into the YOLOv7 network model, an improved YOLOv7 network model is constructed, which realizes the category recognition of engineering vehicles in high-risk areas, improves the detection accuracy, reduces the false detection rate and can realize small target detection. At the same time, it performs better in processing vehicle detection tasks in complex scenarios, especially in the detection of illegal parking of engineering vehicles in high-risk areas, which improves the accuracy and robustness of the overall detection.

[0057] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

[0058] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A method for detecting violations of engineering vehicles in high-risk areas, characterized in that: The following steps are involved: S1, collect RGB images of high-risk areas; S2. Introduce the CBAM attention mechanism and the improved GAM attention mechanism, and introduce the improved spatial pyramid pooling convolutional neural network and DySample dynamic upsampler into the YOLOv7 network model to build an improved YOLOv7 network model for identifying engineering vehicles and categories in high-risk areas; S3, inputting the RGB image of the high-risk area into the improved YOLOv7 network model for training, to obtain a trained improved YOLOv7 network model; S4. Re-acquire the RGB image of the high-risk area and input it into the trained improved YOLOv7 network model for detection to obtain the engineering vehicles and categories in the high-risk area.

2. The method for detecting violations of engineering vehicles in high-risk areas according to claim 1 is characterized in that: The improved YOLOv7 network model in step S2 includes an input end, a backbone network end, a head network end, and an output end; The backbone network includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a first partial residual network, a first maximum pooling module, a second partial residual network, a second maximum pooling module, a third partial residual network, a third maximum pooling module, and a fourth partial residual network; The head network end includes the first convolution block attention module, the second convolution block attention module, the fifth convolution layer, the sixth convolution layer, the improved spatial pyramid pooling convolutional neural network, the seventh convolution layer, the first DySample dynamic upsampler, the first feature map splicing module, the fifth partial residual network, the eighth convolution layer, the second DySample dynamic upsampler, the second feature map splicing module, the sixth partial residual network, the first parameter convolution, the first improved maximum pooling module, the seventh partial residual network, the second parameter convolution, the second improved maximum pooling module, the eighth partial residual network, and the third parameter convolution.

3. The method for detecting violations of engineering vehicles in high-risk areas according to claim 2 is characterized in that: The first convolutional block attention module and the second convolutional block attention module are both CBAM structures, where the CBAM structure includes a channel attention module and a spatial attention module, and is used to achieve accurate feature extraction.

4. The method for detecting violations of engineering vehicles in high-risk areas according to claim 3 is characterized in that: The improved spatial pyramid pooling convolutional neural network includes a first convolution layer, a second convolution layer, a third convolution layer, a fourth maximum pooling operation module, a fifth feature map splicing module, a sixth convolution layer, a seventh convolution layer, an eighth heavy parameter convolution, a ninth feature map splicing module, and a tenth convolution layer; The fourth layer maximum pooling operation module includes a first maximum pooling operation module, a second maximum pooling operation module, and a third maximum pooling operation module; Among them, the stride of the eighth layer of heavy parameter convolution is 1 and the size of the convolution kernel is 3×3; The convolution kernel sizes of the first maximum pooling operation module, the second maximum pooling operation module, and the third maximum pooling operation module are all 9×9.

5. The method for detecting violations of engineering vehicles in high-risk areas according to claim 4 is characterized in that: The first DySample dynamic upsampler and the second DySample dynamic upsampler both include a sampling point generator and a network sample module.

6. The method for detecting violations of engineering vehicles in high-risk areas according to claim 5 is characterized in that: The first improved maximum pooling module and the second improved maximum pooling module both include a maximum pooling operation module, a global attention mechanism module, a ninth convolutional layer, a tenth convolutional layer and a third feature map splicing module.

7. The method for detecting violations of engineering vehicles in high-risk areas according to claim 6 is characterized in that: The global attention mechanism module introduces an improved GAM attention mechanism to achieve key area feature extraction; Among them, the improved GAM attention mechanism includes a channel attention mechanism and a spatial attention mechanism, and the ReLU activation function in the channel attention mechanism and the spatial attention mechanism is set to a SiLU activation function.

8. The method for detecting violations of engineering vehicles in high-risk areas according to claim 7 is characterized in that: Step S3 specifically includes: S31, inputting the RGB image of the high-risk area into the input end of the improved YOLOv7 network model for standardization to obtain a standardized RGB image; Among them, the RGB image of the high-risk area includes three channels: red, green, and blue, and the size of each channel is the same fixed pixel; S32, inputting the standardized RGB image into the first convolution layer, the second convolution layer, the third convolution layer, and the fourth convolution layer of the backbone network in sequence to perform convolution operations and extract low-level features, and then inputting the first partial residual network, the first maximum pooling module, and the second partial residual network in sequence to perform deep convolution operations to extract deep-level features, thereby obtaining first deep features; The first deep feature is sequentially input into the second maximum pooling module and the third partial residual network to perform deep convolution operation and extract deep-level features to obtain a second deep feature; Input the second deep feature into the third maximum pooling module and the fourth partial residual network to perform a deep convolution operation and extract deep-level features to obtain a third deep feature; S33, inputting the third deep feature into the improved spatial pyramid pooling convolutional neural network for feature fusion to obtain a first fused feature, inputting the first fused feature into the seventh convolutional layer for convolution operation, and then inputting the first DySample dynamic upsampler for upsampling operation to obtain a first upsampled feature; At the same time, the second depth feature is input into the second convolutional block attention module, and the CBAM attention mechanism is used to perform attention weighting of the channel and spatial dimensions, and then input into the sixth convolutional layer for convolution operation to obtain the first output feature; Inputting the first output feature and the first up-sampled feature into a first feature map splicing module for splicing to obtain a first splicing feature; The first concatenated feature is sequentially input into the fifth partial residual network and the eighth convolutional layer for convolution operation, and then input into the second DySample dynamic upsampler for upsampling operation to obtain the second upsampled feature; At the same time, the first depth feature is input into the first convolutional block attention module, and the CBAM attention mechanism is used to perform attention weighting of the channel and spatial dimensions, and then input into the fifth convolutional layer for convolution operation to obtain the second output feature; Inputting the second output feature and the second up-sampled feature into a second feature map splicing module for splicing to obtain a second splicing feature; The second concatenated feature is sequentially input into the sixth partial residual network and the first parameter convolution for performing the parameter convolution to obtain the first parameter convolution feature; The second splicing feature is input into the first improved maximum pooling module after passing through the sixth partial residual network, and the first splicing feature is input into the first improved maximum pooling module after passing through the fifth partial residual network, and the second splicing feature and the first splicing feature are fused by using the improved GAM attention mechanism to obtain the first weighted feature; The first weighted feature is sequentially input into the seventh partial residual network and the second parameter convolution for performing the parameter convolution to obtain the second parameter convolution feature; The first weighted feature is input into the second improved maximum pooling module after passing through the seventh part of the residual network, and the first fused feature is input into the second improved maximum pooling module at the same time, and the first weighted feature and the first fused feature are fused by using the improved GAM attention mechanism to obtain the second weighted feature; The second weighted feature is sequentially input into the eighth partial residual network and the third parameter convolution for performing the parameter convolution to obtain the third parameter convolution feature; S34, inputting the first parameter convolution feature, the second parameter convolution feature, and the third parameter convolution feature to the output end for decoding operation, obtaining a predicted detection frame, completing the training of the improved YOLOv7 network model, and obtaining a trained improved YOLOv7 network model; The predicted detection box includes the predicted detection box coordinates, confidence, and classification information.

9. The method for detecting violations of engineering vehicles in high-risk areas according to claim 8 is characterized in that: In step S33, the first fusion feature is input into the seventh convolutional layer for convolution operation and then input into the first DySample dynamic upsampler for upsampling operation. The specific process of obtaining the first upsampling feature is as follows: First, the first fused feature is input into the seventh convolutional layer for convolution operation to obtain the output feature, and the output feature is input into the sampling point generator of the first DySample dynamic upsampler to generate a sample set, and the sample set and the output feature are input into the network sample for dynamic interpolation operation, that is, according to the dynamic sampling points in the sample set, fine feature information is extracted from the output feature to obtain the first upsampling feature.

10. The method for detecting violations of engineering vehicles in high-risk areas according to claim 9 is characterized in that: The sample set consists of the sum of the original sampling grid and the generated offset; Among them, the original sampling grid is generated by using a static scope cell structure through linear operations and pixel shuffling on the output features in the sampling point generator; The generated offset is generated by dynamic changes of the output features using a dynamically acting cell structure.

Images