Model training method and intelligent construction site safety equipment detection method

Abstract

The invention discloses a model training method and an intelligent construction site safety equipment detection method, and belongs to the technical field of target detection. The method comprises the following steps: firstly, acquiring a sample security equipment image, a sample security equipment annotation image and a YOLOv8 improved model, wherein the YOLOv8 improved model comprises a high-level screening feature fusion pyramid structure, a dynamic sampling module, a RepGhostCSPELAN module and an SEAM attention module; and then, based on the sample security equipment image and the sample security equipment annotation image, training a YOLOv8 improved model to obtain a target security equipment detection model. According to the method, the parameter quantity and the calculation quantity can be reduced, the detection effect of the safety equipment in the intelligent construction site is improved, and the detection real-time performance and high efficiency are ensured.

Description

Technical Field

[0001] The present application relates to the field of target detection technology, and in particular to a model training method and a smart construction site safety equipment detection method. Background Art

[0002] In smart construction sites, intelligent monitoring systems have gradually become the key to construction site safety management. By using digital image technology and target detection technology, they can effectively filter out useless information, retain key safety information, and conduct further analysis on this basis to ensure the stable operation of construction site safety equipment.

[0003] At present, the YOLOv8 high-precision detection model has a complex structure, too many parameters and too much computation, making it difficult to meet the needs of low-power embedded devices and mobile devices. Summary of the invention

[0004] In view of this, the present application provides a model training method and a smart construction site safety equipment detection method, which can reduce the number of parameters and calculations, improve the detection effect of safety equipment in smart construction sites, and ensure real-time and high efficiency of detection.

[0005] Specifically, the following technical solutions are included:

[0006] In a first aspect, an embodiment of the present application provides a model training method, the method comprising:

[0007] Obtaining a sample safety device image, a sample safety device annotated image, and a YOLOv8 improved model, wherein the YOLOv8 improved model includes a high-level screening feature fusion pyramid structure, a dynamic sampling module, a RepGhostCSPELAN module, and a SEAM attention module;

[0008] Based on the sample security device image and the sample security device annotated image, a YOLOv8 improved model is trained to obtain a target security device detection model.

[0009] In some embodiments, the training of the YOLOv8 improved model based on the sample safety device image and the sample safety device annotated image to obtain the target safety device detection model includes:

[0010] Based on the sample safety device image and the sample safety device annotated image, training a YOLOv8 improved model to obtain an intermediate YOLOv8 improved model;

[0011] In response to the performance of the intermediate YOLOv8 improved model satisfying the training completion condition, the intermediate YOLOv8 improved model satisfying the training completion condition is used as the target security device detection model; in response to the performance of the intermediate YOLOv8 improved model not satisfying the training completion condition, the training weights and training parameters of the intermediate YOLOv8 improved model are adjusted until the performance of the intermediate YOLOv8 improved model satisfies the training completion condition, and the training parameters include one or more parameters of the initial learning rate, the number of training iterations and the hyperparameters.

[0012] In some embodiments, the YOLOv8 improved model is a model improved on the basis of the YOLOv8 model, and the improvement process includes:

[0013] In the neck network part, a high-level screening feature fusion pyramid structure is introduced to obtain the first improved model;

[0014] The up-sampling module of the first improved model is replaced by the dynamic sampling module to obtain a second improved model;

[0015] The RepGhostCSPELAN module is used to replace the traditional C2f module in the backbone network part and the neck network part of the second improved model to obtain a third improved model;

[0016] The SEAM attention module is used to improve the detection head of the third improved model to obtain the YOLOv8 improved model.

[0017] In some embodiments, the training completion condition is that the number of parameters, the amount of calculation and the weight of the intermediate YOLOv8 improved model do not decrease for 50 consecutive rounds, and the accuracy, recall rate and mAP are 50 No upgrade for 300 consecutive rounds.

[0018] In some embodiments, the high-level screening feature fusion pyramid structure, in the high-level screening feature fusion pyramid structure, first extract a multi-level feature map, the multi-level feature map will be processed by the global average pooling layer and the global maximum pooling layer in the CA module, extract the average response and the maximum response of each channel, the generated feature map is spliced ​​by the connection operation, and then the attention weight value f of each channel is determined by the Sigmoid activation function. CA ∈R C×1×1 , and then multiply the weight value with the feature map of the corresponding scale to generate the filtered attention weight map. Finally, the feature fusion module efficiently fuses the filtered low-level features with the high-level features.

[0019] In some embodiments, the dynamic sampling module is improved by the upsampling module, and is used for dynamic spatial sampling, and the spatial dimension of the input tensor is adjusted by applying the learned offset, and the dynamic sampling module includes a scaling unit, an offset convolution layer, and a range convolution layer connected in sequence;

[0020] The scaling unit is used to adjust the input size, the offset convolution layer is used to generate a sampled offset and is initialized with a normal distribution, and the range convolution layer generates a range map when enabled and is initialized with a constant value.

[0021] In some embodiments, the RepGhostCSPELAN module includes a first convolutional layer, a segmentation layer, a reparameterized convolutional layer, multiple second convolutional layers, a splicing layer and a third convolutional layer connected in sequence. The segmentation layer divides the input feature map into two parts, one part is directly passed to the splicing layer, and the other part is processed by the reparameterized convolutional layer and then spliced ​​with the output of multiple second convolutional layers. The RepGhostCSPELAN module is used to fuse feature information of different branches, enhance feature representation capabilities by connecting partial features across stages, and optimize the gradient flow path to improve the training efficiency and convergence speed of the model.

[0022] In some embodiments, the convolution kernel size of the first convolution layer is 1×1, the step size is 1, the padding is 0, the number of output channels is 2e×c_out, and the first convolution layer is used to adjust the number of channels of the input feature map and perform preliminary fusion of information;

[0023] The convolution kernel size of the second convolution layer is 3×3, the step size is 1, the padding is 1, and the number of output channels is scale×e×c_out. The second convolution layer is used to generate partially redundant feature maps, which reduces the amount of calculation and parameters of the model;

[0024] The convolution kernel size of the third convolution layer is 1×1, the step size is 1, the padding is 0, and the number of output channels is c_out. The third convolution layer is used to perform the final channel number adjustment and feature fusion on the spliced ​​feature map, and output the processing result of the module;

[0025] Among them, e represents the implicit channel coefficient, which is used to control the channel expansion ratio inside the module, c_out represents the number of output channels, that is, the number of feature map channels after module processing, and scale represents the feature map scaling coefficient, which is used to adjust the scale of feature maps of different branches.

[0026] In some embodiments, acquiring a sample safety device image and a sample safety device annotated image includes:

[0027] Constructing an original safety equipment image, wherein the original safety equipment image includes a construction site environment image under different shooting angles, different distances, different weather conditions, and different lighting conditions;

[0028] The safety helmet, safety rope, user head and user body in the original safety equipment image are labeled by using labels, and the label format of the labels is converted from JSON format to a file format supported by YOLO to obtain the original safety equipment labeled image, wherein the labels include target categories, center point coordinates of the labeled bounding box and width of the labeled bounding box, and the target categories include safety helmet, safety rope, user head and user body;

[0029] Dividing the original safety device image and the original safety device annotated image into a training set, a verification set and a test set;

[0030] The training set, the validation set and the test set are sequentially subjected to image cutting and pasting processing, label fusion processing, and multiple image enhancement processing to obtain enhanced training set, validation set and test set, wherein the enhanced training set includes sample safety equipment images and sample safety equipment annotated images.

[0031] In a second aspect, an embodiment of the present application provides a method for detecting safety equipment on a smart construction site, the method comprising:

[0032] Obtaining a target security device detection model in the model training method as described in the first aspect;

[0033] The image of the safety device to be tested is input into the target safety device detection model to obtain a safety device detection result, wherein the safety device detection result includes a target category, a target bounding box position, a confidence level, and a category probability. The target categories include a safety helmet, a safety rope, a user's head, and a user's body.

[0034] The beneficial effects of the technical solution provided by the embodiments of the present application include at least:

[0035] The embodiment of the present application provides a model training method and a smart construction site safety equipment detection method, the method comprising first, obtaining a sample safety equipment image, a sample safety equipment annotated image and a YOLOv8 improved model, the YOLOv8 improved model comprising a high-level screening feature fusion pyramid structure, a dynamic sampling module, a RepGhostCSPELAN module and a SEAM attention module, the high-level screening feature fusion pyramid structure combines high-level screening features with multi-scale fusion, effectively improves the detection accuracy and robustness, and effectively improves the feature extraction capability of the YOLOv8 improved model, the dynamic sampling module dynamically adjusts the sampling strategy to make the upsampling process more accurate and efficient, can improve the resolution and detail retention capability of the feature map, and also enhances the flexibility and robustness of the YOLOv8 improved model in the multi-scale feature extraction process, so that the YOLOv8 improved model can maintain efficient feature extraction and processing capabilities in various complex scenarios, RepGhostCSPELAN module The GhostCSPELAN module combines the design concept of GhostNet, and uses simple and cheap operations to generate a large number of redundant features without significantly increasing the computational complexity, replacing the high-computation part of the traditional convolution operation, effectively reducing the amount of calculation and parameters, maintaining or even improving the model's feature expression ability. At the same time, by adopting structural reparameterization on the gradient flow branch, the feature extraction and gradient flow capabilities are enhanced, efficient feature extraction is achieved, and the number of floating-point operations is kept low. The SEAM attention module enhances the focus on the occluded area, strengthens the focus on key features and suppresses non-critical information. By paying attention to spatial and channel information at the same time, it can better capture the detailed features of the target, significantly improving the detection accuracy and robustness of the model in the case of partial occlusion of the object, and improving the detection accuracy; then, based on the sample safety equipment image and the sample safety equipment annotated image, the YOLOv8 improved model is trained to obtain the target safety equipment detection model. This method can reduce the amount of parameters and calculations, improve the detection effect of safety equipment in smart construction sites, and ensure the real-time and high efficiency of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0037] Figure 1 A method flow chart of the model training method provided in an embodiment of the present application;

[0038] Figure 2 A schematic diagram of the structure of the YOLOv8 improved model in the model training method provided in the embodiment of the present application;

[0039] Figure 3 A schematic diagram of the effect of the enhanced training set in the model training method provided in an embodiment of the present application;

[0040] Figure 4 A schematic diagram of the structure of a high-level screening feature fusion pyramid structure in the model training method provided in an embodiment of the present application;

[0041] Figure 5 A schematic diagram of the structure of the RepGhostCSPELAN module in the model training method provided in an embodiment of the present application;

[0042] Figure 6 A schematic diagram of the structure of the SEAM module in the model training method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0044] In order to make the technical solutions and advantages of the present application more clear, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0045] In a first aspect, the present application provides a model training method, such as Figure 1 As shown, the method includes:

[0046] Step 101, obtain sample security device images, sample security device annotated images and a YOLOv8 improved model, where the YOLOv8 improved model includes a high-level screening feature fusion pyramid structure, a dynamic sampling module, a RepGhostCSPELAN module and a SEAM attention module.

[0047] Sample safety equipment images and sample safety equipment annotated images are important data for training the YOLOv8 improved model, ensuring the diversity and representativeness of the data and effectively improving the detection effect and robustness of the model in practical applications. Figure 2The improved YOLOv8 model includes a high-level screening feature fusion pyramid structure, a dynamic sampling module, a RepGhostCSPELAN module, and a SEAM attention module. The high-level screening feature fusion pyramid structure (HS-FPN structure for short) can more effectively extract and fuse feature information from different levels with its multi-level feature fusion structure. Compared with the original neck network part of YOLOv8, the HS-FPN structure shows better performance when processing multi-scale targets. The HS-FPN structure combines high-level screening features with multi-scale fusion, effectively improving the detection accuracy and robustness, and effectively improving the feature extraction capability of the improved YOLOv8 model. The dynamic sampling module (DySample module for short) introduces the DySample technology in the upsampling module. By dynamically adjusting the sampling strategy, the upsampling process is made more accurate and efficient, which can improve the resolution and detail retention of the feature map. It also enhances the flexibility and robustness of the improved YOLOv8 model in the multi-scale feature extraction process, so that the improved YOLOv8 model can maintain efficient feature extraction and processing capabilities in various complex scenarios. The RepGhostCSPELAN module combines the design concept of GhostNet, uses simple and cheap operations to generate a large number of redundant features, replaces the high-computation part of the traditional convolution operation, effectively reduces the amount of calculation and parameters, maintains or even improves the model's feature expression ability, and at the same time, the module abandons the traditional BottleNeck structure, introduces multiple convolution branches during training by adopting structural reparameterization on the gradient flow branch to enhance feature extraction and gradient flow capabilities, and merges these branches into an equivalent branch in the inference stage, thereby achieving efficient feature extraction and keeping the number of floating-point operations low. The RepGhostCSPELAN module greatly improves the feature expression ability and detection performance of the improved YOLOv8 model without significantly increasing the computational complexity, and is suitable for use in low-power embedded devices and mobile devices, such as smartphones and video real-time monitoring devices. The SEAM attention module can better capture the detailed features of the target by enhancing attention to the occluded area, strengthening the focus on key features and suppressing non-critical information, and by paying attention to spatial and channel information at the same time, significantly improving the detection accuracy and robustness of the model in the case of partial occlusion of the object, and improving the detection accuracy.

[0048] In some embodiments, acquiring a sample safety device image and a sample safety device annotated image includes:

[0049] (1) Constructing original safety equipment images, which include construction site environment images under different shooting angles, different distances, different weather conditions, and different lighting conditions. The original safety equipment images are relatively rich, covering complex construction site scenes in different situations, which are difficult to detect and are conducive to subsequent model training.

[0050] In some embodiments, the original safety device image may come from a safety device dataset SEDS.

[0051] (2) Label the helmet, safety rope, user head and user body in the original safety equipment image, and convert the label format from JSON format to a file format supported by YOLO to obtain the original safety equipment labeled image, where the label includes the target category, the center point coordinates of the labeled bounding box and the width of the labeled bounding box. The target categories include helmet, safety rope, user head and user body. Labeling labels can provide the model with a learning template, which is conducive to improving the subsequent detection accuracy of the model.

[0052] (3) The original safety device images and the original safety device annotated images are divided into a training set, a validation set, and a test set. The data set is divided so that the model can reasonably use the data set for training, validation, and testing.

[0053] In some embodiments, the training set may include 5143 images, the validation set may include 735 images, and the test set may include 1470 images.

[0054] (4) The training set, the validation set, and the test set are sequentially subjected to image cutting and pasting processing, label fusion processing, and multiple image enhancement processing to obtain enhanced training sets, validation sets, and test sets, wherein the enhanced training set includes sample safety equipment images and sample safety equipment annotated images.

[0055] In some embodiments, the image cutting and pasting process is specifically implemented by randomly selecting two images A and B, randomly cutting out a rectangular area from image A, and pasting it to the corresponding position of image B. The size and position of the cut area are randomly determined to increase the diversity of the data.

[0056] In some embodiments, the specific implementation of the label fusion process is: adjust the annotations in the cut area accordingly, and transfer the annotation information in image A to image B. The label information of the pasted area and the original image area is merged to ensure the accuracy of the label.

[0057] In some embodiments, the specific implementation of multiple image enhancement processes is: multiple enhancement processes are performed on each image to generate multiple enhanced images, and the original annotation information is retained. Through repeated enhancement, the diversity and complexity of the data set are improved.

[0058] The image cutting and pasting process, label fusion process, and multiple image enhancement processes are named S-Cutmix enhancement method. For example, Figure 3As shown in the figure, the specific process of the S-Cutmix enhancement method is as follows: extract the target image from the data set as the embedded image sample set, randomly select training images and corresponding labels from the data set, randomly sample rectangular areas in the training images, replace the sampled areas in the training images with embedded images to form new mixed images, and update the sampled area labels in the labels to form mixed labels. During the training process, the model learns features from both the training images and the embedded images on the mixed images, improving the model's learning ability for the target and preventing the model from overfitting to a single category. Through the S-Cutmix enhancement method, a large number of complex and diverse training samples are generated, and the enhanced images and their annotated files are added to the data set to increase the amount of training data and improve the robustness of the model in different scenarios. This method is particularly suitable for small target detection and effectively alleviates the problems of data sparseness and imbalance. The enhanced training set contains more occlusions, lighting changes, and perspective changes, which can improve the generalization ability of the model.

[0059] In some embodiments, the YOLOv8 improved model is a model improved on the basis of the YOLOv8 model, and the improvement process includes:

[0060] (1) In the neck network part, a high-level screening feature fusion pyramid structure is introduced to obtain the first improved model.

[0061] Through the multi-level feature fusion structure, the HS-FPN structure can more effectively extract and fuse feature information from different levels. Compared with the original neck network part of YOLOv8, the HS-FPN structure shows better performance when processing multi-scale targets. The HS-FPN structure combines high-level screening features with multi-scale fusion, effectively improving detection accuracy and robustness.

[0062] In some embodiments, a high-level screening feature fusion pyramid structure is used. In the high-level screening feature fusion pyramid structure, a multi-level feature map is first extracted. The multi-level feature map is processed by the global average pooling layer and the global maximum pooling layer in the CA module to extract the average response and the maximum response of each channel. The generated feature map is spliced ​​by the connection operation, and then the attention weight value f of each channel is determined by the Sigmoid activation function. CA ∈R C×1×1 , and then multiply the weight value with the feature map of the corresponding scale to generate the filtered attention weight map. Finally, the feature fusion module efficiently fuses the filtered low-level features with high-level features. The high-level filtered feature fusion pyramid structure can improve the detection and recognition capabilities of the model when processing security equipment images with scale differences and complex backgrounds. Based on high-level features, the low-level features are finely screened and fused to enhance the effectiveness of the features.

[0063] In some embodiments, Figure 4 As shown in the figure, the high-level screening feature fusion pyramid structure takes full account of the characteristics of safety equipment images (such as helmets and safety ropes) in its design, aiming to improve the detection and recognition capabilities of the model when processing safety equipment images with scale differences and complex backgrounds. The core is to use high-level features as a guide to finely screen and fuse low-level features, thereby enhancing the effectiveness of features, especially for key details in safety equipment images. The feature selection module focuses on screening and strengthening low-level feature information related to safety equipment. It first uses a convolutional neural network to extract a multi-level feature map f from the input safety equipment image. in ∈R C×H×W High-level features are rich in semantic information and can describe the general shape and category of safety equipment, while low-level features retain more spatial details in the image, such as the edge contour of a helmet or the texture of a safety rope. In order to more accurately identify safety equipment, the feature map is passed through the global average pooling layer in the CA module ( Figure 4 Avg Pool layer in) and global maximum pooling layer ( Figure 4 The two pooling methods extract the average response and maximum response of each channel respectively. The generated feature maps are concatenated through the connection operation, and then the Sigmoid activation function is used to determine the attention weight value f of each channel. CA ∈R C×1×1 , and then multiply the weight value by the feature map of the corresponding scale to generate a filtered attention weight map. This weight map pays special attention to the key features in the safety equipment image, such as the iconic color of the helmet or the unique texture of the safety rope, thereby achieving weighted screening of low-level feature maps, strengthening features related to safety equipment, and suppressing irrelevant background information. Finally, through the feature fusion module ( Figure 4 The SFF module in the fusion process efficiently fuses the filtered low-level features with the high-level features. To ensure the consistency of the feature map scale, the module will upsample or downsample the high-level features to match the resolution of the low-level features. During the fusion process, the pixel values ​​of the two feature maps at the same position are added element by element to generate a fused feature map. This fusion method is particularly helpful in highlighting the key features in the safety equipment image, such as the connection point between the helmet and the safety rope or the direction of the safety rope, thereby significantly enhancing the expression ability of the features and improving the detection and recognition accuracy of the model for safety equipment.

[0064] (2) The upsampling module of the first improved model is replaced by the dynamic sampling module to obtain the second improved model.

[0065] The upsampling module of the first improved model is replaced by a dynamic sampling module, and the upsampling process is made more accurate and efficient by dynamically adjusting the sampling strategy. Compared with the traditional fixed sampling method, the DySample module can better adapt to feature maps of different resolutions, thereby improving the accuracy and efficiency of upsampling, and thus improving the detection performance of the overall model.

[0066] In some embodiments, the dynamic sampling module is improved by the upsampling module, and is used for dynamic spatial sampling. The spatial dimension of the input tensor is adjusted by applying the learned offset, including a scaling unit, an offset convolution layer and a range convolution layer connected in sequence; the scaling unit is used to adjust the input size, the offset convolution layer is used to generate a sampling offset and is initialized with a normal distribution, and the range convolution layer generates a range map when enabled and is initialized with a constant value.

[0067] In some embodiments, the dynamic sampling module stores the initial position of the coordinate calculation during sampling via an initial position buffer.

[0068] In the dynamic sampling module (DySample module for short), the offset is initialized through the offset convolution layer, while the range convolution layer is initialized with a constant value. The initial position is calculated by a series of values ​​adjusted by the scaling factor, and these positions are used in the subsequent sampling process. During the sampling process, the DySample module samples the input through a bilinear interpolation function and adjusts the spatial dimensions based on the offset and the initial position. Depending on the selected style (lp or pl), the DySample module samples directly after calculating the offset or after the pixel reorganization operation. The DySample module can enhance the model's ability to handle spatial transformations, making it particularly suitable for tasks that require precise spatial operations, such as multi-scale object detection.

[0069] It should be noted that traditional upsampling methods, such as nearest neighbor interpolation and bilinear interpolation, can enlarge the size with high computational efficiency when processing safety equipment images such as helmets and safety ropes. However, they lack flexibility in the enlargement process and cannot fully utilize the semantic information in the feature map. In particular, it is often difficult to accurately reconstruct the detailed features in the safety equipment images, such as the logo on the helmet or the fine texture of the safety rope. In recent years, dynamic upsampling methods, such as CARAFE, FADE, and SAPA, have significantly improved the flexibility and performance of upsampling by generating content-aware upsampling kernels, so that the enlarged image can better retain the details and semantic information in the original image. However, these methods are usually computationally complex and require additional subnetworks to generate upsampling kernels, which increases the computational overhead and inference time, and may not be applicable in the real-time processing scenario of safety equipment images. To address the above problems, a new ultra-lightweight and efficient dynamic sampling module, namely the DySample module, is introduced. The DySample module avoids the paradigm based on convolution kernels and redefines the upsampling process from the perspective of point sampling, which is particularly suitable for feature enlargement of safety equipment images. The initial design of the DySample module is based on the built-in grid sample function of PyTorch. Given a feature map X of size C×H×W (where C is the number of channels, H and W are the height and width of the feature map, respectively), an offset O is generated through a linear layer. The number of input and output channels of this linear layer is C and 2s^2 (s is the upsampling factor), which means that for each pixel on each channel, a corresponding offset is generated to guide the position of the new pixel during the upsampling process. The offset O is reshaped into a sampling set S of 2×sH×sW through the Pixel Shuffle operation, and then the sampling set S is used to upsample the feature map X through the grid_sample function to obtain the upsampled feature map X'. This process not only takes into account the position of each new pixel, but also learns the offset so that the upsampling process can better adapt to specific features in the safety equipment image, such as the edge of the helmet or the direction of the safety rope. In the preliminary design, the initial sampling positions of all upsampling points were fixed on the grid points of the original feature map, which ignored the positional relationship between adjacent upsampling points and may cause blur or distortion in some details of the security device image. To improve this, the DySample module changed the initial sampling position from "nearest neighbor initialization" to "bilinear initialization" so that zero offset produces a bilinear interpolated feature map, thereby distributing the initial sampling position more evenly and better capturing the detailed features in the security device image. In addition, the DySample module also introduced a "static scope factor" to locally constrain the offset, solving the sampling confusion problem caused by overlapping offset scopes and ensuring stability and consistency during the upsampling process.Through the feature grouping strategy, DySample independently generates offsets for features of different channel groups, which enhances the flexibility and expressiveness of upsampling, allowing the model to more accurately identify and reconstruct key features in security device images. Finally, to further increase the flexibility of the offset, the DySample module generates a point-level "dynamic scope factor" through another linear layer and uses a sigmoid function to limit its value to the range of [0,0.5]. This dynamic scope factor is combined with the static scope factor to jointly modulate the size of the offset, so that the upsampling process can be dynamically adjusted according to the specific content of the security device image, further improving the quality and efficiency of upsampling.

[0070] (3) The RepGhostCSPELAN module is used to replace the C2f module in the feature extraction part of the second improved model to obtain the third improved model.

[0071] The RepGhostCSPELAN module generates redundant feature maps through cheap operations, significantly reducing the amount of computation and parameters. At the same time, the RepGhostCSPELAN module abandons the traditional BottleNeck structure and uses a general convolutional layer on the gradient flow branch to enhance feature extraction and gradient flow capabilities. With this setting, the general convolutional layer is fused during the inference stage to achieve efficient feature extraction, keep the number of floating-point operations low, and significantly improve the inference speed and efficiency.

[0072] It should be noted that although the traditional C2f module in the second improved model has excellent performance, its relatively large amount of computation and parameters limit its deployment on edge devices, making it difficult to meet the real-time requirements of smart construction site scenarios. Therefore, inspired by the RepVGG, GhostNet, CSP (Cross Stage Partial) network architecture and ELAN (Efficient Layer Aggregation Network) ideas, the RepGhostCSPELAN module is used to replace the C2f module in the feature extraction part of the second improved model, aiming to significantly reduce the amount of computation and parameters of the model by reducing computational redundancy, enhancing feature extraction capabilities, and optimizing gradient flow paths, while maintaining or even improving the performance of the model. Figure 5As shown, in some embodiments, the RepGhostCSPELAN module includes a first convolutional layer, a segmentation layer, a re-parameterized convolutional layer, multiple second convolutional layers, a splicing layer and a third convolutional layer connected in sequence. The segmentation layer divides the input feature map into two parts, one part is directly passed to the splicing layer, and the other part is processed by the re-parameterized convolutional layer and then spliced ​​with the output of the multiple second convolutional layers. The RepGhostCSPELAN module is used to fuse feature information of different branches, enhance feature representation capabilities by connecting partial features across stages, and optimize the gradient flow path, thereby improving the training efficiency and convergence speed of the model.

[0073] The RepGhostCSPELAN module combines the idea proposed in GhostNet, that is, there is extensive redundancy in the intermediate feature maps calculated by mainstream CNN, and uses simple and cheap operations to generate some redundant feature maps, replacing the high-computation part of the traditional convolution operation, effectively reducing the computational complexity and parameter volume of the model. At the same time, this module abandons the traditional BottleNeck structure in the YOLOv5 model and the YOLOv8 model, adopts the structural reparameterization technology on the gradient flow branch, and integrates the reparameterized convolution layer to make up for the performance loss caused by abandoning the residual block in the BottleNeck structure.

[0074] In some embodiments, in the reparameterized convolution layer (RepConv layer for short), a multi-branch topology is used in the training phase, including standard 3×3 convolution, 1×1 convolution, and identity mapping. This multi-branch structure helps to achieve efficient feature extraction and fusion, and improves the expressiveness and training effect of the model. In the inference phase, the RepConv layer fuses the convolution kernel parameters and batch normalization parameters of multiple branches to calculate an equivalent single convolution kernel and bias term, which significantly reduces the amount of computation and memory usage in the inference phase, and improves the inference speed and efficiency.

[0075] In the RepGhostCSPELAN module, the two architectural ideas of cross-stage partial feature connection (CSP for short) and optimized gradient transmission path design (ELAN for short) are also cleverly integrated. Through the CSP architecture, the reuse and learning capabilities of features have been greatly improved; through the ELAN architecture, the feature extraction efficiency and convergence speed are further improved. At the same time, this fusion also enables the module to have good scalability and flexibility while maintaining high performance. Finally, in order to enable the RepGhostCSPELAN module to adapt to the needs of different computing resources and application scenarios, a scaling factor coefficient is also introduced. By adjusting the scaling factor, the scale of different branch feature maps can be flexibly adjusted while keeping the basic architecture of the module unchanged, thereby generating model instances of different scales, so that the RepGhostCSPELAN module can take into account the high efficiency of small models and the accuracy requirements of large models.

[0076] In some embodiments, the convolution kernel size of the first convolution layer is 1×1, the step size is 1, the padding is 0, and the number of output channels is 2e×c_out. The first convolution layer is used to adjust the number of channels of the input feature map and perform preliminary information fusion; the convolution kernel size of the second convolution layer is 3×3, the step size is 1, the padding is 1, and the number of output channels is scale×e×c_out. The second convolution layer is used to generate partially redundant feature maps; the convolution kernel size of the third convolution layer is 1×1, the step size is 1, the padding is 0, and the number of output channels is c_out. The third convolution layer is used to perform the final channel number adjustment and feature fusion on the spliced ​​feature map, and output the processing result of the module; wherein, e represents the implicit channel coefficient, c_out represents the output channel number, and scale represents the feature map scaling factor, which is used to control the scaling ratio of the feature map.

[0077] It should be noted that the scale coefficient value can be adjusted. Smaller e values ​​and scale coefficient values ​​can reduce the amount of calculation and parameters, while larger e values ​​and scale coefficient values ​​can improve the feature extraction capability.

[0078] like Figure 5 As shown in the figure, the input feature map of the previous layer is first convolved through the first convolutional layer in the RepGhostCSPELAN module, and then split into two branches in the segmentation layer (Split module in the figure), one of which is directly passed through the general convolutional layer (i.e. Figure 5The main branch is processed by the RepConv module in the network, and the other branch is retained for feature fusion. The main branch is processed by the RepConv module, which helps to reduce the computational complexity during training while maintaining the expressiveness of the model. The convolution kernel can be fused in the inference stage, thereby reducing the amount of calculation and the number of parameters and improving the inference efficiency. The feature map processed by the RepConv module passes through n-1 second convolutional layers, and a reasonable scale value is selected to strike a balance between computational efficiency and feature extraction capability. The convolution operations of multiple second convolutional layers are designed to refine and enrich the information of the feature map layer by layer while maintaining low computational complexity. The feature map processed by multiple layers of convolution is finally spliced ​​in the splicing layer with the feature map of another branch and the feature maps of the middle n-1 second convolutional layers through a third convolutional layer to obtain the spliced ​​feature map. Through this splicing operation, the feature information of different branches can be effectively combined to enhance the feature representation capability. Finally, the spliced ​​feature map passes through a third convolutional layer to obtain the final output feature map. This convolution operation can mainly adjust the number of channels of the output feature map to meet the needs of subsequent network layers.

[0079] (4) The SEAM attention module is used to improve the detection head of the third improved model to obtain the YOLOv8 improved model.

[0080] The SEAM attention module is an occlusion-aware attention module that can enhance attention to occluded areas and improve detection accuracy. By focusing on both spatial and channel information, the SEAM attention module can better capture the detailed features of the target and improve the detection performance of the model in complex scenes.

[0081] It should be noted that the SEAM attention module can perform attention weighting: by introducing an attention module, the importance of each feature channel is learned and weights are assigned accordingly. In this way, the model can enhance useful feature signals while suppressing unimportant or noise signals, especially in the presence of occluders. The SEAM attention module can perform spatial focusing: in the spatial dimension, the spatial layout of the feature map is adjusted through the spatial transformation network, so that the model can better focus on the key parts under occlusion. This spatial focus helps the model obtain more useful information around the objects that are partially occluded visually. The SEAM attention module can dynamically fuse features: combined with dynamic feature fusion technology, the fusion strategy of features at different levels is dynamically adjusted according to the occlusion of the object. Through this adaptive fusion, the model can effectively integrate more contextual information while maintaining high-resolution details, and enhance the ability to analyze occluded areas. The SEAM attention module can optimize real-time performance: although additional attention and spatial transformation calculations are introduced, the SEAM attention module is still designed with the real-time performance of the algorithm in mind. By optimizing the algorithm structure and implementation method, it ensures that effective occlusion processing capabilities are provided without significantly increasing the computational burden.

[0082] like Figure 2 and Figure 6 As shown in the figure, a SEAM module is added to each detection layer of the detection head. The feature map of each detection layer will be processed by the SEAM module first, and then predicted by the subsequent convolution layer. This module improves the detection ability and robustness of the algorithm for occluded targets by strengthening the information expression of the occluded area in the feature map and suppressing the interference of the background area. In the SEAM module, the input feature map first passes through three parallel CSMM (channel and spatial mixing module) modules with different patches. Each CSMM module uses a depthwise separable convolution with residual connection to process the input feature map. Each channel is convolved independently through group convolution, and the features of different channels are fused through pointwise convolution, which greatly reduces the number of parameters and calculations while retaining the information between channels. Subsequently, the SEAM module uses a fully connected layer and an exponential function to weight the feature map. The fully connected layer is responsible for further fusing multi-scale features and mapping the output value to the [1, e] interval through an exponential function to enhance the response of important features and suppress unimportant background information, so that the model can better capture subtle changes in the occluded area.

[0083] In some embodiments, the training environment can be an NVIDIA GeForce RTX3090 GPU with a video memory size of 24GB, an Ubuntu 20.04 operating system, Python3.8 programming language, GPU acceleration using CUDA11.7 and cuDNNv8.2.4, the YOLOv8s dependent library ultralytics version is 8.0.157, and deep learning is based on the Pytroch1.13.1 framework.

[0084] In some embodiments, the training parameters of the YOLOv8 improved model are set as follows: the number of training rounds (Epochs) is 400, SGD is used as the optimizer, the initial learning rate (lr) is 0.01, and the batch size (Batchsize) is 32.

[0085] Step 102: Based on the sample security device images and the sample security device annotated images, the YOLOv8 improved model is trained to obtain a target security device detection model.

[0086] The target safety equipment detection model can perform safety equipment detection in complex scenes and occlusion situations in real time and efficiently, and is particularly suitable for real-time detection of smart construction site safety equipment integrated in embedded devices.

[0087] In some embodiments, the specific implementation of step 102 includes:

[0088] Step 1021: Based on the sample security device image and the sample security device annotated image, train the YOLOv8 improved model to obtain an intermediate YOLOv8 improved model.

[0089] Step 1022: In response to the performance of the intermediate YOLOv8 improved model satisfying the training completion condition, the intermediate YOLOv8 improved model satisfying the training completion condition is used as the target security device detection model; in response to the performance of the intermediate YOLOv8 improved model not satisfying the training completion condition, the training weights and training parameters of the intermediate YOLOv8 improved model are adjusted until the performance of the intermediate YOLOv8 improved model satisfies the training completion condition, and the training parameters include one or more parameters of the initial learning rate, the number of training iterations, and the hyperparameters.

[0090] In some embodiments, the training completion condition is that the number of parameters, computational complexity, and weight of the intermediate YOLOv8 improved model do not decrease for 50 consecutive rounds, and the precision, recall, and mAP are 50 No upgrade for 300 consecutive rounds.

[0091] The performance index comparison table of the security device detection model obtained by training the model training method provided in the embodiment of the present application on the basis of the YOLOv8 model by sequentially introducing the feature fusion pyramid module (HSFPN in Table 1), the dynamic sampling module (Dysample in Table 1), the RepGhostCSPELAN module (RGCSPELAN in Table 1) and the SEAM attention module (SEAM in Table 1) is shown in Table 1. Compared with the YOLOv8 model, the target security device detection model has a parameter amount (P AR ) from 11.1×10 6 Down to 4.5×10 6 The computational effort (F L ) from 28.8×10 9 Down to 12.9×10 9 , reduced by 55.2%; mAP 50 It increased from 93.4% to 94.0%, and FPS (frame number) increased from 249 to 350, an increase of 41%; the model weight size decreased from 21.5M to 9.1M, a decrease of 58%.

[0092] Table 1 Comparison of model performance indicators

[0093]

[0094] In summary, the model training method provided in the embodiment of the present application can reduce the number of parameters and the amount of calculation, improve the detection effect of safety equipment in smart construction sites, and ensure the real-time and high efficiency of detection.

[0095] In a second aspect, an embodiment of the present application provides a method for detecting safety equipment on a smart construction site, the method comprising:

[0096] Obtaining a target security device detection model in the model training method of the first aspect;

[0097] The image of the safety device to be tested is input into the target safety device detection model to obtain the safety device detection result. The safety device detection result includes the target category, target bounding box position, confidence and category probability. The target categories include safety helmets, safety ropes, user heads and user bodies.

[0098] In summary, the smart construction site safety equipment detection method provided in the embodiment of the present application can reduce the number of parameters and calculations, improve the detection effect of safety equipment in the smart construction site, and ensure the real-time and high efficiency of the detection.

[0099] In the present application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. The term "plurality" refers to two or more than two, unless otherwise clearly defined.

[0100] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the present application disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only.

[0101] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A model training method, characterized in that: The method comprises: Obtaining a sample safety device image, a sample safety device annotated image, and a YOLOv8 improved model, wherein the YOLOv8 improved model includes a high-level screening feature fusion pyramid structure, a dynamic sampling module, a RepGhostCSPELAN module, and a SEAM attention module; Based on the sample security device image and the sample security device annotated image, a YOLOv8 improved model is trained to obtain a target security device detection model.

2. The model training method according to claim 1, characterized in that: The step of training the YOLOv8 improved model based on the sample security device image and the sample security device annotated image to obtain a target security device detection model includes: Based on the sample safety device image and the sample safety device annotated image, training a YOLOv8 improved model to obtain an intermediate YOLOv8 improved model; In response to the performance of the intermediate YOLOv8 improved model satisfying the training completion condition, the intermediate YOLOv8 improved model satisfying the training completion condition is used as the target security device detection model; in response to the performance of the intermediate YOLOv8 improved model not satisfying the training completion condition, the training weights and training parameters of the intermediate YOLOv8 improved model are adjusted until the performance of the intermediate YOLOv8 improved model satisfies the training completion condition, and the training parameters include one or more parameters of the initial learning rate, the number of training iterations and the hyperparameters.

3. The model training method according to claim 1, characterized in that: The YOLOv8 improved model is a model improved on the basis of the YOLOv8 model, and the improvement process includes: In the neck network part, a high-level screening feature fusion pyramid structure is introduced to obtain the first improved model; The up-sampling module of the first improved model is replaced by the dynamic sampling module to obtain a second improved model; The RepGhostCSPELAN module is used to replace the traditional C2f module in the backbone network part and the neck network part of the second improved model to obtain a third improved model; The SEAM attention module is used to improve the detection head of the third improved model to obtain the YOLOv8 improved model.

4. The model training method according to claim 2, characterized in that: The training completion condition is that the number of parameters, calculation amount and weight of the intermediate YOLOv8 improved model do not decrease for 50 consecutive rounds, and the precision, recall rate and mAP are 50 No upgrade for 300 consecutive rounds.

5. The model training method according to any one of claims 1 to 4, characterized in that: The high-level screening feature fusion pyramid structure, in which a multi-level feature map is first extracted, and the multi-level feature map is processed by the global average pooling layer and the global maximum pooling layer in the CA module to extract the average response and the maximum response of each channel. The generated feature map is spliced ​​by the connection operation, and then the attention weight value f of each channel is determined by the Sigmoid activation function. CA ∈R C×1×1 , and then multiply the weight value with the feature map of the corresponding scale to generate the filtered attention weight map. Finally, the feature fusion module efficiently fuses the filtered low-level features with the high-level features.

6. The model training method according to any one of claims 1 to 4, characterized in that: The dynamic sampling module is obtained by improving the upsampling module, and is used for dynamic spatial sampling. The spatial dimension of the input tensor is adjusted by applying the learned offset. The dynamic sampling module includes a scaling unit, an offset convolution layer, and a range convolution layer connected in sequence; The scaling unit is used to adjust the input size, the offset convolution layer is used to generate a sampled offset and is initialized with a normal distribution, and the range convolution layer generates a range map when enabled and is initialized with a constant value.

7. The model training method according to any one of claims 1 to 4, characterized in that: The RepGhostCSPELAN module includes a first convolutional layer, a segmentation layer, a re-parameterized convolutional layer, multiple second convolutional layers, a splicing layer and a third convolutional layer connected in sequence. The segmentation layer divides the input feature map into two parts, one part is directly passed to the splicing layer, and the other part is processed by the re-parameterized convolutional layer and then spliced ​​with the outputs of the multiple second convolutional layers. The RepGhostCSPELAN module is used to fuse feature information of different branches, enhance feature representation capabilities by connecting partial features across stages, and optimize the gradient flow path, thereby improving the training efficiency and convergence speed of the model.

8. The model training method according to claim 7, characterized in that: The convolution kernel size of the first convolution layer is 1×1, the step size is 1, the padding is 0, and the number of output channels is 2e×c_out. The first convolution layer is used to adjust the number of channels of the input feature map and perform preliminary fusion of information; The convolution kernel size of the second convolution layer is 3×3, the step size is 1, the padding is 1, and the number of output channels is scale×e×c_out. The second convolution layer is used to generate partially redundant feature maps, which reduces the amount of calculation and parameters of the model; The convolution kernel size of the third convolution layer is 1×1, the step size is 1, the padding is 0, and the number of output channels is c_out. The third convolution layer is used to perform the final channel number adjustment and feature fusion on the spliced ​​feature map, and output the processing result of the module; Among them, e represents the implicit channel coefficient, which is used to control the channel expansion ratio inside the module, c_out represents the number of output channels, that is, the number of feature map channels after module processing, and scale represents the feature map scaling coefficient, which is used to adjust the scale of feature maps of different branches.

9. The model training method according to claim 7, characterized in that: Acquiring a sample safety device image and a sample safety device annotated image includes: Constructing an original safety equipment image, wherein the original safety equipment image includes a construction site environment image under different shooting angles, different distances, different weather conditions, and different lighting conditions; The safety helmet, safety rope, user head and user body in the original safety equipment image are labeled by using labels, and the label format of the labels is converted from JSON format to a file format supported by YOLO to obtain the original safety equipment labeled image, wherein the labels include target categories, center point coordinates of the labeled bounding box and width of the labeled bounding box, and the target categories include safety helmet, safety rope, user head and user body; Dividing the original safety device image and the original safety device annotated image into a training set, a verification set and a test set; The training set, the validation set and the test set are sequentially subjected to image cutting and pasting processing, label fusion processing, and multiple image enhancement processing to obtain enhanced training set, validation set and test set, wherein the enhanced training set includes sample safety equipment images and sample safety equipment annotated images.

10. A smart construction site safety equipment detection method, characterized in that: The method comprises: Obtaining a target safety device detection model in the model training method as described in any one of claims 1 to 9; The image of the safety device to be tested is input into the target safety device detection model to obtain a safety device detection result, wherein the safety device detection result includes a target category, a target bounding box position, a confidence level, and a category probability. The target categories include a safety helmet, a safety rope, a user's head, and a user's body.

Images