A safety belt detection method and system based on a new YOLOv10

Abstract

The application provides a safety belt detection method and system based on a novel YOLOv10, which comprises the following steps: acquiring image data and labeling, that is, labeling the safety belt position / wearing state; constructing a detection model based on the novel YOLOv10 network, that is, using a local attention regularization convolution to replace a standard convolution Conv in a Backbone network; using a separable convolution star module to replace an original feature extraction module of the Backbone network; using a CARAFE module to replace an up-sampling operation in a Neck network, and additionally adding a multi-scale spatial attention dilated convolution fusion module before three groups of feature maps output by the Neck network, and then outputting the feature maps to a Head network; and training the detection model by using the labeled image data to obtain a safety belt detection model. The novel YOLOv10 network is used for safety belt detection, and the detection efficiency and performance can be significantly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and particularly relates to a safety belt detection method and system based on a new YOLOv10. BACKGROUND

[0002] As an important branch of artificial intelligence, computer vision has made remarkable progress in recent years. With the wide application of deep learning technology, the performance of computer vision in image recognition, target detection, semantic segmentation and other aspects has been greatly improved. In particular, in the field of target detection, through the application of convolutional neural network (CNN), the model can automatically extract image features and accurately locate the target, greatly improving the accuracy and efficiency of detection. In addition, some advanced target detection frameworks such as Faster R-CNN, YOLO and SSD have also been proposed, which realize faster detection speed and higher accuracy through improving network structure and optimizing algorithm. The development of these technologies provides strong support for the upgrading of intelligent traffic management, autonomous driving and other fields.

[0003] With the increasing number of cars, road traffic is increasingly dense, and the incidence of traffic accidents also rises. In this context, the wearing of safety belts is particularly important. Safety belts are the first line of defense to protect passengers from injury in the event of a vehicle collision. They can firmly fix passengers on the seat at critical moments, avoid being thrown out of the car or causing secondary collisions in the car, thereby greatly reducing the risk of injury or even death.

[0004] Existing safety belt detection methods can be broadly divided into two categories. One is to rely on the sensors and algorithms built into the vehicle to monitor the wearing state of the safety belt, but this method is sometimes limited by various methods, resulting in false positives or false negatives. In addition, these detection methods are too simple in design, making passengers pay insufficient attention to safety belt wearing. Therefore, safety belt detection technology needs to be more accurate and intelligent, and the development of target detection in the field of computer vision has promoted the development of another safety belt detection method. By detecting whether the driver and passengers are wearing safety belts in image data, the safety of the driver's driving behavior can be effectively improved. However, the models used in existing research still have many shortcomings when applied to safety belt detection, and the efficiency and performance need to be further improved and enhanced. SUMMARY

[0005] The purpose of the present application is to solve the technical problem of low efficiency and performance of existing models applied to safety belt detection in the field of vision. The present application provides a safety belt detection method and system based on a new YOLOv10, which improves the efficiency and performance of the model. The network architecture of the YOLOv10 network is optimized. On the one hand, the separable convolution star module StarBlock-SC is used to replace the original feature extraction module C2f of the Backbone network. On the other hand, before the three groups of feature maps output by the Neck network, a multi-scale spatial attention expansion convolution fusion module DAS-Block is added, and then the feature maps are output to the Head network. The introduction of StarBlock-SC and DAS-Block can improve the detection efficiency and the target detection performance.

[0006] To this end, the following technical solutions are provided:

[0007] The present application provides a safety belt detection method based on a new YOLOv10, which comprises the following steps:

[0008] S1: Obtain image data and perform labeling, wherein the image data at least contains safety belt images, and the safety belt position / wearing state is labeled;

[0009] S2: Construct a detection model based on a new YOLOv10 network, wherein the new YOLOv10 network is sequentially provided with an Input input end, a Backbone network, a Neck network and a Head network;

[0010] The original feature extraction module C2f of the Backbone network is replaced by the separable convolution star module StarBlock-SC, and before the three groups of feature maps output by the Neck network, a multi-scale spatial attention expansion convolution fusion module DAS-Block is added, and then the feature maps are output to the Head network;

[0011] S3: Train the detection model constructed in step S2 using the labeled image data to obtain a safety belt detection model for safety belt wearing detection.

[0012] Optionally, the data processing process of the separable convolution star module StarBlock-SC is as follows:

[0013] First, perform separable convolution operation on the feature map data x0 input into the separable convolution star module StarBlock-SC to extract features;

[0014] Then, perform ConvBN operation twice on the result of the separable convolution operation to obtain x1 and x2;

[0015] Then, x1 is processed by using an activation function ReLU6, and then an element-wise multiplication operation ReLU6(x1)*x2 is performed on x1 and x2;

[0016] Then, x1 and x2 are restored to the dimension of the input feature by using a convolution layer, and a second separable convolution operation is performed to extract second features;

[0017] Finally, the input feature map data x0 is connected in residual with the result after the second separable convolution operation to obtain the output of the separable convolution star module StarBlock-SC.

[0018] The element-wise multiplication operation ReLU6(x1)*x2 is as follows:

[0019] Let ReLU6(x1) be y1 and x2 be y2, and the element-wise multiplication can be written as:

[0020]

[0021] In the formula, i and j are used to index the channels; y1 and y2 are the element values corresponding to the index i, j, alpha describes the mutual influence or weight relationship between the channels, and beta is the product or sum of the weights between different channels.

[0022]

[0023] In the formula, ω1 and ω2 are weight vectors of y1 and y2, respectively, y1 and y2 are the element values corresponding to the index i, j, T is the transpose symbol, d is the number of input channels, i.e., the depth of the input data, and different features can be transmitted to the convolution layer for processing by setting different d, and different feature information can be extracted.

[0024] Optionally, the operation process of the multi-scale spatial attention dilated convolution fusion module DAS-Block is to sequentially perform dilated convolution operation, splicing fusion, spatial attention mechanism module execution operation, and residual connection, specifically:

[0025] The input data of the multi-scale spatial attention dilated convolution fusion DAS-Block is defined as X, and the input data X is subjected to three dilated convolution operations, and the dilated convolution operation is represented as:

[0026] Y d =X*K d +b

[0027] In the formula, Y d is the output of the dilated convolution, K dis an expansion convolution kernel, b is a bias term, and * represents a convolution operation;

[0028] The outputs of different expansion convolution paths are fused by using a multi-scale block, denoted as:

[0029] Y ms d1 d2 d3

[0030] In the formula, Y ms represents the fusion result, Y d1 , Y d2 , and Y d3 are the results of the first, second, and third expansion convolution operations, respectively;

[0031] The fusion result Y m is compressed using a 1x1 convolution, denoted as:

[0032] Y ms ′=W 1×1 ·Y ms

[0033] In the formula, Y ms ′ represents the result after channel compression, and W 1×1 is the weight of the 1x1 convolution;

[0034] Then, a spatial attention mechanism module is used to generate an attention map and weight the fusion result Y ms , that is, the spatial attention of the features is calculated through a convolution operation, and the weighted result is used as:

[0035] A=Sigmoid(W s ·Y ms ′+b s )

[0036] Z=A·Y ms

[0037] In the formula, W s is the weight of the spatial attention convolution, b s is the bias, A is the attention map, and Z is the weighted result;

[0038] Finally, the weighted result is connected in residual connection with the result Y ms ′ after channel compression, denoted as:

[0039] Y output =Z+Y ms ′

[0040] Wherein, the residual connection is connected as the output of the multi-scale spatial attention expansion convolution fusion module DAS-Block.​​​​

[0041] Optionally, the CARAFE module is used in the new YOLOv10 network to replace the bilinear interpolation upsampling operation in the Neck network.

[0042] The CARAFE module is composed of a kernel prediction module and a content-aware reorganization module.

[0043] The feature map X input into the CARAFE module is defined as: l The kernel prediction module is:

[0044] W l′ = ψ(N(X l ,k encoder ))

[0045] In the formula, X l is the target position in the input feature map X, the target position l' is the position in the output feature map obtained after the CARAFE operation, W l′ is the reorganization kernel predicted at the target position l', ψ represents the kernel prediction module, which extracts a sub-region centered at the target position l' from the input feature map X l and processes it through the encoder kernel k encoder , N is a nonlinear function using ReLU6 or other activation functions;

[0046] The content-aware reorganization module is represented as:

[0047] X′ l′ = Φ(N(X l ,k up ), W l′ )

[0048] In the formula, X l is the new feature map generated at the target position l', Φ represents the content-aware reorganization module, which reorganizes the input feature using the upsampling kernel k up and the predicted reorganization kernel W l′ .

[0049] Optionally, the LAR-Conv structure is used in the new YOLOv10 network to replace part of the standard convolution Conv in the Backbone network and the Neck network.

[0050] The LAR-Conv structure sequentially performs standard convolution Conv, local attention module, residual connection, and regularization, specifically:

[0051] First, the feature map input to the LAR-Conv structure is convolved using the standard convolution Conv;

[0052] Then, the feature map after convolution is weighted to obtain a weighted feature map using the attention weight of the local attention module;

[0053] Next, the feature map after convolution and the weighted feature map are connected using a residual connection;

[0054] Finally, the feature map after residual connection is regularized.

[0055] Optionally, four separable convolution star modules StarBlock-SC are arranged in the Backbone network;

[0056] The Backbone network includes, in sequence, a standard convolution layer Conv, a LAR-Conv_1 structure, a separable convolution star module StarBlock-SC_1, a LAR-Conv_2 structure, a separable convolution star module StarBlock-SC_2, a down-sampling module SCDown_1, a separable convolution star module StarBlock-SC_3, a down-sampling module SCDown_2, a separable convolution star module StarBlock-SC_4, an attention module SPPF, and an attention module PSA;

[0057] The separable convolution star module StarBlock-SC_2, the separable convolution star module StarBlock-SC_3, and the attention module PSA correspond to output feature maps P3, P4, and P5, respectively, and input the Neck network, wherein the feature maps P3, P4, and P5 are processed in the Neck network as follows:

[0058] The feature map P5 is fused with the feature map P4 in a full connection layer Concat_1 through a up-sampling module CARAFE_1 of the Neck network; after fusion, the feature map is fused with the feature map P3 in a full connection layer Concat_2 through a feature extraction module C2f_1 of the Neck network, a up-sampling module CARAFE_2, and a feature extraction module C2f_2; then, a feature F1 is obtained, and the feature F1 is input to a first multi-scale spatial attention dilated convolution fusion DAS-Block_1;

[0059] The feature F1 is fused with the feature F2 through a local attention regularization convolution layer LAR-Conv_3, a full connection layer Concat_3, and a feature extraction module C2f_3, and the feature F2 is input to a second multi-scale spatial attention dilated convolution fusion DAS-Block_2.

[0060] The feature F2 is fused with the feature map P5 at the full connection layer Concat_4 through the downsampling module SCDown_3, and the feature F3 is obtained through the feature extraction module C2fCIB, the feature F3 is input into the third multi-scale spatial attention dilated convolution fusion DAS-Block_3;

[0061] The outputs of the three multi-scale spatial attention dilated convolution fusions DAS-Block are input into the Head network, and the Head network is correspondingly provided with three target detection modules v10Detect, and the results of the three target detection modules v10Detect are convolved and fused to obtain the insulator detection result.

[0062] Optionally, the image data obtained in the step S1 comprises: a windshield image, a personnel image, a safety belt image, or a windshield image containing personnel and a safety belt, and the windshield, the personnel, and the safety belt wearing state are labeled respectively;

[0063] The detection model based on the novel YOLOv10 network built in the step S2 is used for training the windshield detection model, the personnel detection model, and the safety belt detection model respectively;

[0064] Then, the trained windshield detection model, the personnel detection model, and the safety belt detection model are used for sequentially performing windshield detection, personnel detection, and safety belt detection on the image to be detected.

[0065] Among them, the windshield is segmented based on the windshield detection result, then the segmented windshield image is subjected to personnel detection, and the personnel is segmented based on the personnel detection result, and then the segmented personnel image is subjected to safety belt detection.

[0066] The application further provides a detection system based on the above detection method, comprising:

[0067] An image acquisition module is configured to acquire image data and perform labeling, wherein the image data at least comprises a safety belt image, and the safety belt position / wearing state is labeled;

[0068] A detection model construction module is configured to construct a detection model based on a novel YOLOv10 network, wherein the novel YOLOv10 network is sequentially provided with an Input input end, a Backbone network, a Neck network, and a Head network.

[0069] Wherein, the separable convolution star module StarBlock-SC is used to replace the original feature extraction module C2f of the Backbone network; and before the three groups of feature maps output by the Neck network, a multi-scale spatial attention expansion convolution fusion module DAS-Block is additionally arranged respectively, and then the feature maps are output to the Head network.

[0070] A model training module is configured to train the detection model constructed in step S2 by using the labeled image data, so as to obtain a safety belt detection model for safety belt wearing detection.

[0071] The application further provides a computer terminal, which at least comprises:

[0072] One or more processors;

[0073] A memory storing one or more computer programs;

[0074] The processor calls the computer program to realize:

[0075] A safety belt detection method based on a new YOLOv10.

[0076] The application further provides a computer readable storage medium storing a computer program, and the computer program is called by a processor to realize:

[0077] A safety belt detection method based on a new YOLOv10.

[0078] Advantages

[0079] Compared with the prior art, the application has the following advantages:

[0080] 1. The technical scheme of the application is based on the YOLOv10 network, and the network architecture is optimized to improve the efficiency and performance of safety belt detection.

[0081] 2. A further optimization scheme of the present invention uses local attention regularized convolution LAR-Conv to replace some standard convolution Conv in order to better detect and focus on features related to the target task. In particular, when dealing with complex backgrounds and occlusions, it can more effectively distinguish the difference between the target and the background, thereby improving the accuracy and reliability of seat belt detection, and also improving the generalization ability of the model on unseen data; and in the upsampling operation, CARAFE is used to replace the original bilinear interpolation, which is flexible, content-aware, and has higher expressive power; finally, the improved YOLOv10 model has higher performance and efficiency, as well as better generalization ability.

[0082] In summary, the StarBlock-SC used in the technical solution of the present invention is efficient and concise, DAS-block can improve the performance of target detection, LA-Conv pays more attention to local features and improves detection capabilities, and CARAFE is lightweight and effective. The present invention integrates it into the YOLOv10 network, cleverly sets its position and structure, and significantly improves the seat belt detection performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 This is the architecture diagram of the seat belt recognition system based on the improved YOLOv10 of the present invention;

[0084] Figure 2 A structural diagram of a seat belt identification system in an example of the present invention;

[0085] Figure 3 This is the improved structure diagram of the YOLOv10 algorithm of the present invention;

[0086] Figure 4 This is the structure diagram of the separable convolution star module StarBlock-SC in the example of the present invention.

[0087] Figure 5 This is the LAR-Conv structure diagram in the example of the present invention.

[0088] Figure 6 This is a structural diagram of CARAFE in an example of the present invention.

[0089] Figure 7 This is a structural diagram of the DAS-Block in an example of the present invention. DETAILED DESCRIPTION

[0090] The present invention will be further described below with reference to the embodiments.

[0091] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not used to limit the present application. In addition, the technical features involved in the various embodiments of the present application described below can be combined with each other as long as they do not conflict with each other. The present application will be further described in detail below with reference to the drawings and specific embodiments, as follows:

[0092] The safety belt detection method based on the new YOLOv10 provided by the embodiment of the present application comprises the following steps:

[0093] S1: Obtain image data and perform labeling, wherein the image data at least contains safety belt images, and the safety belt position / wearing state is labeled.

[0094] In the embodiment, the windshield detection model, the personnel detection model and the safety belt detection model are preferably constructed, and then in actual image detection, the windshield is segmented based on the windshield detection result, the personnel is detected on the segmented windshield image, the personnel is segmented based on the personnel detection result, and the safety belt is detected on the segmented personnel image. Therefore, the image data obtained in the embodiment preferably contains the windshield image, the personnel image, the safety belt image or the windshield image containing the personnel and the safety belt at the same time, and the wearing state of the windshield, the personnel and the safety belt is labeled respectively. In other feasible embodiments, the safety belt image is obtained and the safety belt is labeled, which also falls within the protection scope of the present application.

[0095] In the embodiment, the training data and the verification data are labeled by a labeling tool to obtain a training data set and a verification data set, and the quantity ratio is 9:1. The training data and the verification data are obtained from Kaggle. Specifically, the image is resized to a 512x512 pixel image, the training data and the verification data are labeled by Make sense, including three types of data sets, one type of data is labeled as windshield, one type of data is labeled as driver, copilot and rear passenger, and the last type of data is labeled as wearing safety belt and not wearing safety belt.

[0096] S2: a detection model based on a new YOLOv10 network is constructed, the new YOLOv10 network is sequentially provided with an Input input end, a Backbone network, a Neck network and a Head network. The technical scheme of the present application uses a separable convolution star module StarBlock-SC to replace the original feature extraction module C2f of the Backbone network; and before three groups of feature maps output by the Neck network, a multi-scale spatial attention expansion convolution fusion module DAS-Block is additionally provided respectively, and then the feature maps are output to the Head network. In addition, the present embodiment also preferably uses a CARAFE module to replace the original bilinear interpolation upsampling operation in the Neck network in the new YOLOv10 network, and uses a LAR-Conv structure to replace part of the standard convolution Conv in the Backbone network and the Neck network in the new YOLOv10 network.

[0097] Further, the Backbone network of the new YOLOv10 network provided by the present embodiment is provided with four separable convolution star modules StarBlock-SC;

[0098] The Backbone network includes a standard convolution layer Conv, a local attention regularization convolution LAR-Conv_1 structure, a separable convolution star module StarBlock-SC_1, a local attention regularization convolution LAR-Conv_2 structure, a separable convolution star module StarBlock-SC_2, a downsampling module SCDown_1, a separable convolution star module StarBlock-SC_3, a downsampling module SCDown_2, a separable convolution star module StarBlock-SC_4, an attention module SPPF and an attention module PSA connected in sequence; the separable convolution star module StarBlock-SC_2, the separable convolution star module StarBlock-SC_3 and the attention module PSA correspond to the output feature map P3, the feature map P4 and the feature map P5 respectively, and input the Neck network.

[0099] Regarding the Neck network:

[0100] The processing process of the feature map P3, the feature map P4 and the feature map P5 in the Neck network is:

[0101] The feature map P5 is fused with the feature map P4 in a full connection layer Concat_1 through an up-sampling module CARAFE_1 of the Neck network; after the fusion, the feature is fused with the feature map P3 in a full connection layer Concat_2 through a feature extraction module C2f_1 of the Neck network and an up-sampling module CARAFE_2; then, the feature is extracted through a feature extraction module C2f_2 to obtain a feature F1, and the feature F1 is input into a first multi-scale spatial attention dilated convolution fusion DAS-Block_1;

[0102] The feature F1 is further fused through a local attention regularization convolution layer LAR-Conv_3, a full connection layer Concat_3 and a feature extraction module C2f_3 to obtain a feature F2, and the feature F2 is input into a second multi-scale spatial attention dilated convolution fusion DAS-Block_2;

[0103] The feature F2 is fused with the feature map P5 in a full connection layer Concat_4 through a down-sampling module SCDown_3, and a feature F3 is obtained through a feature extraction module C2fCIB, and the feature F3 is input into a third multi-scale spatial attention dilated convolution fusion DAS-Block_3;

[0104] The outputs of the three multi-scale spatial attention dilated convolution fusions DAS-Block are input into a Head network, and the Head network is correspondingly provided with three target detection modules v10Detect, and the results of the three target detection modules v10Detect are fused through convolution to obtain an insulator detection result.

[0105] The network optimization part of the application will be described in detail below.

[0106] Regarding the separable convolution star module StarBlock-SC, the data processing process is as follows:

[0107] First, the feature of the feature map data x0 input into the separable convolution star module StarBlock-SC is extracted through a separable convolution operation;

[0108] Then, the result processed through the separable convolution operation is processed twice through ConvBN operations to obtain x1 and x2;

[0109] Then, x1 is processed through an activation function ReLU6, and then multiplied with x2 element by element ReLU6(x1)*x2;

[0110] Then, the dimensions of x1 and x2 are restored to the dimensions of the input feature through a convolution layer, and secondary features are extracted through a secondary separable convolution operation;

[0111] Finally, the input feature map data x0 is connected in residual with the result after the quadratic separable convolution operation to obtain the output of the separable convolution star module StarBlock-SC.

[0112] where the element-wise multiplication operation ReLU6(x1)*x2 is as follows:

[0113] Let ReLU6(x1) be y1 and x2 be y2, and the element-wise multiplication can be written as:

[0114]

[0115] where i, j are used to index the channels; are the element values corresponding to the index i, j in y1 and y2, respectively, and a describes the mutual influence or weight relationship between channels, is the coefficient of different channel combinations, and is the product or sum of weights between different channels;

[0116]

[0117] where ω1, ω2 are weight vectors of y1 and y2, respectively, are the element values corresponding to the index i, j in ω1 and ω2, respectively, T is the transpose symbol, d is the number of input channels, i.e., the depth of the input data, and by setting different d, different features can be respectively transmitted to the convolution layer for processing and different feature information can be extracted.

[0118] Regarding the multi-scale spatial attention dilated convolution fusion module DAS-Block, the operation process is to sequentially perform dilated convolution operation (dilated convolution that produces different feature maps), splicing fusion (multi-scale block that splices and fuses the output feature maps of different dilated convolution paths), spatial attention mechanism module execution operation (highlighting important regions), and residual connection, specifically:

[0119] Defining the input data of the multi-scale spatial attention dilated convolution fusion DAS-Block as X, first performing three dilated convolution operations on the input data X, and the dilated convolution operation is represented as:

[0120] Y d =X*K d +b

[0121] where Y d is the output of the dilated convolution, K d is the dilated convolution kernel, b is the bias term, and * represents the convolution operation;

[0122] Then, the outputs of different dilated convolution paths are fused using a multi-scale block, represented as:

[0123] Yms =Concat(Y d1 ,Y d2 ,Y d3 )

[0124] wherein Y ms represents the fusion result, Y d1 , Y d2 , Y d3 are the results of the first, second and third expansion convolution operations respectively;

[0125] The fusion result Y m is compressed using a 1x1 convolution, denoted as:

[0126] Y ms ′=W 1×1 ·Y ms

[0127] wherein Y ms ′ represents the result after channel compression, W 1×1 is the weight of the 1x1 convolution;

[0128] Then, a spatial attention mechanism module is used to generate an attention map and the fusion result Y ms is weighted using the attention map, i.e. the spatial attention of the features is calculated through a convolution operation, and is used;

[0129] A=Sigmoid(W s ·Y ms ′+b s )

[0130] Z=A·Y ms

[0131] wherein W s is the weight of the spatial attention convolution, b s is the bias, A is the attention map, and Z is the weighted result;

[0132] Finally, the weighted result is connected in residual connection with the result Y ms ′ after channel compression, denoted as:

[0133] Y output =Z+Y ms ′

[0134] wherein the residual connection is the output of the multi-scale spatial attention expansion convolution fusion module DAS-Block.

[0135] Regarding the CARAFE module, it is composed of a kernel prediction module and a content-aware reorganization module. The kernel prediction module generates reorganization kernels according to the content of the input features through operations such as channel compression, content encoding, and kernel normalization. The content-aware reorganization module reorganizes the local region of the input feature map into a new output feature map using the reorganization kernel through operations such as zero padding, neighborhood extraction, and matrix multiplication. This process ensures that the output feature map not only has higher resolution, but also effectively preserves and enhances the semantic information of the input features.

[0136] Definition of the feature map X input to the CARAFE module l The kernel prediction module is:

[0137] W l′ = ψ(N(X l ,k encoder ))

[0138] In the formula, W l′ is the reorganization kernel predicted at the target position l', ψ represents the kernel prediction module, which extracts a sub-region (neighborhood) centered at the target position l' from the input feature map X l and processes it through the encoder kernel k encoder , N is a nonlinear function using ReLU6 or other activation functions; the purpose of this formula is to generate a content-aware kernel for each target position for subsequent reorganization.

[0139] The content-aware reorganization module is represented as:

[0140] X′ l′ = Φ(N(X l ,k up ), W l′ )

[0141] In the formula, X l is the new feature map generated at the target position l', Φ represents the content-aware reorganization module, which reorganizes the input features using the up-sampling kernel k up and the predicted reorganization kernel W l′ . This formula describes how to reorganize the original feature map through kernel reorganization to generate a new high-resolution feature map.

[0142] As to the LAR-Conv structure, it sequentially performs a standard convolution Conv, a local attention module, a residual connection, and a regularization. The local attention module is used to calculate the local attention weight of the input feature map to emphasize important local features, that is, the attention weight is applied to the output feature map to highlight important features while retaining the information of other features; the regularization layer is used to prevent overfitting of the model. Based on the above structure, the LAR-Conv can enhance the response of the model to important features, improve the detection performance, effectively reduce overfitting, and improve the generalization ability of the model.

[0143] The operation process of the LAR-Conv structure is specifically as follows:

[0144] 1. First, the standard convolution Conv is used to perform convolution operation on the feature map input into the LAR-Conv structure. The input feature map (B, C, H, W) is first processed by the convolution layer, and the output feature map (B, C', H', W') is obtained, where B is the batch size, C is the channel number, H is the height of the feature map, W is the width of the feature map, C' is the channel number of the output of the convolution layer, and H' and W' are the height and width of the feature map after convolution processing.

[0145] 2. Then, the attention weight of the local attention module is used to weight the feature map after the convolution operation to obtain a weighted feature map. This step can be divided into the following steps:

[0146] The first step is to use a 1x1 convolution to generate query (Q) and key (K) features for calculating attention weight, which is represented as:

[0147] Q=W q ·X

[0148] K=W k ·X

[0149] wherein W q and W k are the weights of the 1x1 convolution, and X is the feature map after the convolution operation.

[0150] The second step is to calculate the attention score between the query and the key, which uses a dot product operation to form an attention matrix, which is represented as:

[0151]

[0152] wherein d k is the dimension of the key, which is used to scale the attention score, T is the transpose matrix, and Attention(Q, K) is the attention score.

[0153] The third step is to apply a softmax function to the attention matrix to obtain the attention weight, which represents the importance of each channel, which is represented as:

[0154] a = softmax(Attention(Q, K))

[0155] where a is the attention weight, the softmax function is a normalization operation, and Attention(Q, K) is the attention score.

[0156] The fourth step is feature weighting, i.e., applying the attention weight to the input feature map to highlight important features while preserving information of other features, represented as:

[0157] Z = a X

[0158] where X is the feature map after convolution operation, and a is the attention weight.

[0159] Then, the feature map after convolution operation and the weighted feature map are connected using residual connection, represented as:

[0160] X' = Z + X

[0161] where X' is the feature map after residual connection, Z is the weighted feature map, and X is the feature map after convolution operation.

[0162] Finally, the feature map after residual connection is subjected to regularization processing, represented as:

[0163] X" = Dropout(X')

[0164] where X' is the feature map after residual connection, and X" is the feature map after regularization operation.

[0165] S3: Train the detection model constructed in step S2 using the labeled image data to obtain a safety belt detection model for safety belt wearing detection.

[0166] The training parameters are the hyperparameters in the modified configuration file, and the model is trained based on the new YOLOv10 network. The specific process is as follows: the hyperparameters set in the configuration file are loaded into the model; a group of images is randomly extracted from the training data set as the current training sample; each image in the training sample is input into the model in turn to update the weight parameters, wherein the update of the weight parameters is as follows: the weight parameters are randomly initialized, then the forward propagation calculation of the convolutional neural network is performed to obtain a group of intermediate parameters, and then the intermediate parameters are used to update the weight parameters; the new weight parameters will replace the old weight parameters used for calculating the forward propagation before iteration; the process of completing the forward and backward propagation of all images in the training sample is called one training of the model; the above steps are repeated until the number of model training reaches the set iteration number; after the entire training is completed, the candidate weight file is screened out. The target weight is obtained by evaluating and quantifying the candidate weight according to the validation data set; the target weight is used to update the candidate weight, and the construction of the model is completed.

[0167] In this embodiment, the detection model based on the new YOLOv10 network built in step S2 is used to train the windshield detection model, the personnel detection model and the safety belt detection model respectively.

[0168] Step 4: The trained windshield detection model, personnel detection model and safety belt detection model are used to perform windshield detection, personnel detection and safety belt detection on the image to be detected in turn.

[0169] Among them, the windshield is segmented based on the windshield detection result, then the segmented windshield image is subjected to personnel detection, and the personnel is segmented based on the personnel detection result, and then the segmented personnel image is subjected to safety belt detection.

[0170] To sum up, the original YOLOv10 network is optimized, a brand new simple and efficient separable convolution star module StarBlock-SC is used to replace the feature extraction module C2f, which not only reduces the parameter quantity and calculation quantity of the model, but also fuses the information between different channels and improves the feature expression ability. In order to better detect and focus on the features related to the target task, a local attention regularization convolution LAR-Conv is proposed to replace part of the standard convolution Conv. The feature is that when dealing with complex background and occlusion, it can more effectively distinguish the difference between the target and the background, thereby improving the accuracy and reliability of the safety belt detection, and can help to reduce overfitting, thereby improving the generalization ability of the model on unseen data. In order to improve the detection performance of the safety belt, in the Neck network, the CARAFE is used to replace the original bilinear interpolation up-sampling operation, and the multi-scale spatial attention dilated convolution fusion DAS-Block is used after the last three feature extraction modules, which can significantly improve the feature extraction ability, context understanding ability and robustness of the model in dealing with complex scenes.

[0171] In some embodiments, the present application also provides a detection system based on the above detection method, comprising: an image acquisition module, a detection model construction module and a model training module.

[0172] The image acquisition module is used to acquire image data and perform labeling, wherein the image data at least contains safety belt images, and the safety belt position / wearing state is labeled.

[0173] The detection model construction module is used to construct a detection model based on a new YOLOv10 network, wherein the new YOLOv10 network is sequentially provided with an Input input end, a Backbone network, a Neck network and a Head network.

[0174] The separable convolution star module StarBlock-SC is used to replace the original feature extraction module C2f of the Backbone network, and the multi-scale spatial attention dilated convolution fusion module DAS-Block is additionally provided before the three groups of feature maps output by the Neck network, and then the feature maps are output to the Head network.

[0175] The model training module is used to train the detection model constructed in step S2 by using the labeled image data, so as to obtain a safety belt detection model for safety belt wearing detection.

[0176] In some embodiments, the detection system further comprises a detection module for performing seat belt wearing detection using the constructed seat belt detection model. If the windshield detection model, the person detection model and the seat belt detection model are constructed synchronously, the windshield detection model, the person detection model and the seat belt detection model are used in sequence to perform windshield detection, person detection and seat belt detection.

[0177] It should be understood that the specific implementation process of each module can refer to the above method content, and the present application will not be repeated here, and the division of the above functional modules is only for illustration. In some embodiments, part of the functional modules can be combined, part of the functional modules can be split, and each functional module can be implemented in software, hardware or a combination of software and hardware. The soft and hardware devices include but are not limited to general-purpose computer devices, programmable gate arrays, digital signal processors, microprocessors and their corresponding programming or burning software.

[0178] In some embodiments, the present application also provides a computer terminal, comprising at least one or more processors and a memory storing one or more computer programs.

[0179] The processor calls the computer program to implement a new YOLOv10-based seat belt detection method.

[0180] Specifically, the processor calls the computer program to implement:

[0181] S1: Obtain image data and perform labeling, wherein the image data at least contains seat belt images, and the seat belt position / wearing state is labeled.

[0182] S2: Construct a detection model based on the new YOLOv10 network.

[0183] S3: Train the detection model constructed in step S2 using the labeled image data to obtain a seat belt detection model for seat belt wearing detection.

[0184] The specific implementation process of each step can refer to the description of the foregoing method.

[0185] It should be understood that in the embodiments of the present application, the processor can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The memory can include read-only memory and random access memory, and provide instructions and data for the processor. A part of the memory can also include non-volatile random access memory. For example, the memory can also store device type information.

[0186] In some embodiments, the present application also provides a computer readable storage medium storing a computer program, wherein the computer program is called by a processor to implement the steps of a safety belt detection method based on a new YOLOv10.

[0187] Specifically, the steps are as follows:

[0188] S1: Obtain image data and perform labeling, wherein the image data at least contains safety belt images, and the safety belt position / wearing state is labeled.

[0189] S2: Construct a detection model based on a new YOLOv10 network.

[0190] S3: Train the detection model constructed in step S2 using the labeled image data to obtain a safety belt detection model for safety belt wearing detection.

[0191] The specific implementation process of each step is described in the foregoing method.

[0192] The readable storage medium is a computer readable storage medium, which can be an internal storage unit of the software and hardware device in any of the foregoing embodiments, such as a hard disk or a memory of the controller. The readable storage medium can also be an external storage device of the controller, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the readable storage medium can include both the internal storage unit and the external storage device of the controller. The readable storage medium is used to store the computer program and other programs and data required by the controller. The readable storage medium can also be used to temporarily store data that has been output or will be output.

[0193] Based on such understanding, the technical solutions of the present application, essentially or in the sense of contribution to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The foregoing readable storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0194] Those skilled in the art will appreciate that embodiments of the present application can be readily used as a method, apparatus, or computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer readable storage media (including, but not limited to, disk memory, CD-ROMs, optical storage devices, etc.) embodying computer readable program code embodied thereon. The present application is directed to a method, apparatus, and computer program product for performing the functions described herein. The present application is directed to a method, apparatus, and computer program product for performing the functions described herein. The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks or in the flowchart or block diagram block or blocks.

[0195] Those skilled in the art will appreciate that the embodiments of the present application can be readily used as a method, apparatus, or computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer readable storage media (including, but not limited to, disk memory, CD-ROMs, optical storage devices, etc.) embodying computer readable program code embodied thereon. The present application is directed to a method, apparatus, and computer program product for performing the functions described herein. The present application is directed to a method, apparatus, and computer program product for performing the functions described herein. The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks or in the flowchart or block diagram block or blocks.

[0196] While the present application has been shown and described with reference to particular embodiments thereof, it will be understood by those skilled in the art that various changes in form and details can be made therein without departing from the spirit and scope of the application. Accordingly, the scope of the present application is defined only by the appended claims and equivalents thereof.

[0197] It should be emphasized that the examples described herein are intended to be illustrative only and are not intended to limit the present application, which is defined by the claims. Any other embodiments, modifications, and improvements derived from the technical solutions of the present application, whether modified or replaced, as long as they do not depart from the spirit and scope of the present application, are also within the protection scope of the present application.

Claims

1. A new seat belt detection method based on YOLOv10, characterized by: The following steps are involved: S1: Acquire and annotate image data, wherein the image data at least includes a seat belt image and annotates the seat belt position / wearing status; S2: Build a detection model based on the new YOLOv10 network, where the new YOLOv10 network is sequentially provided with an input terminal, a backbone network, a neck network, and a head network; Among them, the separable convolutional star module StarBlock-SC is used to replace the original feature extraction module C2f of the Backbone network; and before the three sets of feature maps output by the Neck network, a multi-scale spatial attention expansion convolution fusion module DAS-Block is added, and then the feature maps are output to the Head network; S3: Use the labeled image data to train the detection model constructed in step S2 to obtain a seat belt detection model for seat belt wearing detection; The Backbone network is equipped with four separable convolution star modules StarBlock-SC; The Backbone network includes the standard convolution layer Conv, LAR-Conv_1 structure, separable convolution star module StarBlock-SC_1, LAR-Conv_2 structure, separable convolution star module StarBlock-SC_2, downsampling module SCDown_1, separable convolution star module StarBlock-SC_3, downsampling module SCDown_2, separable convolution star module StarBlock-SC_4, attention module SPPF and attention module PSA connected in sequence; Among them, the separable convolution star module StarBlock-SC_2, the separable convolution star module StarBlock-SC_3, and the attention module PSA respectively output feature maps P3, P4, and P5, and input them into the Neck network. The processing process of the feature maps P3, P4, and P5 in the Neck network is: The feature map P5 is fused with the feature map P4 in the fully connected layer Concat_1 through the upsampling module CARAFE_1 of the Neck network. After fusion, it is fused with the feature map P3 in the fully connected layer Concat_2 through the feature extraction module C2f_1 and the upsampling module CARAFE_2 of the Neck network. The feature F1 is then obtained through the feature extraction module C2f_2. The feature F1 is input into the first multi-scale spatial attention dilation convolution fusion DAS-Block_1. Feature F1 then passes through LAR-Conv_3, the fully connected layer Concat_3, and the feature extraction module C2f_3 to obtain feature F2, which is then input into the second multi-scale spatial attention dilated convolution fusion DAS-Block_2; Feature F2 is fused with feature map P5 in the fully connected layer Concat_4 through the downsampling module SCDown_3, and then the feature F3 is obtained through the feature extraction module C2fCIB. The feature F3 is input into the third multi-scale spatial attention dilation convolution fusion DAS-Block_3; The outputs of the three multi-scale spatial attention expansion convolution fusion DAS-Blocks are all input into the Head network. The Head network is correspondingly provided with three target detection modules v10Detect. The results of the three target detection modules v10Detect are convolutionally fused to obtain the insulator detection result. The data processing process of the separable convolution star module StarBlock-SC is as follows: First, a separable convolution operation is performed on the feature map data x0 input into the separable convolution star module StarBlock-SC to extract features; Then perform two ConvBN operations on the result of the separable convolution operation to obtain x1 and x2; Next, use the activation function ReLU6 to process x1, and then perform element-by-element multiplication operation ReLU6(x1)*x2 with x2; Then, a convolutional layer is used to restore the x1 and x2 dimensions to the dimensions of the input features, and a secondary separable convolution operation is performed to extract secondary features; Finally, the input feature map data x0 is residually connected with the result after the secondary separable convolution operation to obtain the output of the separable convolution star module StarBlock-SC.

2. The method according to claim 1, wherein: The operation process of the multi-scale spatial attention dilated convolution fusion module DAS-Block is to sequentially perform dilated convolution operations, splicing fusion, execution operations of the spatial attention mechanism module, and residual connections, specifically: The input data of the multi-scale spatial attention dilated convolution fusion DAS-Block is defined as X. First, three dilated convolution operations are performed on the input data X. The dilated convolution operation is expressed as: Y d =X*K d +b Where Y d is the output of the dilated convolution, K d is the dilated convolution kernel, b is the bias term, and * represents the convolution operation; The outputs of different dilated convolution paths are then fused using multi-scale blocks, expressed as: AND ms =Concat(And d1 ,AND d2 ,AND d3 ) Where Y ms represents the fusion result, Y d1 ,Y d2 ,Y d3 These are the results of the first, second, and third dilated convolution operations, respectively; Use 1x1 convolution to fusion result Y m Compression is performed, expressed as: Y ms ′=W 1×1 ·Y ms Where Y ms ′ represents the result after channel compression, W 1×1 is the weight of 1x1 convolution; Then, the spatial attention mechanism module is used to generate the attention map and the attention map is used to weight the fusion result Y ms , that is, the spatial attention of the features is calculated through convolution operation and used; A=Sigmoid(W s ·Y ms ′+b s ) Z=A·Y ms Where W s is the weight of the spatial attention convolution, b s is the bias, A is the attention map, and Z is the weighted result; Finally, the weighted result is combined with the channel compressed result Y ms ' performs residual connection, which is expressed as: AND output =Z+Y ms ′ Among them, the residual connection is connected to the output of the multi-scale spatial attention expansion convolution fusion module DAS-Block.

3. The method according to claim 1, wherein: The novel YOLOv10 network uses the CARAFE module to replace the upsampling operation of the original bilinear interpolation in the Neck network; The CARAFE module consists of a kernel prediction module and a content-aware reassembly module. The kernel prediction module is used to generate a reassembly kernel according to the input feature map data, and the content-aware reassembly module is used to reassemble the local area of ​​the input feature map into a new output feature map based on the reassembly kernel. Define the feature map input to the CARAFE module, the kernel prediction module is: W l′ ψ(N(X l ,k encoder )) Where, X l is the target position in the input feature map X, the target position l′ is the position in the output feature map obtained by the CARAFE operation, and W l′ is the recombinant kernel predicted at the target location l′, and ψ represents the kernel prediction module, which is derived from the input feature map X l The sub-region centered at the target position l′ is extracted from the kernel k encoder Processing, N is a nonlinear function, using ReLU6 or other activation functions; The content-aware recombinant module is represented as: X′ l′ =Φ(N(X l ,k up ),W l′ ) Where, X l is the new feature map generated at the target location l′, Φ represents the content-aware reconstruction module that uses the upsampling kernel k up and predicted recombinant nuclear W l′ Reorganize the input features.

4. The method according to claim 1, wherein: The new YOLOv10 network uses the LAR-Conv structure to replace the Backbone network and part of the standard convolution Conv in the Neck network; The LAR-Conv structure sequentially performs standard convolution Conv, local attention module, residual connection, and regularization, specifically: First, the standard convolution Conv is used to perform a convolution operation on the feature map input into the LAR-Conv structure; Then, the attention weight of the local attention module is used to perform feature weighting on the feature map after the convolution operation to obtain a weighted feature map; Next, the feature map after the convolution operation and the weighted feature map are connected using residuals; Finally, the feature map after residual connection is regularized.

5. The method according to claim 1, wherein: The image data acquired in step S1 includes: a windshield image, a person image, a seat belt image, or a windshield image containing both a person and a seat belt, and the wearing status of the windshield, the person, and the seat belt are marked respectively; Use the detection model based on the new YOLOv10 network built in step S2 to train the windshield detection model, the personnel detection model, and the seat belt detection model respectively; Then, the trained windshield detection model, person detection model, and seat belt detection model are used to perform windshield detection, person detection, and seat belt detection on the image to be detected in sequence; The windshield is segmented based on the windshield detection result, and then personnel detection is performed on the segmented windshield image. The personnel are segmented based on the personnel detection result, and then seat belt detection is performed on the segmented personnel image.

6. A detection system based on the method according to any one of claims 1 to 5, characterized in that: include: An image acquisition module, configured to acquire and annotate image data, wherein the image data at least includes a seat belt image and annotates the seat belt position / wearing status; A detection model construction module is used to build a detection model based on the new YOLOv10 network, wherein the new YOLOv10 network is sequentially provided with an input terminal, a backbone network, a neck network, and a head network; Among them, the separable convolutional star module StarBlock-SC is used to replace the original feature extraction module C2f of the Backbone network; and before the three sets of feature maps output by the Neck network, a multi-scale spatial attention expansion convolution fusion module DAS-Block is added, and then the feature maps are output to the Head network; The model training module is used to train the detection model constructed in step S2 using the labeled image data to obtain a seat belt detection model for seat belt wearing detection.

7. A computer terminal, characterized in that: At least: one or more processors; a memory storing one or more computer programs; The processor calls the computer program to implement: The steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: A computer program is stored, which is called by a processor to implement: The steps of the method according to any one of claims 1 to 5.

Images