Safety belt detection method and system based on novel YOLOv10

Abstract

The invention provides a novel YOLOv10-based safety belt detection method and system, and the method comprises the steps: obtaining image data, and marking the image data, namely marking the position / wearing state of a safety belt; constructing a detection model based on a novel YOLOv10 network, namely replacing partial standard convolution Conv of the Backbone network with local attention regularization convolution; using a separable convolution star module to replace an original feature extraction module of the Backbone network; a CARAFE module is used for replacing up-sampling operation in a Neck network, multi-scale space attention dilation convolution fusion modules are additionally arranged before three sets of feature maps output by the Neck network, and then the feature maps are output to a Head network; and training a detection model by using the labeled image data to obtain a safety belt detection model. When the novel YOLOv10 network is used for safety belt detection, the detection efficiency and performance can be remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a seat belt detection method and system based on the novel YOLOv10. Background Art

[0002] As an important branch of artificial intelligence, computer vision has made remarkable progress in recent years. With the widespread application of deep learning technology, the performance of computer vision in image recognition, target detection, semantic segmentation, etc. has been greatly improved. Especially in the field of target detection, through the application of convolutional neural network (CNN), the model can automatically extract image features and accurately locate targets, greatly improving the accuracy and efficiency of detection. In addition, some advanced target detection frameworks such as Faster R-CNN, YOLO and SSD have also been proposed one after another. These frameworks have achieved faster detection speed and higher accuracy by improving network structure and optimizing algorithms. The development of these technologies has provided strong support for the upgrading of intelligent traffic management, autonomous driving and other fields.

[0003] As the number of cars continues to increase, road traffic becomes more intensive, and the incidence of traffic accidents also rises. In this context, wearing seat belts is particularly important. Seat belts are the first line of defense to protect passengers from injuries in the event of a vehicle collision. They can firmly secure passengers in their seats at critical moments, preventing them from being thrown out of the vehicle due to a violent collision or a secondary collision inside the vehicle, thereby greatly reducing the risk of injury or even death.

[0004] The existing seat belt detection methods can be roughly divided into two types. One is to rely on the vehicle's built-in sensors and algorithms to monitor the wearing status of the seat belts, but this method is sometimes limited by various methods, resulting in false detection or missed detection. In addition, these detection methods are too simple in design, which makes passengers pay insufficient attention to wearing seat belts. Therefore, seat belt detection technology needs to be more accurate and intelligent. The development of target detection in the field of computer vision has promoted the development of another seat belt detection method. By detecting whether the driver and passengers are wearing seat belts in image data, the safety of the driver's driving behavior can be effectively improved. However, the models used in existing studies still have many shortcomings when applied to seat belt detection, and the efficiency and performance need to be further improved and enhanced. Summary of the invention

[0005] The purpose of the present invention is to solve the technical problem that the existing models of target detection in the visual field are applied to seat belt detection with low efficiency and performance. The technical solution of the present invention proposes a seat belt detection method and system based on the new YOLOv10 to improve the efficiency and performance of the model. The present invention optimizes the network architecture based on the YOLOv10 network. On the one hand, the separable convolution star module StarBlock-SC is used to replace the original feature extraction module C2f of the Backbone network; on the other hand, before the three groups of feature maps output by the Neck network, multi-scale spatial attention expansion convolution fusion modules DAS-Block are respectively added, and then the feature maps are output to the Head network. Among them, the introduction of StarBlock-SC and DAS-block, the former can improve the detection efficiency, and the latter can significantly improve the target detection performance.

[0006] To this end, the following technical solutions are provided:

[0007] The present invention provides a safety belt detection method based on the novel YOLOv10, comprising the following steps:

[0008] S1: Acquire image data and mark it, wherein the image data at least includes a seat belt image and marks the seat belt position / wearing status;

[0009] S2: construct a detection model based on a new YOLOv10 network, wherein the new YOLOv10 network is provided with an input terminal, a backbone network, a neck network and a head network in sequence;

[0010] Among them, the separable convolutional star module StarBlock-SC is used to replace the original feature extraction module C2f of the Backbone network; and before the three sets of feature maps output by the Neck network, a multi-scale spatial attention expansion convolution fusion module DAS-Block is added respectively, and then the feature maps are output to the Head network;

[0011] S3: Use the labeled image data to train the detection model constructed in step S2 to obtain a seat belt detection model for seat belt wearing detection.

[0012] Optionally, the data processing process of the separable convolution star module StarBlock-SC is:

[0013] First, a separable convolution operation is performed on the feature map data x0 input into the separable convolution star module StarBlock-SC to extract features;

[0014] Then perform two ConvBN operations on the result after the separable convolution operation to obtain x1 and x2;

[0015] Next, use the activation function ReLU6 to process x1, and then perform element-by-element multiplication with x2 ReLU6(x1)*x2;

[0016] Then, a convolutional layer is used to restore the x1 and x2 dimensions to the dimensions of the input features, and a secondary separable convolution operation is performed to extract secondary features;

[0017] Finally, the input feature map data x0 is residually connected with the result after the secondary separable convolution operation to obtain the output of the separable convolution star module StarBlock-SC.

[0018] Among them, the element-by-element multiplication operation ReLU6(x1)*x2 is as follows:

[0019] Let ReLU6(x1) be y1, x2 be y2, and element-by-element multiplication can be written as:

[0020]

[0021] In the formula, i and j are used to index the channel; are the element values ​​corresponding to index i and j in y1 and y2 respectively. α describes the mutual influence or weight relationship between channels. It is the coefficient of different channel combinations of y1 and y2, and is the product or sum of weights between different channels.

[0022]

[0023] Where ω1 and ω2 are the weight vectors of y1 and y2 respectively. are the element values ​​corresponding to index i and j in ω1 and ω2 respectively, T is the transposition sign, d is the number of input channels, that is, the depth of the input data. By setting different d, different features can be passed to the convolutional layer for processing and different feature information can be extracted.

[0024] Optionally, the operation process of the multi-scale spatial attention dilated convolution fusion module DAS-Block is to sequentially perform dilated convolution operations, splicing fusion, execution operations of the spatial attention mechanism module, and residual connections, specifically:

[0025] The input data of the multi-scale spatial attention dilated convolution fusion DAS-Block is defined as X. First, three dilated convolution operations are performed on the input data X. The dilated convolution operation is expressed as:

[0026] Y d =X*K d +b

[0027] Where Y d is the output of the dilated convolution, K dis the dilated convolution kernel, b is the bias term, and * represents the convolution operation;

[0028] The outputs of different dilated convolution paths are then fused using multi-scale blocks, expressed as:

[0029] Y ms =Concat(Y d1 ,Y d2 ,Y d3 )

[0030] Where Y ms represents the fusion result, Y d1 ,Y d2 ,Y d3 These are the results of the first, second, and third dilated convolution operations, respectively;

[0031] Use 1x1 convolution to fusion result Y m Compression is performed, expressed as:

[0032] Y ms ′=W 1×1 ·Y ms

[0033] Where Y ms ′ represents the result after channel compression, W 1×1 is the weight of the 1x1 convolution;

[0034] Next, the spatial attention mechanism module is used to generate an attention map and the attention map is used to weight the fusion result Y ms , that is, the spatial attention of the features is calculated through convolution operations and used;

[0035] A=Sigmoid(W s ·Y ms ′+b s )

[0036] Z=A·Y ms

[0037] Where W s is the weight of the spatial attention convolution, b s is the bias, A is the attention map, and Z is the weighted result;

[0038] Finally, the weighted result is combined with the channel compressed result Y ms ′ performs residual connection, which is expressed as:

[0039] Y output =Z+Y ms '

[0040] Among them, the residual connection is connected to the output of the multi-scale spatial attention expansion convolution fusion module DAS-Block.

[0041] Optionally, the novel YOLOv10 network uses a CARAFE module to replace the original bilinear interpolation upsampling operation in the Neck network;

[0042] The CARAFE module is composed of a kernel prediction module and a content-aware reorganization module, wherein the kernel prediction module is used to generate a reorganization kernel according to the input feature map data, and the content-aware reorganization module is used to reorganize the local area of ​​the input feature map into a new output feature map based on the reorganization kernel;

[0043] Define the characteristic map X of the input to the CARAFE module l , the core prediction module is:

[0044] W l′ =ψ(N(X l ,k encoder ))

[0045] Where, X l is the target position in the input feature map X, the target position l′ is the position in the output feature map obtained after the CARAFE operation, and W l′ is the recombinant kernel predicted at the target location l′, and ψ represents the kernel prediction module, which is derived from the input feature map X l The sub-region centered at the target position l′ is extracted from the encoder kernel k encoder Processing, N is a nonlinear function, using ReLU6 or other activation functions;

[0046] The content-aware reorganization module is represented as:

[0047] X′ l′ =Φ(N(X l ,k up ),W l′ )

[0048] Where, X l is the new feature map generated at the target position l′, Φ represents the content-aware reconstruction module, that is, using the upsampling kernel k up and predicted recombinant nuclear W l′ Reorganize the input features.

[0049] Optionally, the novel YOLOv10 network uses a LAR-Conv structure to replace part of the standard convolution Conv in the Backbone network and the Neck network;

[0050] The LAR-Conv structure sequentially performs standard convolution Conv, local attention module, residual connection, and regularization, specifically:

[0051] First, the standard convolution Conv is used to perform a convolution operation on the feature map input into the LAR-Conv structure;

[0052] Then, the attention weight of the local attention module is used to perform feature weighting on the feature map after the convolution operation to obtain a weighted feature map;

[0053] Next, the feature map and weighted feature map after the convolution operation are connected using residuals;

[0054] Finally, the feature map after residual connection is regularized.

[0055] Optionally, the Backbone network is provided with four separable convolution star modules StarBlock-SC;

[0056] The Backbone network includes a standard convolution layer Conv, a LAR-Conv_1 structure, a separable convolution star module StarBlock-SC_1, a LAR-Conv_2 structure, a separable convolution star module StarBlock-SC_2, a downsampling module SCDown_1, a separable convolution star module StarBlock-SC_3, a downsampling module SCDown_2, a separable convolution star module StarBlock-SC_4, an attention module SPPF and an attention module PSA connected in sequence;

[0057] Among them, the separable convolution star module StarBlock-SC_2, the separable convolution star module StarBlock-SC_3, and the attention module PSA correspond to the output feature map P3, feature map P4, and feature map P5 respectively, and input the Neck network. The processing process of the feature map P3, feature map P4, and feature map P5 in the Neck network is:

[0058] The feature map P5 is fused with the feature map P4 in the fully connected layer Concat_1 through the upsampling module CARAFE_1 of the Neck network; after fusion, it is fused with the feature map P3 in the fully connected layer Concat_2 through the feature extraction module C2f_1 and the upsampling module CARAFE_2 of the Neck network; then it is fused with the feature map F1 through the feature extraction module C2f_2, and the feature F1 is input into the first multi-scale spatial attention dilation convolution fusion DAS-Block_1;

[0059] Feature F1 then passes through the local attention regularization convolution layer LAR-Conv_3, the fully connected layer Concat_3, and the feature extraction module C2f_3 to obtain feature F2, and the feature F2 is input into the second multi-scale spatial attention expansion convolution fusion DAS-Block_2;

[0060] Feature F2 is fused with feature map P5 in the fully connected layer Concat_4 through the downsampling module SCDown_3, and feature F3 is obtained through the feature extraction module C2fCIB. Feature F3 is input into the third multi-scale spatial attention dilation convolution fusion DAS-Block_3;

[0061] The outputs of the three multi-scale spatial attention dilation convolution fusion DAS-Blocks are all input into the Head network. The Head network is correspondingly provided with three target detection modules v10Detect. The results of the three target detection modules v10Detect are convolutionally fused to obtain the insulator detection results.

[0062] Optionally, the image data acquired in step S1 includes: a windshield image, a person image, a seat belt image, or a windshield image including both a person and a seat belt, and the wearing status of the windshield, the person, and the seat belt are respectively marked;

[0063] Use the detection model based on the new YOLOv10 network built in step S2 to train the windshield detection model, the personnel detection model, and the seat belt detection model respectively;

[0064] Then, the trained windshield detection model, person detection model, and seat belt detection model are used to perform windshield detection, person detection, and seat belt detection on the image to be detected in sequence;

[0065] The windshield is segmented based on the windshield detection result, and then personnel detection is performed on the segmented windshield image. The personnel are segmented based on the personnel detection result, and then seat belt detection is performed on the segmented personnel image.

[0066] The present invention also provides a detection system based on the above detection method, comprising:

[0067] An image acquisition module, used to acquire image data and mark it, wherein the image data at least includes a seat belt image and marks the seat belt position / wearing status;

[0068] A detection model building module, used to build a detection model based on a new YOLOv10 network, wherein the new YOLOv10 network is provided with an Input input terminal, a Backbone network, a Neck network and a Head network in sequence;

[0069] Among them, the separable convolutional star module StarBlock-SC is used to replace the original feature extraction module C2f of the Backbone network; and before the three sets of feature maps output by the Neck network, a multi-scale spatial attention expansion convolution fusion module DAS-Block is added respectively, and then the feature maps are output to the Head network;

[0070] The model training module is used to train the detection model constructed in step S2 using the labeled image data to obtain a seat belt detection model for seat belt wearing detection.

[0071] The present invention also provides a computer terminal, comprising at least:

[0072] one or more processors;

[0073] a memory storing one or more computer programs;

[0074] The processor calls the computer program to implement:

[0075] Steps for a novel YOLOv10 based seatbelt detection method.

[0076] The present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program is called by a processor to implement:

[0077] Steps for a novel YOLOv10 based seatbelt detection method.

[0078] Beneficial Effects

[0079] Compared with the existing method, the advantages of the present invention are:

[0080] 1. The technical solution of the present invention is based on the YOLOv10 network, and the network architecture is optimized to improve the efficiency and performance of seat belt detection. Among them, the present invention uses the separable convolution star module StarBlock-SC to replace part of the original C2f module, which enhances the expression ability and performance of the target features, while suppressing the interference of background noise. Moreover, without significantly reducing the detection performance, the number of parameters and the amount of calculation of the model are reduced, thereby improving the running speed of the model, and being more adaptable to the illumination and occlusion factors in the process of seat belt detection. The factors put forward higher requirements on the accuracy and generalization ability of the model; furthermore, the present invention also uses multi-scale spatial attention expansion convolution fusion DAS-Block between the three feature extraction modules after the Neck network and the detection head, which significantly improves the model's feature extraction ability, context understanding ability and robustness in handling complex scenes.

[0081] 2. A further optimization scheme of the present invention uses local attention regularized convolution LAR-Conv to replace part of the standard convolution Conv in order to better detect and focus on the features related to the target task. Especially when dealing with complex backgrounds and occlusions, it can more effectively distinguish the difference between the target and the background, thereby improving the accuracy and reliability of seat belt detection, and also improving the generalization ability of the model on unseen data; and in the upsampling operation, CARAFE is used to replace the original bilinear interpolation, which is flexible, content-aware and has higher expression ability; finally, the improved YOLOv10 model has higher performance and efficiency, as well as better generalization ability.

[0082] In summary, StarBlock-SC used in the technical solution of the present invention is efficient and concise, DAS-block can improve the performance of target detection, LA-Conv pays more attention to local features and improves the detection capability, CARAFE is lightweight and effective, and the present invention integrates it into the YOLOv10 network, cleverly sets its position and structure, and significantly improves the seat belt detection performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 This is an architecture diagram of the seat belt recognition system based on improved YOLOv10 of the present invention;

[0084] Figure 2 A structural schematic diagram of a seat belt identification system in an example of the present invention;

[0085] Figure 3 It is a diagram showing the improved structure of the YOLOv10 algorithm of the present invention;

[0086] Figure 4 This is a structural diagram of the separable convolution star module StarBlock-SC in an example of the present invention.

[0087] Figure 5 This is the LAR-Conv structure diagram in the example of the present invention.

[0088] Figure 6 This is a structural diagram of CARAFE in an example of the present invention.

[0089] Figure 7 This is a structural diagram of the DAS-Block in an example of the present invention. DETAILED DESCRIPTION

[0090] The present invention will be further described below with reference to the embodiments.

[0091] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. In addition, the technical features involved in the various embodiments of the present application described below can be combined with each other as long as they do not conflict with each other. The present application is further described in detail below in conjunction with the accompanying drawings for its specific implementation methods, as follows:

[0092] An embodiment of the present invention provides a seat belt detection method based on the novel YOLOv10, comprising the following steps:

[0093] S1: Acquire image data and mark it, wherein the image data at least includes a seat belt image and marks the seat belt position / wearing status.

[0094] In this embodiment, it is preferred to construct a windshield detection model, a personnel detection model, and a seat belt detection model. Then, in actual image detection, the windshield is first segmented based on the windshield detection result, and then personnel detection is performed on the segmented windshield image. The personnel are segmented based on the personnel detection result, and then the seat belt detection is performed on the segmented personnel image. Therefore, the image data preferably obtained in this embodiment includes: annotated windshield images, personnel images, seat belt images, or windshield images containing personnel and seat belts at the same time, and the wearing status of the windshield, personnel, and seat belts are respectively marked. In other feasible embodiments, the method of obtaining seat belt images and marking seat belts also falls within the protection scope of the present invention.

[0095] In this embodiment, the training data and the verification data are annotated by an annotation tool to obtain a training data set and a verification data set, respectively, with a quantity ratio of 9:1. The training data and the verification data are obtained from Kaggle. Specifically, the image is resized to an image of 512x512 pixels, and the training data and the verification data are annotated by Make sense, including three types of data sets, one type of data is annotated as windshield, one type of data is annotated as driver, co-pilot and rear passenger, and the last type of data is annotated as wearing seat belts and not wearing seat belts.

[0096] S2: Construct a detection model based on the new YOLOv10 network. The new YOLOv10 network is provided with an Input input terminal, a Backbone network, a Neck network and a Head network in sequence. The technical solution of the present invention uses a separable convolution star module StarBlock-SC to replace the original feature extraction module C2f of the Backbone network; and before the three sets of feature maps output by the Neck network, a multi-scale spatial attention expansion convolution fusion module DAS-Block is respectively added, and then the feature map is output to the Head network. In addition, this embodiment also preferably uses the CARAFE module in the new YOLOv10 network to replace the original bilinear interpolation upsampling operation in the Neck network, and uses the LAR-Conv structure in the new YOLOv10 network to replace some standard convolution Conv in the Backbone network and the Neck network.

[0097] Furthermore, the backbone network of the novel YOLOv10 network provided in this embodiment is provided with four separable convolutional star modules StarBlock-SC;

[0098] Among them, the Backbone network includes the standard convolution layer Conv, the local attention regularized convolution LAR-Conv_1 structure, the separable convolution star module StarBlock-SC_1, the local attention regularized convolution LAR-Conv_2 structure, the separable convolution star module StarBlock-SC_2, the downsampling module SCDown_1, the separable convolution star module StarBlock-SC_3, the downsampling module SCDown_2, the separable convolution star module StarBlock-SC_4, the attention module SPPF and the attention module PSA, which are connected in sequence; the separable convolution star module StarBlock-SC_2, the separable convolution star module StarBlock-SC_3 and the attention module PSA correspond to the output feature map P3, the feature map P4 and the feature map P5 respectively, and are input into the Neck network.

[0099] About Neck Network:

[0100] The processing process of feature map P3, feature map P4, and feature map P5 in the Neck network is:

[0101] The feature map P5 is fused with the feature map P4 in the fully connected layer Concat_1 through the upsampling module CARAFE_1 of the Neck network. After fusion, it is fused with the feature map P3 in the fully connected layer Concat_2 through the feature extraction module C2f_1 and the upsampling module CARAFE_2 of the Neck network. Then, it passes through the feature extraction module C2f_2 to obtain the feature F1, which is input into the first multi-scale spatial attention dilation convolution fusion DAS-Block_1.

[0102] Feature F1 then passes through the local attention regularization convolution layer LAR-Conv_3, the fully connected layer Concat_3, and the feature extraction module C2f_3 to obtain feature F2, and feature F2 is input into the second multi-scale spatial attention expansion convolution fusion DAS-Block_2;

[0103] Feature F2 is fused with feature map P5 in the fully connected layer Concat_4 through the downsampling module SCDown_3, and feature F3 is obtained through the feature extraction module C2fCIB. Feature F3 is input into the third multi-scale spatial attention dilation convolution fusion DAS-Block_3;

[0104] The outputs of the three multi-scale spatial attention dilation convolution fusion DAS-Blocks are all input into the Head network. The Head network correspondingly sets up three target detection modules v10Detect. The results of the three target detection modules v10Detect are convolutionally fused to obtain the insulator detection results.

[0105] The network optimization part of the present invention will be described in detail below:

[0106] Regarding the separable convolutional star module StarBlock-SC, its data processing process is as follows:

[0107] First, perform a separable convolution operation on the feature map data x0 of the input separable convolution star module StarBlock-SC to extract features;

[0108] Then perform two ConvBN operations on the result after the separable convolution operation to obtain x1 and x2;

[0109] Next, use the activation function ReLU6 to process x1, and then perform element-by-element multiplication with x2 ReLU6(x1)*x2;

[0110] Then, a convolutional layer is used to restore the x1 and x2 dimensions to the dimensions of the input features, and a secondary separable convolution operation is performed to extract secondary features;

[0111] Finally, the input feature map data x0 is residually connected with the result after the secondary separable convolution operation to obtain the output of the separable convolution star module StarBlock-SC.

[0112] Among them, the element-by-element multiplication operation ReLU6(x1)*x2 is as follows:

[0113] Let ReLU6(x1) be y1, x2 be y2, and element-by-element multiplication can be written as:

[0114]

[0115] In the formula, i and j are used to index the channel; are the element values ​​corresponding to index i and j in y1 and y2 respectively. α describes the mutual influence or weight relationship between channels. It is the coefficient of different channel combinations of y1 and y2, and is the product or sum of weights between different channels.

[0116]

[0117] Where ω1 and ω2 are the weight vectors of y1 and y2 respectively. are the element values ​​corresponding to index i and j in ω1 and ω2 respectively, T is the transposition sign, d is the number of input channels, that is, the depth of the input data. By setting different d, different features can be passed to the convolutional layer for processing and different feature information can be extracted.

[0118] Regarding the multi-scale spatial attention dilated convolution fusion module DAS-Block, its operation process is to sequentially perform dilated convolution operations (generating dilated convolutions with different feature maps), splicing fusion (splicing and fusion of multi-scale blocks of output feature maps of different dilated convolution paths), execution operations of the spatial attention mechanism module (highlighting important areas), and residual connections. Specifically:

[0119] The input data of the multi-scale spatial attention dilated convolution fusion DAS-Block is defined as X. First, three dilated convolution operations are performed on the input data X. The dilated convolution operation is expressed as:

[0120] Y d =X*K d +b

[0121] Where Y d is the output of the dilated convolution, K d is the dilated convolution kernel, b is the bias term, and * represents the convolution operation;

[0122] The outputs of different dilated convolution paths are then fused using multi-scale blocks, expressed as:

[0123] Yms =Concat(Y d1 ,Y d2 ,Y d3 )

[0124] Where Y ms represents the fusion result, Y d1 ,Y d2 ,Y d3 These are the results of the first, second, and third dilated convolution operations, respectively;

[0125] Use 1x1 convolution to fusion result Y m Compression is performed, expressed as:

[0126] Y ms ′=W 1×1 ·Y ms

[0127] Where Y ms ′ represents the result after channel compression, W 1×1 is the weight of the 1x1 convolution;

[0128] Next, the spatial attention mechanism module is used to generate an attention map and the attention map is used to weight the fusion result Y ms , that is, the spatial attention of the features is calculated through convolution operations and used;

[0129] A=Sigmoid(W s ·Y ms ′+b s )

[0130] Z=A·Y ms

[0131] Where W s is the weight of the spatial attention convolution, b s is the bias, A is the attention map, and Z is the weighted result;

[0132] Finally, the weighted result is combined with the channel compressed result Y ms ′ performs residual connection, which is expressed as:

[0133] Y output =Z+Y ms '

[0134] Among them, the residual connection is connected to the output of the multi-scale spatial attention expansion convolution fusion module DAS-Block.

[0135] Regarding the CARAFE module, it consists of a kernel prediction module and a content-aware reorganization module. The kernel prediction module generates a reorganization kernel based on the content of the input feature through operations such as channel compression, content encoding, and kernel normalization; the content-aware reorganization module uses the reorganization kernel to reorganize the local area of ​​the input feature map into a new output feature map through operations such as zero padding, neighborhood extraction, and matrix multiplication. This process ensures that the output feature map not only has a higher resolution, but also effectively retains and strengthens the semantic information of the input features.

[0136] Define the characteristic map X of the input to the CARAFE module l , the core prediction module is:

[0137] W l′ =ψ(N(X l ,k encoder ))

[0138] Where W l′ is the recombinant kernel predicted at the target location l′, and ψ represents the kernel prediction module, which is derived from the input feature map X l The sub-region (neighborhood) centered at the target position l′ is extracted from the kernel k of the encoder. encoder Processing, N is a nonlinear function, using ReLU6 or other activation functions; the purpose of this formula is to generate a content-aware kernel for each target position for subsequent recombination.

[0139] The content-aware reorganization module is represented as:

[0140] X′ l′ =Φ(N(X l ,k up ),W l′ )

[0141] Where, X l is the new feature map generated at the target position l′, Φ represents the content-aware reconstruction module, that is, using the upsampling kernel k up and predicted recombinant nuclear W l′ Reorganize the input features. This formula describes how to reorganize the original feature map through the kernel to generate a new high-resolution feature map.

[0142] Regarding the LAR-Conv structure, it sequentially executes standard convolution Conv, local attention module, residual connection, and regularization. Among them, the purpose of the local attention module is to calculate the local attention weight of the input feature map to emphasize important local features, that is, to apply the attention weight to the output feature map to highlight important features while retaining the information of other features; the regularization layer is used to prevent the model from overfitting. Based on the above structure, LAR-Conv can enhance the model's response to important features, improve detection performance, and effectively reduce overfitting and improve the model's generalization ability.

[0143] The specific operation process of the LAR-Conv structure is:

[0144] 1. First, use the standard convolution Conv to perform convolution operation on the feature map of the input LAR-Conv structure. Among them, the input feature map (B, C, H, W) first passes through the convolution layer, and the output feature map (B, C′, H′, W′) is output. B is the batch size, C is the number of channels, H is the height of the feature map, W is the width of the feature map, C′ is the number of channels output by the convolution layer, and H′ and W′ are the height and width of the feature map after convolution processing.

[0145] 2. Use the attention weight of the local attention module to perform feature weighting on the feature map after the convolution operation to obtain a weighted feature map. This step can be divided into the following steps:

[0146] The first step is to use 1x1 convolution to generate query and key features for calculating attention weights, expressed as:

[0147] Q=W q ·X

[0148] K=W k ·X

[0149] Where W q and W k is the weight of the 1x1 convolution, and X is the feature map after the convolution operation.

[0150] The second step is to calculate the attention score between the query and the key, using the dot product operation to form an attention matrix. It can be expressed as:

[0151]

[0152] Where, d k is the dimension of the key, used to scale the attention score, T matrix transpose, Attention(Q,K) attention score.

[0153] The third step is to apply the softmax function to the attention matrix to obtain the attention weights, which represent the importance of each channel, expressed as:

[0154] α=softmax(Attention(Q,K))

[0155] Where α is the attention weight, the softmax function is the normalization operation, and Attention(Q,K) is the attention score.

[0156] The fourth step is feature weighting, which is to apply attention weights to the input feature map to highlight important features while retaining the information of other features, expressed as:

[0157] Z=α·X

[0158] Where X is the feature map after the convolution operation, and α is the attention weight.

[0159] Next, the feature map and weighted feature map after the convolution operation are connected using residuals, expressed as:

[0160] X′=Z+X

[0161] Where X′ is the feature map after residual connection, Z is the weighted feature map, and X is the feature map after convolution operation.

[0162] Finally, the feature map after residual connection is regularized, which is expressed as:

[0163] X' = ​​Dropout(X')

[0164] Where X′ is the feature map after residual connection, and X” is the feature map after regularization operation.

[0165] S3: Use the labeled image data to train the detection model constructed in step S2 to obtain a seat belt detection model for seat belt wearing detection.

[0166] The training parameters are the hyperparameters in the modified configuration file, and the model training is performed based on the new YOLOv10 network. The specific process is: load the hyperparameters set in the configuration file into the model; randomly select a group of images in the training data set as the current training samples; input each image in the training sample into the model in turn to update the weight parameters, where the weight parameter update is specifically: randomly initialize the weight parameters and then perform the forward propagation calculation of the convolutional neural network to obtain a set of intermediate parameters, and then use the intermediate parameters to perform back propagation to update the weight parameters. The new weight parameters will be the old weight parameters used to calculate the forward propagation before the iteration. The process of completing the forward and back propagation of all images in the training sample is recorded as one training of the model; repeat the above steps until the number of model trainings reaches the set number of iterations. After the entire training is completed, the candidate weight file is screened out. The candidate weight is evaluated and quantified according to the verification data set to obtain the target weight, and the target weight is used to update the candidate weight to complete the model construction.

[0167] This embodiment uses the detection model based on the new YOLOv10 network built in step S2 to train the windshield detection model, the personnel detection model, and the seat belt detection model respectively.

[0168] Step 4: Use the trained windshield detection model, person detection model, and seat belt detection model to perform windshield detection, person detection, and seat belt detection on the image to be detected in turn;

[0169] The windshield is segmented based on the windshield detection result, and then personnel detection is performed on the segmented windshield image. The personnel are segmented based on the personnel detection result, and then seat belt detection is performed on the segmented personnel image.

[0170] In summary, the present invention optimizes the original YOLOv10 network, and uses a new, simple and efficient separable convolution star module StarBlock-SC to replace the feature extraction module C2f, which not only reduces the number of model parameters and the amount of calculation, but also integrates the information between different channels and improves the feature expression ability. In order to better detect and focus on the features related to the target task, a local attention regularized convolution LAR-Conv is proposed to replace part of the standard convolution Conv. The special change is that when dealing with complex backgrounds and occlusions, it can more effectively distinguish the difference between the target and the background, thereby improving the accuracy and reliability of seat belt detection, and can help reduce overfitting, thereby improving the generalization ability of the model on unseen data. In order to improve the detection performance of seat belts, in the Neck network, the lightweight and fast CARAFE is used to replace the original bilinear interpolation upsampling operation, and the multi-scale spatial attention expansion convolution fusion DAS-Block is used after the last three feature extraction modules, which significantly improves the model's feature extraction ability, context understanding ability and robustness in dealing with complex scenes.

[0171] In some embodiments, the present invention also provides a detection system based on the above detection method, including: an image acquisition module, a detection model building module, and a model training module.

[0172] The image acquisition module is used to acquire and annotate image data, wherein the image data at least includes a seat belt image and annotates the seat belt position / wearing status;

[0173] The detection model construction module is used to construct a detection model based on a new YOLOv10 network, wherein the new YOLOv10 network is provided with an Input input terminal, a Backbone network, a Neck network and a Head network in sequence;

[0174] Among them, the separable convolutional star module StarBlock-SC is used to replace the original feature extraction module C2f of the Backbone network; and before the three sets of feature maps output by the Neck network, a multi-scale spatial attention expansion convolution fusion module DAS-Block is added respectively, and then the feature maps are output to the Head network;

[0175] The model training module is used to train the detection model constructed in step S2 using the labeled image data to obtain a seat belt detection model for seat belt wearing detection.

[0176] In some embodiments, the detection system further includes a detection module for performing seat belt wearing detection using the constructed seat belt detection model. If a windshield detection model, a person detection model, and a seat belt detection model are constructed simultaneously, the windshield detection model, the person detection model, and the seat belt detection model are used in sequence to perform windshield detection, person detection, and seat belt detection.

[0177] It should be understood that the specific implementation process of each module please refer to the above method content, the present invention will not be repeated here, and the division of the above functional modules is only for example illustration. In some embodiments, some functional modules can be merged, and some functional modules can be split. Each functional module can be implemented in software or hardware or a combination of software and hardware. Among them, the hardware and software equipment includes but is not limited to general-purpose computer equipment, programmable gate arrays, digital signal processors, microprocessors and their corresponding programming or burning software.

[0178] In some embodiments, the present invention further provides a computer terminal, comprising at least: one or more processors; and a memory storing one or more computer programs;

[0179] Among them, the processor calls the computer program to implement: a step of a seat belt detection method based on the new YOLOv10.

[0180] Specifically, the following are implemented:

[0181] S1: Acquire image data and mark it, wherein the image data at least includes a seat belt image and marks the seat belt position / wearing status.

[0182] S2: Build a detection model based on the new YOLOv10 network.

[0183] S3: Use the labeled image data to train the detection model constructed in step S2 to obtain a seat belt detection model for seat belt wearing detection.

[0184] For the specific implementation process of each step, please refer to the description of the above method.

[0185] It should be understood that in the embodiments of the present invention, the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the type of device.

[0186] In some embodiments, the present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program is called by a processor to implement: steps of a seat belt detection method based on the new YOLOv10.

[0187] Specifically, the following are implemented:

[0188] S1: Acquire image data and mark it, wherein the image data at least includes a seat belt image and marks the seat belt position / wearing status.

[0189] S2: Build a detection model based on the new YOLOv10 network.

[0190] S3: Use the labeled image data to train the detection model constructed in step S2 to obtain a seat belt detection model for seat belt wearing detection.

[0191] For the specific implementation process of each step, please refer to the description of the above method.

[0192] The readable storage medium is a computer-readable storage medium, which may be an internal storage unit of the software and hardware device described in any of the aforementioned embodiments, such as a hard disk or memory of a controller. The readable storage medium may also be an external storage device of the controller, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the controller. Furthermore, the readable storage medium may also include both an internal storage unit of the controller and an external storage device. The readable storage medium is used to store the computer program and other programs and data required by the controller. The readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0193] Based on this understanding, the technical solution of the present invention, in essence or in other words, the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned readable storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., various media that can store program codes.

[0194] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The present application is a device for realizing the function specified in one flow or multiple flows of the flow chart and / or one or multiple blocks of the block diagram with reference to the instructions executed by the processor according to the method, device (system) and computer program product of the embodiment of the present application. These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which realizes the function specified in one flow or multiple flows of the flow chart and / or one or multiple blocks of the block diagram. These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0195] At the same time, the contents not described in detail in this specification belong to the prior art known to those skilled in the art. It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or equipment.

[0196] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0197] It should be emphasized that the examples described in the present invention are illustrative rather than restrictive, and therefore the present invention is not limited to the examples described in the specific embodiments. Any other embodiments derived by those skilled in the art based on the technical solution of the present invention that do not depart from the purpose and scope of the present invention, whether modified or replaced, also fall within the scope of protection of the present invention.

Claims

1. A seat belt detection method based on the new YOLOv10, characterized by: The following steps are involved: S1: Acquire image data and mark it, wherein the image data at least includes a seat belt image and marks the seat belt position / wearing status; S2: construct a detection model based on a new YOLOv10 network, wherein the new YOLOv10 network is provided with an input terminal, a backbone network, a neck network and a head network in sequence; Among them, the separable convolutional star module StarBlock-SC is used to replace the original feature extraction module C2f of the Backbone network; and before the three sets of feature maps output by the Neck network, a multi-scale spatial attention expansion convolution fusion module DAS-Block is added respectively, and then the feature maps are output to the Head network; S3: Use the labeled image data to train the detection model constructed in step S2 to obtain a seat belt detection model for seat belt wearing detection.

2. The method according to claim 1, characterized in that: The data processing process of the separable convolution star module StarBlock-SC is as follows: First, a separable convolution operation is performed on the feature map data x0 input into the separable convolution star module StarBlock-SC to extract features; Then perform two ConvBN operations on the result after the separable convolution operation to obtain x1 and x2; Next, use the activation function ReLU6 to process x1, and then perform element-by-element multiplication with x2 ReLU6(x1)*x2; Then, a convolutional layer is used to restore the x1 and x2 dimensions to the dimensions of the input features, and a secondary separable convolution operation is performed to extract secondary features; Finally, the input feature map data x0 is residually connected with the result after the secondary separable convolution operation to obtain the output of the separable convolution star module StarBlock-SC.

3. The method according to claim 1, characterized in that: The operation process of the multi-scale spatial attention dilated convolution fusion module DAS-Block is to sequentially perform dilated convolution operations, splicing fusion, execution operations of the spatial attention mechanism module, and residual connections, specifically: The input data of the multi-scale spatial attention dilated convolution fusion DAS-Block is defined as X. First, three dilated convolution operations are performed on the input data X. The dilated convolution operation is expressed as: Y d =X*K d +b Where Y d is the output of the dilated convolution, K d is the dilated convolution kernel, b is the bias term, and * represents the convolution operation; The outputs of different dilated convolution paths are then fused using multi-scale blocks, expressed as: AND ms =Concat(And d1 ,AND d2 ,AND d3 ) Where Y ms represents the fusion result, Y d1 ,Y d2 ,Y d3 These are the results of the first, second, and third dilated convolution operations, respectively; Use 1x1 convolution to fusion result Y m Compression is performed, expressed as: Y ms ′=W 1×1 ·Y ms Where Y ms ′ represents the result after channel compression, W 1×1 is the weight of the 1x1 convolution; Next, the spatial attention mechanism module is used to generate an attention map and the attention map is used to weight the fusion result Y ms , that is, the spatial attention of the features is calculated through convolution operations and used; A=Sigmoid(W s ·Y ms ′+b s ) Z=A·Y ms Where W s is the weight of the spatial attention convolution, b s is the bias, A is the attention map, and Z is the weighted result; Finally, the weighted result is combined with the channel compressed result Y ms ′ performs residual connection, expressed as: AND output =Z+Y ms ′ Among them, the residual connection is connected to the output of the multi-scale spatial attention expansion convolution fusion module DAS-Block.

4. The method according to claim 1, characterized in that: The novel YOLOv10 network uses a CARAFE module to replace the original bilinear interpolation upsampling operation in the Neck network; The CARAFE module is composed of a kernel prediction module and a content-aware reorganization module, wherein the kernel prediction module is used to generate a reorganization kernel according to the input feature map data, and the content-aware reorganization module is used to reorganize the local area of ​​the input feature map into a new output feature map based on the reorganization kernel; Define the feature map input to the CARAFE module, the kernel prediction module is: W l′ ψ(N(X l ,k encoder )) Where, X l is the target position in the input feature map X, the target position l′ is the position in the output feature map obtained after the CARAFE operation, and W l′ is the recombinant kernel predicted at the target location l′, and ψ represents the kernel prediction module, which is derived from the input feature map X l The sub-region centered at the target position l′ is extracted from the encoder kernel k encoder Processing, N is a nonlinear function, using ReLU6 or other activation functions; The content-aware reorganization module is represented as: X′ l′ =Φ(N(X l ,k up ),W l′ ) Where, X l is the new feature map generated at the target position l′, Φ represents the content-aware reconstruction module, that is, using the upsampling kernel k up and predicted recombinant nuclear W l′ Reorganize the input features.

5. The method according to claim 1, characterized in that: The new YOLOv10 network uses a LAR-Conv structure to replace part of the standard convolution Conv in the Backbone network and the Neck network; The LAR-Conv structure sequentially performs standard convolution Conv, local attention module, residual connection, and regularization, specifically: First, the standard convolution Conv is used to perform a convolution operation on the feature map input into the LAR-Conv structure; Then, the attention weight of the local attention module is used to perform feature weighting on the feature map after the convolution operation to obtain a weighted feature map; Next, the feature map and weighted feature map after the convolution operation are connected using residuals; Finally, the feature map after residual connection is regularized.

6. The method according to claim 1, characterized in that: The Backbone network is provided with four separable convolutional star modules StarBlock-SC; The Backbone network includes a standard convolution layer Conv, a LAR-Conv_1 structure, a separable convolution star module StarBlock-SC_1, a LAR-Conv_2 structure, a separable convolution star module StarBlock-SC_2, a downsampling module SCDown_1, a separable convolution star module StarBlock-SC_3, a downsampling module SCDown_2, a separable convolution star module StarBlock-SC_4, an attention module SPPF and an attention module PSA connected in sequence; Among them, the separable convolution star module StarBlock-SC_2, the separable convolution star module StarBlock-SC_3, and the attention module PSA correspond to the output feature map P3, feature map P4, and feature map P5 respectively, and input the Neck network. The processing process of the feature map P3, feature map P4, and feature map P5 in the Neck network is: The feature map P5 is fused with the feature map P4 in the fully connected layer Concat_1 through the upsampling module CARAFE_1 of the Neck network; after fusion, it is fused with the feature map P3 in the fully connected layer Concat_2 through the feature extraction module C2f_1 and the upsampling module CARAFE_2 of the Neck network; then it is fused with the feature map F1 through the feature extraction module C2f_2, and the feature F1 is input into the first multi-scale spatial attention dilation convolution fusion DAS-Block_1; Feature F1 then passes through LAR-Conv_3, fully connected layer Concat_3, and feature extraction module C2f_3 to obtain feature F2, and the feature F2 is input into the second multi-scale spatial attention dilated convolution fusion DAS-Block_2; Feature F2 is fused with feature map P5 in the fully connected layer Concat_4 through the downsampling module SCDown_3, and feature F3 is obtained through the feature extraction module C2fCIB. Feature F3 is input into the third multi-scale spatial attention dilation convolution fusion DAS-Block_3; The outputs of the three multi-scale spatial attention dilation convolution fusion DAS-Blocks are all input into the Head network. The Head network is correspondingly provided with three target detection modules v10Detect. The results of the three target detection modules v10Detect are convolutionally fused to obtain the insulator detection results.

7. The method according to claim 1, characterized in that: The image data acquired in step S1 includes: a windshield image, a person image, a seat belt image, or a windshield image containing both a person and a seat belt, and the wearing status of the windshield, the person, and the seat belt are marked respectively; Use the detection model based on the new YOLOv10 network built in step S2 to train the windshield detection model, the personnel detection model, and the seat belt detection model respectively; Then, the trained windshield detection model, person detection model, and seat belt detection model are used to perform windshield detection, person detection, and seat belt detection on the image to be detected in sequence; The windshield is segmented based on the windshield detection result, and then personnel detection is performed on the segmented windshield image. The personnel are segmented based on the personnel detection result, and then seat belt detection is performed on the segmented personnel image.

8. A detection system based on the method according to any one of claims 1 to 7, characterized in that: include: An image acquisition module, used to acquire image data and mark it, wherein the image data at least includes a seat belt image and marks the seat belt position / wearing status; A detection model building module, used to build a detection model based on a new YOLOv10 network, wherein the new YOLOv10 network is provided with an Input input terminal, a Backbone network, a Neck network and a Head network in sequence; Among them, the separable convolutional star module StarBlock-SC is used to replace the original feature extraction module C2f of the Backbone network; and before the three sets of feature maps output by the Neck network, a multi-scale spatial attention expansion convolution fusion module DAS-Block is added respectively, and then the feature maps are output to the Head network; The model training module is used to train the detection model constructed in step S2 using the labeled image data to obtain a seat belt detection model for seat belt wearing detection.

9. A computer terminal, characterized in that: At least: one or more processors; a memory storing one or more computer programs; The processor calls the computer program to implement: The steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored, which is called by a processor to implement: The steps of the method according to any one of claims 1 to 7.

Images