A Detection Method for Bicycle Helmets Based on Improved FCOS and Embedded Grouping

Abstract

The present invention discloses a two-wheeled vehicle helmet detection method based on improved FCOS and embedded grouping, comprising the following steps: S1, acquiring image data of two-wheeled vehicle roads and making a two-wheeled vehicle helmet detection data set; S2, preprocessing the two-wheeled vehicle helmet detection data set; S3, constructing an object detection model based on improved FCOS and association grouping; S4, optimizing the constructed object detection model based on improved FCOS and association grouping by using the stochastic gradient descent method. It not only uses the improved FCOS object detection algorithm for pixel-level prediction, greatly reducing the number of model training parameters and meeting the requirements of high precision and real-time performance, but also can integrate the detection task and the grouping task in one network, measuring the member distribution differences between the predicted grouping and the true grouping and between different predicted groupings through cosine similarity, and accurately matching the two-wheeled vehicle and the rider.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and specifically refers to a method for detecting the safety helmets of two-wheeled vehicles based on improved FCOS and embedded grouping. Background Art

[0002] In recent years, the number of two-wheeled vehicles represented by motorcycles and electric bicycles in China has increased rapidly. At the same time, due to the lack of safety protection awareness among riders, traffic accidents involving two-wheeled vehicles occur frequently, and the number of deaths has been increasing year by year. Relevant research shows that correctly wearing a safety helmet can reduce the risk of death in traffic accidents by 60% to 70%, which plays an important role in protecting the lives of riders. Therefore, a national deployment has been carried out to launch a "safety protection action for helmets and seat belts", and the behavior of motorcycle and electric bicycle riders not wearing safety helmets has been investigated and corrected according to law. However, the above-mentioned method of manual interception consumes a large amount of police force and has a small coverage area, and cannot achieve good control. With the gradual improvement of video acquisition equipment at intersections and non-motorized lanes, the demand for using computer vision to detect specific targets in videos has become increasingly prominent. Thus, efficiently detecting the wearing situation of safety helmets of two-wheeled vehicle riders has become an important research content in two-wheeled vehicle traffic control.

[0003] With the development of deep learning technology, methods for detecting the helmets of riders on two-wheeled vehicles based on computer vision have been continuously proposed. Currently, there are mainly three types of detection methods. The first type regards the rider and the two-wheeled vehicle as a whole and performs image classification on the whole of the person and the vehicle. However, this method does not locate the helmet and only judges whether the helmet is worn by performing image classification on the whole of the person and the vehicle, resulting in an extreme class imbalance problem. For example, the number of instances with two or more riders is very small, and the image categories in the collected data sets cannot cover all situations in reality. The second type is a two-stage method, that is, first detect the whole person and vehicle to obtain the cropped image patch, and then identify and locate the helmet wearing situation. However, in a crowded actual scene, it is easy to lead to misjudgment. To address this shortcoming, relevant researchers introduced Human-Object Interaction in the second step. If a human interaction point is detected, it is determined that there is a wearing relationship between the helmet and the person. Obviously, this method utilizes deep semantic information through human interaction and effectively alleviates the misjudgment problem caused by only using position information in a crowded scene. However, all such methods require training two independent object detection models, resulting in cumulative errors. The detection effect of the previous step greatly affects the detection accuracy of the next step, and it cannot meet the real-time requirement. The third type is a one-stage method that simultaneously detects the helmet and the whole person and vehicle, and then uses the position information IoF (Intersection over Foreground), that is, the overlapping part of the two regions divided by the part that is used as the foreground region, for grouping and pairing. However, only using the position information IoF is prone to misjudgment in a crowded scene.

[0004] Most of the existing grouping methods for two-wheeled vehicle helmet detection use the facial features of the rider for identity recognition. Due to the complexity of the real traffic scene, the facial features of the rider are affected by factors such as clothing, posture, background, or occlusion, and it is easy to have missed detections and misdetections. Currently, most provinces in the country have completed the registration and licensing of electric bicycles, and the license plate recognition technology is quite mature. Effectively using the license plate information can better identify the identity of riders not wearing helmets. Therefore, correctly grouping and pairing the helmet and the two-wheeled vehicle has become an important link in two-wheeled vehicle helmet detection.

[0005] In addition, the vast majority of current methods are based on object detection algorithms with anchor boxes, such as Faster RCNN, SSD, and YOLOv4. However, the artificially set anchor boxes carry a large number of hyperparameters, which have a great impact on indicators such as the recall rate and speed of detection. In the training stage, to determine whether each anchor box is a positive sample or a negative sample, it is usually necessary to calculate the intersection over union (IoU) of each anchor box with all the labels, which will consume a large amount of memory resources and computing time. The dense anchor box setting is prone to a large number of simple negative samples, and these samples have no effect on the detector. Traditional anchor-free object detection algorithms, such as YOLOv1, have problems such as low bounding box recall rate and average detection effect for overlapping objects. To solve the above problems, an anchor-free object detection algorithm, FCOS (Fully Convolutional One-Stage Object Detection), directly uses the pixel points containing the object to regress the object, suppresses the low-quality bounding boxes far from the center point through center-ness, and improves the recall rate. The Feature Pyramid Networks are used to solve the problem of ambiguous samples. However, the weight assignment strategy called center-ness assigns a smaller weight to small objects with a small number of positive samples, reducing their contribution to the loss; large objects have more positive samples and are assigned a larger weight, resulting in redundant loss contribution, which in turn affects the final object detection effect.

[0006] Based on the above considerations, for the actual scenario of traffic applications, there is an urgent need to design a method for detecting the riders of two-wheeled vehicles and their helmet-wearing situations in real time while meeting the accuracy requirements, and accurately grouping and pairing helmets with two-wheeled vehicles to facilitate subsequent operations for identifying the identities of illegal persons, such as license plate recognition and face recognition. Summary of the Invention

[0007] The purpose of the present invention is to provide a method for detecting two-wheeled vehicle helmets based on improved FCOS and embedded grouping, aiming at the deficiencies and unaddressed aspects of existing methods. This method uses the improved FCOS object detection algorithm for pixel-level prediction, greatly reducing the number of model training parameters and meeting the requirements of high accuracy and real-time performance. At the same time, it can integrate the detection task and the grouping task in one network, measure the difference in member distribution between the predicted grouping and the true grouping and between different predicted groupings through cosine similarity, and accurately match two-wheeled vehicles with riders.

[0008] To solve the above technical problems, the technical solution of the present invention is as follows:

[0009] The present invention provides a method for detecting two-wheeled vehicle helmets based on improved FCOS and embedded grouping, including the following steps:

[0010] S1. Obtain the image data of the two-wheeler road and create a two-wheeler helmet detection dataset;

[0011] S2. Preprocess the two-wheeler helmet detection dataset;

[0012] S3. Build an object detection model based on improved FCOS and association grouping

[0013] S3-1. Extract features from the input image and output multi-scale feature maps;

[0014] S3-2. Input the multi-scale feature maps to achieve local bidirectional feature fusion;

[0015] S3-3. Build a detector with 4 branches to achieve classification, regression, centerness, and grouping, and output class labels, bounding boxes, and grouping information, and calculate the losses of classification, regression, centerness, and grouping;

[0016] S4. Use the stochastic gradient descent method to optimize the built object detection model based on improved FCOS and association grouping.

[0017] Preferably, the step S1 includes:

[0018] S1-1. Use 910 consecutive frames collected from the traffic roads publicly available by OSF to obtain a total of 5448 two-wheeler road images;

[0019] S1-2. Make a two-wheeler helmet detection dataset according to the annotation format of the Pascal VOC dataset.

[0020] Preferably, the preprocessing method in the step S2 is: respectively locate, classify, and label the group IDs of the heads of the riders wearing helmets, the heads of the riders not wearing helmets, two-wheelers, pedestrians, and bicycles in the two-wheeler helmet detection dataset, and divide the dataset into a training set, a validation set, and a test set according to a certain ratio and data distribution.

[0021] Preferably, in the step S3-1, the method for image feature extraction is:

[0022] S3-1-1. Use ResNet-50 as the backbone network to extract features, input the RGB image I ∈ R with height H, width W, and number of channels 3, denoted as (H, W, 3), where R is the real number field, and sequentially pass through 5 stages; H×W×3 Input, denoted as (H, W, 3), where R is the real number field, and sequentially pass through 5 stages;

[0023] S3-1-2. Stage 1 is regarded as the preprocessing of the input (H, W, 3). It successively passes through a convolutional layer, a BN layer, a ReLU activation function, and a max pooling layer, and outputs a feature map with a resolution of 1 / 4 of the original image and 64 channels. Among them, the convolutional layer has a convolutional kernel size of 7×7 and 64 convolutional kernels, that is, the number of output channels of this convolutional layer, and the convolutional kernel stride is 2; the max pooling layer has a convolutional kernel size of 3×3 and a stride of 2.

[0024] S3-1-3. In stages 2 to 5, residual blocks with skip connections and downsampling residual blocks are used. In addition, a 1×1 convolutional operation is added to the main branch of the downsampling residual blocks in stages 2 to 5. Stages 2, 3, 4, and 5 all start with downsampling residual blocks, followed by 2, 3, 2, and 5 residual blocks respectively. Stages 2 to 5 of ResNet-50 output 4 feature maps respectively, with resolutions of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image in sequence, and the number of channels are 256, 512, 1024, and 2048 in sequence.

[0025] S3-1-4. Select the feature maps C3, C4, and C5 output by stages 3, 4, and 5 of ResNet-50 as the output. In ResNet-50, the network layers with the same resolution output are network layers of the same stage. The feature map C output by the last layer of stage i i is called the feature map extracted in this stage.

[0026] Preferably, in step S3-2, the local bidirectional feature fusion method is as follows:

[0027] S3-2-1. Design a feature pyramid network for local bidirectional fusion to give more attention to features of specific scales. Since the road width in the actual traffic scene is generally 1.5m - 3.5m, the installation height of the road monitoring camera is 3m - 5m, and the scales of the tricycle and helmet in the image are relatively balanced and generally below 256×256.

[0028] S3-2-2. Partial-FPN adds local bottom-up feature fusion on the basis of FPN, detects targets of different sizes on feature maps of different scales, and defines the 5 output feature maps as {P3, P4, P5, P6, P7}.

[0029] S3-2-3. Input the C3, C4, and C5 feature maps into Partial-FPN, first perform top-down feature fusion, and then perform bottom-up feature fusion, and use a 3×3 convolution to eliminate the aliasing effect to obtain P3, P4, and P5; P6 and P7 are respectively generated from P5 and P6 through a convolutional layer with a stride of 2.

[0030] Preferably, in the step S3-3, the following sub-steps are included:

[0031] S3-3-1. The true value is B i , where and are the upper left and lower right vertex coordinates of the true value bounding box respectively, c (i) is the category of the true value. The center sampling strategy is used to define a sub-box (c x -rs, c y -rs, c x +rs, c y +rs) in the true value bounding box, where (c x , c y ) is the center of the true value bounding box, s is the stride of the feature map, r is a hyperparameter, r = 1.5; for each position (x, y) on the feature map, it is mapped to the original image according to the following formula:

[0032]

[0033] If it is located within the above sub-box, it is considered a positive sample, and the value of the category label c * is the value of the category label of this true value bounding box; otherwise, it is a negative sample, c * = 0;

[0034] S3-3-2. Use the 4D real vector t * = (l * , t * , r * , b * ) as the regression target for this position, where l * , t * , r * , b * are the distances to the four sides of the bounding box, l is left, t is top, r is right, and b is bottom; the formula used to calculate t * is as follows:

[0035]

[0036] Among them, the regression target is scaled using the stride s of the feature map. Since the 4 regression values l * , t * , r * , b * must all be greater than 0, the final output uses ReLU to ensure that the range of the regression value is (0, +∞);

[0037] S3-3-3. The detector includes two independent 4-layer convolutional networks with a convolutional kernel size of 3×3 and 256 channels. One branch is used to generate class prediction results and grouping prediction results, which is simply referred to as the classification branch. The other branch is used to generate the regression results of the center-ness and bounding boxes. Among them, the center-ness is denoted as center-ness, and this branch is simply referred to as the regression branch;

[0038] S3-3-4. The last layer of the detector network predicts a C-dimensional vector p as the class label and a 4-dimensional vector t = (l, t, r, b) representing the coordinates of the bounding box. Among them, C represents the number of classes, which are the heads of riders wearing helmets, the heads of riders not wearing helmets, two-wheeled vehicles, pedestrians, and bicycles respectively. The classification detector finally outputs a feature vector of H×W×5, and the regression detector finally outputs a feature vector of H×W×4;

[0039] S3-3-5. In the step S3-3-3, center-ness is used to suppress the low-quality detection boxes predicted by positions deviating from the center of the target, and the output is a feature vector of H×W×1. Center-ness predicts a value: the normalized distance between the current position and the center point of the predicted object, which is distributed between [0, 1]. For a given regression target t * =(l * ,t * ,r * ,b * ), center-ness can be formulated as:

[0040]

[0041] Among them, since the value of center-ness is distributed between 0 and 1, Binary Cross Entropy loss is used for training. In the test phase, the final confidence is the product of center-ness and the classification probability;

[0042] S3-3-6. By restricting the regression values of the targets on different feature maps, the allocation of targets at different scales on the feature maps is achieved. Each feature map P i sets the lower limit value m i -1 and the upper limit value m i , i ∈ {3, 4, 5, 6, 7}, m2, m3, m4, m5, m6, m7 are set to 0, 64, 128, 256, 512, +∞ respectively, and can be expressed by the formula as:

[0043] max(l * ,t * ,r * ,b * ) < mi and max(l * ,t * ,r * ,b * ) > m i -1

[0044] The obtained bounding boxes after regression are processed by non-maximum suppression to retain the prediction bounding box with the best effect;

[0045] S3-3-7. For the grouped prediction in step S3-3-3, the detection task and the grouping task are integrated into a network structure, sharing the parameters of the convolutional layer with the classification detector, and obtaining the embedding feature vector corresponding to the target through the convolutional calculation of the embedding layer, specifically H×W×128;

[0046] S3-3-8. In step S2, the grouping ID label is applied to the grouping detector, and the loss function Groupingloss is defined to measure the matching degree between the predicted label and the true label grouping situation; retrieve the labels of all the heads of the two-wheeled vehicles and the riders at the corresponding true positions in the image, and then compare the labels between each two-wheeled vehicle. The labels belonging to the same two-wheeled vehicle are the same, and vice versa, the labels belonging to different two-wheeled vehicles are different.

[0047] Preferably, in step S4, the optimization method of the object detection model is: obtaining the prediction result through the forward propagation of the object detection model of the two-wheeled vehicle helmet based on the improved FCOS and embedded grouping, calculating the loss function, and then performing backpropagation to update the parameters of the object detection model to obtain the final model.

[0048] Preferably, the parameters of the object detection model include the category and position of the two-wheeled vehicle obtained by sequentially passing the new image through steps S2 to S3, the position of the rider's head, the group of the two-wheeled vehicle, and the classification information of whether the helmet is worn.

[0049] Preferably, the loss function is specifically as follows:

[0050]

[0051] where Take 1 when it is a positive sample and 0 when it is a negative sample; λ is the weight coefficient; N pos represents the number of positive samples; L cls is the loss function for classification, using focal loss; L reg is the loss function for regression, L center-ness is the loss function for center-ness, using BCE; L grounpingIt is a loss function for grouping, using grouping loss. The specific formula of grouping loss is as follows:

[0052] L grounping =W pull ×L pull +W push ×L push

[0053]

[0054]

[0055]

[0056] Pull means "pull", and its purpose is to make the embeddings of all members in the same group as close as possible; push means "push", and its purpose is to make the embeddings of members in different groups as far away as possible; W pull is the loss weight of L pull ; W push is the loss weight of L push ; f i is the embedding feature vector of the sample; margin is an adjustable offset, and its value range is [-1, 1].

[0057] The present invention has the following characteristics and beneficial effects:

[0058] 1) Using the anchor-free FCOS object detection algorithm for pixel-level prediction, significantly reducing the number of model training parameters, and meeting the requirements of high precision and real-time performance;

[0059] 2) Designing a network architecture in which the detection task and the grouping task are tightly coupled, and measuring the member distribution differences between the predicted grouping and the true grouping and between different predicted groupings through cosine similarity, accurately matching the two-wheeler and the rider. This lays a foundation for the actual application scenario of two-wheeler helmet detection. Brief Description of the Drawings

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0061] Figure 1 It is a schematic flowchart of the two-wheeler helmet detection method based on improved FCOS and embedding grouping in the embodiments of the present invention;

[0062] Figure 2 is the network architecture diagram of the tricycle helmet detection method based on improved FCOS and embedded grouping in the embodiments of the present invention;

[0063] Figure 3 is the architecture diagram of the feature pyramid network with local bidirectional fusion in the embodiments of the present invention;

[0064] Figure 4 is the schematic diagram of the central sampling strategy in the embodiments of the present invention;

[0065] Figure 5 is the schematic diagram of the implementation process of the tricycle helmet detection model based on improved FCOS and embedded grouping in the embodiments of the present invention;

[0066] Figure 6 is the schematic diagram of the matching ambiguity problem based on grouping by location information;

[0067] Figure 7 is the schematic diagram of embedded grouping in the embodiments of the present invention. Detailed implementation manners

[0068] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments may be combined with each other.

[0069] A tricycle helmet detection method based on improved FCOS and embedded grouping, as shown in Figure 1 and Figure 5 includes the following steps:

[0070] S1. Obtain the image data of the tricycle road and make a tricycle helmet detection data set,

[0071] Specifically, it includes the following steps:

[0072] S1-1. Use 910 consecutive frames collected from the open traffic roads of OSF (Open Science Framework) to obtain a total of 5448 tricycle road images. The open traffic road data comes from the traffic road data of Myanmar; use web crawler technology to obtain 1032 tricycle road images to increase the complexity of the background. In this embodiment, a total of 6480 tricycle road images are obtained.

[0073] S1-2. Make a tricycle helmet detection data set according to the annotation format of the Pascal VOC data set.

[0074] S2. Preprocess the tricycle helmet detection data set,

[0075] Locate, classify, and assign group IDs to the heads of riders wearing helmets, the heads of riders not wearing helmets, bicycles, pedestrians, and bicycles in the bicycle helmet detection dataset, and divide the dataset into a training set, a validation set, and a test set according to a certain ratio and data distribution.

[0076] In this embodiment, the data division ratio of the training set, the validation set, and the test set is 3:1:2.

[0077] S3. Build an object detection model based on improved FCOS and associated grouping

[0078] S3-1. Extract features from the input image and output multi-scale feature maps.

[0079] Specifically, as Figure 2 shown, in step S3-1, the method for image feature extraction is:

[0080] S3-1-1. Use ResNet-50 as the backbone network to extract features. Input the RGB image I ∈ R with height H, width W, and number of channels 3, denoted as (H, W, 3), where R is the real number field, and go through 5 stages in sequence. H×W×3 Input, denoted as (H, W, 3), where R is the real number field, and go through 5 stages in sequence.

[0081] S3-1-2. Consider stage 1 as the preprocessing of the input (H, W, 3). First, pass through a convolutional layer, a BN layer, a ReLU activation function, and a max pooling layer to output a resolution of 1 / 4 of the original image with 64 channels. Among them, the convolutional layer has a convolutional kernel size of 7×7, the number of convolutional kernels is 64, that is, the number of output channels of this convolutional layer, and the convolutional kernel stride is 2; the max pooling layer has a convolutional kernel size of 3×3 and a stride of 2.

[0082] S3-1-3. In stages 2 to 5, both residual blocks with skip connections and downsampling residual blocks are used. In addition, a 1×1 convolutional operation is added to the main branch of the downsampling residual blocks in stages 2 to 5. Stages 2, 3, 4, and 5 all start with downsampling residual blocks, followed by 2, 3, 2, and 5 residual blocks respectively. Stages 2 to 5 of ResNet-50 output 4 feature maps respectively, with resolutions of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image in sequence, and the number of channels is 256, 512, 1024, and 2048 in sequence.

[0083] S3-1-4. Select the feature maps C3, C4, and C5 output by stages 3, 4, and 5 of ResNet-50 as the output. The network layers with the same resolution output in ResNet-50 are network layers of the same stage. The feature map C i output by the last layer of stage i is called the feature map extracted by this stage.

[0084] S3-2. Input feature maps of multiple scales to achieve local bidirectional feature fusion;

[0085] Specifically, in the step S3-2, the method of local bidirectional feature fusion is as follows:

[0086] S3-2-1. Design a Partial Feature Pyramid Network (Partial-FPN) for local bidirectional fusion, which gives more attention to features of specific scales.

[0087] It is easy to understand that since the road width in the actual traffic scene is generally 1.5m - 3.5m, the installation height of the road monitoring camera is 3m - 5m. Therefore, the scales of the tricycle and the helmet in the image are relatively balanced and generally below 256×256.

[0088] S3-2-2. Partial-FPN adds local bottom-up feature fusion on the basis of FPN, detects targets of different sizes on feature maps of different scales, and the 5 output feature maps are defined as {P3, P4, P5, P6, P7}.

[0089] Input the C3, C4, and C5 feature maps into Partial-FPN. First, perform top-down feature fusion, then select some scales for bottom-up feature fusion, and use a 3×3 convolution in the middle to eliminate the aliasing effect, obtaining P3, P4, and P5; P6 and P7 are generated by P5 and P6 respectively through a convolutional layer with a stride of 2.

[0090] Specifically, C5 is fused with C4 through 2-fold upsampling to obtain C'4; C'4 is fused with C3 through 2-fold upsampling to obtain C'3, and C'3 passes through a 3×3 convolution to eliminate the aliasing effect, obtaining P3; after P3 undergoes a convolution operation with a 3×3 convolution kernel and a stride of 2, it is fused with C'4, and then passes through a 3×3 convolution to obtain P4. Similarly, P5 is obtained; P6 and P7 are generated by P5 and P6 respectively through a convolutional layer with a stride of 2. This method reduces the number of parameters, and the generated feature maps have stronger semantics and better effects; among them, fusion means performing an addition operation on the feature maps, and the feature maps obtained by bidirectional feature fusion are P4 and P5, corresponding to the scale ranges of (63, 128) and (128, 256) respectively; the aliasing effect is caused by the superposition of the upsampled feature map and the original image, resulting in discontinuous features and chaotic features on the original feature map. The 3×3 convolution is equivalent to the operation of a filter, performing re-extraction of features to ensure the stability of features.

[0091] S3-3. Construct a detector with 4 branches, namely bounding box regression, classification prediction, centrality, and grouping prediction, to achieve classification, regression, centrality, and grouping. The output is the class label, bounding box, and grouping information, and calculate the losses of classification, regression, centrality, and grouping.

[0092] Specifically, in step S3-3, the following sub-steps are included:

[0093] S3-3-1. The ground truth is B i , where and are the top-left and bottom-right vertex coordinates of the ground truth bounding box respectively, c (i) is the class of the ground truth. Use the center sampling strategy to define a sub-box (c x -rs, c y -rs, c x +rs, c y +rs) in the ground truth bounding box, where (c x , c y ) is the center of the ground truth bounding box, s is the stride of the feature map, r is a hyperparameter, r = 1.5. For each position (x, y) on the feature map, map it to the original image according to the following formula:

[0094]

[0095] The position mapped by this formula is close to the center of the receptive field of the aforementioned position (x, y). If the mapped position is within the above sub-box, it is considered a positive sample, and the value of the class label c * is the value of the class label of this ground truth bounding box; otherwise, it is a negative sample, c * = 0. The center sampling strategy well compensates for the problem that the centrality strategy assigns smaller weights to small objects with a small number of positive samples and larger weights to large objects with a large number of positive samples, thereby affecting the final object detection, and balances the influence of the size of objects on the weights.

[0096] S3-3-2. Use a 4-dimensional real vector t * =(l * , t * , r * , b * ) as the regression target for this position, where l * , t * , r * , b * are the distances to the four sides of the bounding box, l is left, t is top, r is right, and b is bottom. The formula used to calculate t * is as follows:

[0097]

[0098] Among them, the step size s of the feature map is used to scale the regression target to avoid gradient explosion that may occur during the training process. Since the four regression values l * , t * , r * , b * are all greater than 0, the final output uses ReLU to ensure that the range of the regression value is (0, +∞);

[0099] Compared with the regression target (x, y, w, h) of the early anchor-free algorithm YOLOv1, where (x, y) are the center coordinates of the bounding box, and w and h are the width and height of the bounding box, using the above 4D real vector as the regression target is simpler and more direct, and at the same time reduces the computational complexity during regression.

[0100] S3-3-3. The detector includes two independent 4-layer convolutional networks with a convolutional kernel size of 3×3 and 256 channels; one branch is used to generate class prediction results and grouping prediction results, and this branch is simply referred to as the classification branch; the other branch is used to generate the center-ness and the regression results of the bounding box, and this branch is simply referred to as the regression branch;

[0101] S3-3-4. The last layer of the detector network predicts a C-dimensional vector p as the class label and a 4-dimensional vector t = (l, t, r, b) representing the coordinates of the bounding box; where C represents the number of classes. In this embodiment, C = 5, which are the heads of riders wearing helmets, the heads of riders not wearing helmets, two-wheel vehicles, pedestrians, and bicycles respectively; the classification detector finally outputs a feature vector of H×W×5, and the regression detector finally outputs a feature vector of H×W×4;

[0102] S3-3-5. In the step S3-3-3, center-ness is used to suppress the low-quality detection boxes predicted at positions deviating from the center of the target, and the output is a feature vector of H×W×1; center-ness predicts a value: the normalized distance between the current position and the center point of the predicted object, which is distributed between [0, 1]; for a given regression target t * =(l * , t * , r * , b * ), center-ness can be formulated as:

[0103]

[0104] Among them, since the values of center-ness are distributed between 0 and 1, Binary Cross Entropy loss is used for training. In the test phase, the final confidence is the product of center-ness and the classification probability.

[0105] S3-3-6. By restricting the regression values of the targets on different feature maps, the distribution of targets of different scales on the feature maps is realized; for each feature map P i Set the lower limit value m i -1 and the upper limit value m i , i ∈ {3, 4, 5, 6, 7}, m3, m4, m5, m6, m7 are set to 0, 64, 128, 256, 512, +∞ respectively, and it can be expressed by the formula:

[0106] max(l * , t * , r * , b * ) < m i and max(l * , t * , r * , b * ) > m i -1

[0107] The regression-generated bounding boxes are processed by non-maximum suppression to retain the prediction bounding box with the best effect.

[0108] Specifically, non-maximum suppression is used to suppress elements that are not the maximum value. For the set C of bounding boxes and their corresponding confidences S, select the bounding box m with the maximum confidence, remove it from the set B and add it to the final detection result, and remove the bounding boxes in the remaining bounding boxes whose IoU with m is greater than the set threshold from the set B; repeat this process until the set B is empty.

[0109] S3-3-7. For the grouped prediction in step S3-3-3, the embedding grouping method is used. Inspired by the multi-person human pose estimation algorithm associate embedding, the detection task and the grouping task are integrated into a network structure, sharing the parameters of the convolutional layer with the classification detector, and obtaining the embedding feature vector corresponding to the target through the convolutional calculation of the embedding layer.

[0110] Specifically, the convolution kernel is 3×3, the stride is 1, the padding is 1, the number of input channels is the number of channels of the feature map, and the number of output channels is set manually; the normalization process normalizes the feature matrix output by the embedding layer row by row, and each row is divided by the square root of the sum of the squares of all elements below that row; in this embodiment, the embedding finally becomes a feature vector of H×W×128.

[0111] Such as Figure 6 and Figure 7 , grouping based on position information is a common grouping method. However, in the actual road scenario of a two-wheeler, the head features of the rider are easily affected by factors such as posture, background, or occlusion, and misalignment caused by the limitation of the road camera's perspective is relatively common. The grouping method based on position information outputs incorrect groupings due to fuzzy matching, while the embedding grouping method uses the network to learn deep semantic relationships by itself, represented as the aforementioned embedding vectors, and then calculates the similarity to assign corresponding labels to the targets, which can better complete the grouping task.

[0112] S3-3-8. In the step S2, the grouping ID label is applied to the grouping detector, and a loss function Groupingloss is defined to measure the matching degree between the predicted label and the true label grouping situation; retrieve the labels of all two-wheelers and the rider's head at the corresponding true positions in the image, and then compare the labels between each two-wheeler. The labels belonging to the same two-wheeler are the same, and conversely, the labels belonging to different two-wheelers are different.

[0113] It should be noted that this method does not need to input the true labels into the network for prediction because what matters is not the specific label values but the differences between them; the network can freely determine the predicted label values as long as they are consistent with the grouping of the true labels; group and pair the rider's head and the two-wheeler according to the cosine similarity between the embeddings; the smaller the cosine similarity between the embeddings corresponding to the targets in the same group, and the larger the cosine similarity between the embeddings corresponding to the targets in different groups;

[0114] Furthermore, the calculation formula of the cosine similarity is as follows:

[0115]

[0116] where a and b represent the two input embedding vectors, y represents the true group label, belonging to {1, -1}, representing similarity and dissimilarity respectively, margin is an adjustable offset, and its value range is [-1, 1]. A better value is between 0 and 0.5. In this embodiment, margin = 0.3;

[0117] S4. Optimize the constructed object detection model based on improved FCOS and associative grouping using the stochastic gradient descent method.

[0118] Specifically, in step S4, the optimization method of the object detection model is as follows: The parameters of the object detection model include the category and position of the tricycle obtained by sequentially passing the input new image through steps S2 - S3, the head position of the rider, the group of the tricycle, and the classification information of whether a helmet is worn.

[0119] Obtain the prediction result through the forward propagation of the object detection model of the tricycle helmet based on improved FCOS and embedded grouping, calculate the loss function, and then perform backpropagation to update the parameters of the object detection model to obtain the final model. Specifically, this model is optimized using the stochastic gradient descent method, with the initial learning rate set to 0.01 and the mini - batch set to 16 images, through 80K iterations.

[0120] Among them, the loss function is specifically as follows:

[0121]

[0122] Among them Take 1 when it is a positive sample and 0 when it is a negative sample; λ is the weight coefficient, λ = 1; N pos represents the number of positive samples; L cls is the classification loss function, using focal loss; L reg is the regression loss function, using GIoU (Generalized Intersection over Union) loss, with better effect than IoU loss; L center-ness is the center - ness loss function, using BCE; L grounping is the grouping loss function, using grouping loss, and the specific formula of grouping loss is as follows:

[0123] L grounping = W pull ×L pull + W push ×L push

[0124]

[0125]

[0126]

[0127] "pull" means "pull", aiming to make the embeddings of all members of the same group as close as possible; "push" means "push", aiming to make the embeddings of members of different groups as far away as possible; W pull is the loss weight of L pull In this embodiment, W pull = 0.01; W push is the loss weight of L push In this embodiment, W push = 0.49; f i is the embedding feature vector of the sample; margin is an adjustable offset, with a value range of [-1, 1]. In this embodiment, margin = 0.3.

[0128] It can be understood that in the above technical solution, a tricycle helmet detection model based on improved FCOS and embedding grouping is constructed, the target detection model is trained using the training set, the models saved at each node are evaluated and selected using the validation set, the generalization ability of the model is tested using the test set, and finally the tricycle helmet detection method based on improved FCOS and embedding grouping is realized.

[0129] The embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, without departing from the principles and spirit of the present invention, various changes, modifications, substitutions, and variations to these embodiments including components still fall within the protection scope of the present invention.

Claims

1. A detection method for two-wheeler helmets based on improved FCOS and embedded grouping, characterized in that, It includes the following steps: S1. Obtain the image data of the two-wheeler road and make a two-wheeler helmet detection dataset; S2. Preprocess the two-wheeler helmet detection dataset; S3. Build an object detection model based on improved FCOS and association grouping S3-1. Extract features from the input image and output multi-scale feature maps; S3-2. Input the multi-scale feature maps to achieve local bidirectional feature fusion; S3-3. Build a detector with 4 branches to achieve classification, regression, centerness, and grouping. The output is class labels, bounding boxes, and grouping information, and calculate the losses of classification, regression, centerness, and grouping; It includes the following sub-steps: S3-3-1, the ground truth is B i , where and are the upper left and lower right vertex coordinates of the ground truth bounding box respectively, c (i) is the class of the ground truth. A sub-box (c x -rs, c y -rs, c x +rs, c y +rs) is defined in the ground truth bounding box using the center sampling strategy, where (c x , c y ) is the center of the ground truth bounding box, s is the stride of the feature map, r is a hyperparameter, r = 1.5; for each position (x, y) on the feature map, it is mapped to the original image according to the following formula: If it is located within the above-mentioned sub-box, it is considered a positive sample, and the value of the class label c * is the value of the class label of this ground truth bounding box; otherwise, it is a negative sample, and c * = 0; S3-3-2. Using the 4D real vector t * =(l * , t * , r * , b * ) as the regression target for the said position, where l * , t * , r * , b * are the distances to the four sides of the bounding box, l is left, t is top, r is right, and b is bottom; the formula used to calculate t * is as follows: Among them, the step size s of the feature map is used to scale the regression target. Since the four regression values l * , t * , r * , b * are all greater than 0, the ReLU is finally used to ensure that the range of the regression value is (0, +∞); S3-3-3. The detector includes two independent 4-layer convolutional networks with a convolutional kernel size of 3×3 and 256 channels. One branch is used to generate class prediction results and grouping prediction results, which is simply called the classification branch; the other branch is used to generate regression results of centerness and bounding boxes. Among them, centerness is denoted as center-ness, and this branch is simply called the regression branch; S3-3-4. The last layer of the detector network predicts a C-dimensional vector p as the class label and a 4-dimensional vector t=(l, t, r, b) representing the coordinates of the bounding box; where C represents the number of classes, which are the heads of riders wearing helmets, the heads of riders not wearing helmets, two-wheelers, pedestrians, and bicycles; the classification detector finally outputs a feature vector of H×W×5, and the regression detector finally outputs a feature vector of H×W×4; S3-3-5. In the step S3-3-3, center-ness is used to suppress the low-quality detection boxes predicted by deviating from the center position of the target, and the output is a feature vector of H×W×1; center-ness predicts a value: the normalized distance between the current position and the center point of the predicted object, which is distributed between [0, 1]; for a given regression target t * =(l * , t * , r * , b * ), center-ness can be formulated as: Among them, since the value of center-ness is distributed between 0 and 1, Binary Cross Entropy loss is used for training. In the test stage, the final confidence is the product of center-ness and the classification probability; S3-3-6. By restricting the regression values of the targets on different feature maps, the allocation of targets of different scales on the feature maps is achieved; each feature map P i Set the lower limit value m i -1 and the upper limit value m i , i ∈ {3, 4, 5, 6, 7}, m2, m3, m4, m5, m6, m7 are respectively set to 0, 64, 128, 256, 512, +∞, and can be expressed by the formula as: max(l * ,t * ,r * ,b * ) < m i and max(l * ,t * ,r * ,b * ) > m i -1 The regression-derived bounding boxes are processed by non-maximum suppression to retain the best predicted bounding box; S3-3-7. For the grouping prediction in step S3-3-3, the detection task and the grouping task are integrated into a network structure, sharing the parameters of the convolutional layer with the classification detector, and obtaining the embedding feature vector corresponding to the target through the convolutional calculation of the embedding layer, specifically H×W×128; S3-3-8. In step S2, the grouping ID label is applied to the grouping detector, and the loss function Grouping loss is defined to measure the matching degree between the predicted label and the true label grouping situation; retrieve the labels of all two-wheelers and rider heads in the corresponding true positions in the image, and then compare the labels between each two-wheeler. The labels belonging to the same two-wheeler are the same, and vice versa, the labels belonging to different two-wheelers are different; S4. Use the stochastic gradient descent method to optimize the built object detection model based on improved FCOS and association grouping.

2. The two-wheeler helmet detection method based on improved FCOS and embedding grouping according to claim 1, wherein The step S1 includes: S1-1. Use 910 consecutive frames collected from the traffic roads publicly available by OSF to obtain a total of 5448 two-wheeler road images; S1-2. Make a two-wheeler helmet detection dataset according to the annotation format of the Pascal VOC dataset.

3. The method for detecting a two-wheeler helmet based on improved FCOS and embedded grouping according to claim 1, wherein The preprocessing method in step S2 is as follows: Locate, classify, and label the group IDs for the heads of riders wearing helmets, the heads of riders not wearing helmets, the bicycles, pedestrians, and bicycles in the bicycle helmet detection dataset, and divide the dataset into a training set, a validation set, and a test set according to a certain ratio and data distribution.

4. The method for detecting a two-wheeler helmet based on improved FCOS and embedded grouping according to claim 1, wherein, In step S3-1, the method for image feature extraction is as follows: S3-1-1. Use ResNet-50 as the backbone network to extract features. The RGB image \(I\in\mathbb{R}\) with height \(H\), width \(W\), and 3 channels is H×W×3 input, denoted as \((H, W, 3)\), where \(\mathbb{R}\) is the real number field, and it goes through 5 stages in sequence; S3-1-2. Stage 1 is regarded as the preprocessing of the input (H, W, 3). It passes through a convolutional layer, a BN layer, a ReLU activation function, and a max pooling layer in sequence, and the output resolution is 1 / 4 of the original image, and the number of channels is 64. The convolutional kernel size of the convolutional layer is 7×7, and the number of convolutional kernels is 64, that is, the number of output channels of this convolutional layer. The convolutional kernel stride is 2. The convolutional kernel size of the max pooling layer is 3×3, and the stride is 2. S3-1-3. In stages 2 to 5, residual blocks with skip connections and downsampling residual blocks are used. In addition, a 1×1 convolutional operation is added to the main branch of the downsampling residual blocks in stages 2 to 5. Stages 2, 3, 4, and 5 all start with downsampling residual blocks, followed by 2, 3, 2, and 5 residual blocks respectively. Stages 2 to 5 of ResNet-50 output 4 feature maps respectively, with resolutions of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image in sequence, and the number of channels is 256, 512, 1024, and 2048 in sequence. S3-1-4. Select the feature maps C3, C4, and C5 output by stages 3, 4, and 5 of ResNet-50 as the output; in ResNet-50, the network layers with the same resolution output are network layers of the same stage, and the feature map C output by the last layer of stage i i is called the feature map extracted in this stage.

5. The method for detecting a two-wheeler helmet based on improved FCOS and embedded grouping according to claim 4, wherein In step S3-2, the method for local bidirectional feature fusion is as follows: S3-2-1. Design a feature pyramid network for local bidirectional fusion. The scales of bicycles and helmets in the image are relatively balanced and generally below 256×256. S3-2-2. Partial-FPN adds local bottom-up feature fusion on the basis of FPN, detects targets of different sizes on feature maps of different scales, and the 5 output feature maps are defined as {P3, P4, P5, P6, P7}. S3-2-3. Input the C3, C4, and C5 feature maps into Partial-FPN. First, perform top-down feature fusion, and then perform bottom-up feature fusion. Use a 3×3 convolution to eliminate the aliasing effect to obtain P3, P4, and P5. P6 and P7 are generated by P5 and P6 through convolutional layers with a stride of 2 respectively.

6. The method for detecting a two-wheeler helmet based on improved FCOS and embedded grouping according to claim 5, wherein, In step S4, the optimization method of the object detection model is as follows: Obtain the prediction results through the forward propagation of the object detection model for bicycle helmets based on the improved FCOS and embedded grouping, calculate the loss function, and then perform backpropagation to update the parameters of the object detection model.

7. The method for detecting a two-wheeler helmet based on improved FCOS and embedded grouping according to claim 6, wherein The parameters of the object detection model include the bicycle category, position, the head position of the rider, the group of the bicycle, and the classification information of whether the helmet is worn obtained by passing the new input image through steps S2 to S3 in sequence.

8. The method for detecting a two-wheeler helmet based on improved FCOS and embedded grouping according to claim 6, characterized in that, The specific loss function is as follows: Take 1 when it is a positive sample and 0 when it is a negative sample; λ is the weight coefficient; N pos represents the number of positive samples; L cls is the loss function for classification, using focal loss; L reg is the loss function for regression, L center-ness is the loss function for center-ness, using BCE; L grounping is the loss function for grouping, using grouping loss, and the specific formula of grouping loss is as follows: "pull" means "pull", aiming to make the embeddings of all members of the same group as close as possible; "push" means "push", aiming to make the embeddings of members of different groups as far away as possible; W pull is the loss weight of L pull ; W push is the loss weight of L push ; f i is the embedding feature vector of the sample; margin is an adjustable offset with a value range of [-1, 1].

Images