A real-time pedestrian detection method based on improved FCOS algorithm

Abstract

The present invention provides a real-time pedestrian detection method based on an improved FCOS algorithm, and belongs to the technical field of image processing and motor vehicle road safety driving. It solves the technical problem of low accuracy and real-time performance in detecting occluded pedestrians and small-target pedestrians in complex scenes. The technical solution is as follows: first, the FCOS algorithm is improved, and data enhancement is used for the data set to simulate the scene where the pedestrian's body is occluded and indirectly add small pedestrian targets to enrich the data set; secondly, a lightweight backbone network is introduced to replace the lightweight network model of the original feature extraction network, reduce network parameters, adjust the feature pyramid and head network; finally, a lightweight attention network is used and the backbone network feature fusion is adjusted. The beneficial effects of the present invention are: effective real-time pedestrian detection with more occlusions and small targets is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field, and in particular to a real-time pedestrian detection method based on an improved FCOS algorithm. Background Art

[0002] Pedestrian detection has a wide range of applications in autonomous driving, assisted driving, collision warning, robotics, etc. A safer and more reliable algorithm mechanism is the current development goal of related industries. Autonomous driving technology has high real-time requirements for pedestrian detection and recognition. In vision-based pedestrian detection systems and collision warnings, pedestrian detection is a basic and necessary function that can effectively avoid traffic accidents and improve road safety to a certain extent.

[0003] Most existing pedestrian detection technologies are based on the premise that the complete pedestrian can be displayed normally. Viola and Jones [1] AA EA AMZS I. Deep Learning-Based Pedestrian Detection in Autonomous Vehicles: Substantial Issues and Challenges [J]. Electronics, 2022, 11 (21): 3551-3551, for the first time used the intensity information and motion information of the image combined with the Adaboost classifier to achieve pedestrian detection and tracking, which attracted the attention of researchers. Dalal and Triggs[2]N.Dalal and B.Triggs,"Histograms of oriented gradients for human detection,"2005IEEE Computer Society Conference on Computer Vision and Pattern Recognition(CVPR'05),San Diego,CA,USA,2005,pp.886-893vol.1,doi:10.1109 / CVPR.2005.177,proposed a pedestrian detection method based on histogram of oriented gradients (HOG) and support vector machine (SVM) classifier,and achieved a detection effect close to 100% on the MIT pedestrian dataset. Krizhevsky[3]Krizhevsky A,Sutskever I,Hinton G.ImageNet Classification with Deep Convolutional Neural Networks[J].Advances in neural information processing systems,2012,25(2).DOI:10.1145 / 3065386,proposed the AlexNet model based on deep convolutional neural network,which has performance far superior to traditional machine learning algorithms.Girshick[4]R.Girshick,J.Donahue,T.Darrell and J.Malik,"Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation,"2014IEEE Conference on Computer Vision and Pattern Recognition,Columbus,OH,USA,2014,pp.580-587,doi:10.1109 / CVPR.2014.81, in 2014, a sliding window was used to extract pedestrian proposals, and a convolutional neural network (CNN) was used to extract the deep features of pedestrians in the proposals. Although the use of CNN for feature extraction greatly improved the detection effect, it also encountered many problems, such as cumbersome training process and long detection time. Redmon et al. proposed the first single-stage detector YOLO [5] J. Redmon, S. Divvala, R. Girshick et al., "You only look once: unified, real-time object detection," in Proceedings Of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788, Las Vegas, NV, USA, 2016, which applies a single neural network to the entire image and divides the image into multiple regions. This mode greatly improves the detection speed while simultaneously predicting the bounding box and the probability of each region. The above methods are not ideal in terms of accuracy and real-time performance when the pedestrian's body is blocked by other objects, the pedestrian suddenly jumps out from behind the occlusion, such as "ghosting", and the pedestrian becomes smaller when the target is far away. The present invention proposes a pedestrian detection method based on an improved FCOS algorithm, which can effectively improve the pedestrian detection effect in the above situations. Summary of the invention

[0004] The object of the present invention is to provide a real-time pedestrian detection method based on an improved FCOS algorithm, which can effectively realize real-time pedestrian detection with many occlusions and small targets.

[0005] The inventive idea of ​​the present invention is: the present invention designs a pedestrian detection method based on an improved FCOS algorithm; firstly, the FCOS algorithm is improved, and data enhancement is used for the data set to simulate the scene where the pedestrian's body is blocked and indirectly add small pedestrian targets to enrich the data set; secondly, a lightweight backbone network is introduced to replace the original feature extraction network lightweight network model, reduce network parameters, and adjust the feature pyramid and head network to further reduce network parameters; finally, a lightweight attention network is used and the backbone network feature fusion is adjusted to effectively deal with the influence of occlusion and small targets on detection.

[0006] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is specifically: a real-time pedestrian detection method based on an improved FCOS algorithm, comprising the following steps:

[0007] S1: Obtain pedestrian detection video, extract fixed frames, and pre-process and convert them into images.

[0008] S2: Perform data enhancement on the preprocessed images, simulate occlusion and add small target scene images to expand the dataset;

[0009] S3: Divide the expanded dataset into training set, validation set and test set;

[0010] S4: Construct FCOS network model;

[0011] S5: Replace the main feature extraction network Resnet50 of the FCOS network model with the lightweight backbone network Mobilenetv3 to lightweight the model structure;

[0012] S6: According to the structure of Movilenetv3, the depthwise separable convolution (DW) convolution is used to replace the ordinary convolution in the feature pyramid FPN and Head to further reduce the number of parameters;

[0013] S7: Pedestrians are generally small and medium-sized targets, and their details are mainly retained in the low-level feature layers. In the backbone feature extraction network, an additional C2 is added on the basis of C3 as an effective feature layer, and the P2 layer is formed through convolution to perform multi-scale feature fusion in the FPN structure;

[0014] S8: Based on the idea of ​​attention mechanism, a lightweight ECA attention mechanism is added to the improved FCOS network to obtain the fused FCOS model, and the training set is input into the improved model for training.

[0015] S9: Input the validation set into the improved FCOS network model for verification, obtain the optimal weights for model training through the validation set, test the test set, obtain the prediction results of the model, and finally obtain effective and real-time detection of pedestrians.

[0016] As a further optimization scheme of a real-time pedestrian detection method based on an improved FCOS algorithm provided by the present invention, in the step S1, a public data set Caltech video data set is used for preprocessing and conversion into an image. The Caltech data set is a video shot by a vehicle-mounted camera with a resolution of 640x480, 30 frames / second, annotated for about 137 minutes, with 350,000 annotation boxes and 2,300 pedestrians. There are 11 folders in total. The seq file is converted into a jpg file through a python script and placed in the same directory folder JPERImages. Call the script file vbb2voc to convert the vbb format into an xml file and place it in the same folder Annotations. The xml file contains the image name, image path, image tag name, and target location coordinates. There is no person target in most images. The select file is used to filter the jpg image according to the xml file so that the jpg file and the xml file correspond one to one, and the jpg containing the xml file is saved in the same directory. Call the rename file to rename the xml file and the jpg file to keep them consistent for subsequent operations. Since there are too many images generated, the content is repetitive and single, and the training volume is too large, the delete file is called to sample the images and annotations every 8 frames, reduce the training images, rearrange the names, and finally complete the data set conversion.

[0017] As a further optimization scheme of a real-time pedestrian detection method based on an improved FCOS algorithm provided by the present invention, in the step S2, data enhancement is performed on the preprocessed image, and two types of data enhancement are adopted. One is to fuse each pixel of the input image according to a certain ratio, and the output result is also proportionally fused; a few samples are randomly selected from the training samples for random weighted fusion, and the labels of the samples are also weightedly fused, and then the prediction result and the label after weighted summation are calculated; the following formula is used: Backward derivation updates parameters. Expand the data set in the pedestrian data set to simulate pedestrian occlusion scenes to increase the diversity of the training set. Another method is to randomly select four pictures and splice them by randomly scaling, randomly cropping, and randomly arranging them in a certain proportion, which is equivalent to adding many small target scenes. At the same time, directly calculating 4 pictures at a time indirectly expands the batch size and reduces the GPU memory requirements.

[0018] As a further optimization scheme of the real-time pedestrian detection method based on the improved FCOS algorithm provided by the present invention, in step S3, the data set is divided into a training set, a validation set and a test set; the ratio of the created data set to the training set and the test set is specified to be 9:1, the data set is randomly divided, and then the training set is randomly divided into the training set and the validation set according to the ratio of 9:1. At the same time, the labels of the data set are processed to generate 2007_train.txt and 2007_val.txt files for training, and the contents of the txt files include the absolute path of the picture, the number of the picture, and the specific coordinates of the target label.

[0019] As a further optimization scheme of the real-time pedestrian detection method based on the improved FCOS algorithm provided by the present invention, in step S4, a FCOS network model is constructed. The structure of the FCOS network model includes:

[0020] Backbone feature extraction network is used to extract features from images. The extracted effective features are used as feature layers. The model selects three effective feature layers C3, C4, and C5.

[0021] The enhanced feature network FPN is used to fuse the effective feature layers of different sizes to adapt to targets of different sizes. The model forms the P3, P4, P5, P6, and P7 feature layers through up and down convolution sampling;

[0022] The prediction network Head is used to classify and regress the final features and predict the final results. The head includes classification branch, regression branch and centrality branch.

[0023] As a further optimization scheme of a real-time pedestrian detection method based on an improved FCOS algorithm provided by the present invention, in step S5, a lightweight backbone network Mobilenetv3 network is used to replace the Resnet50 backbone feature extraction network. The Mobilenetv3 network is introduced to create a specific instance. For the specific instance of Mobilenetv3, feature maps with 40, 112, and 160 channels are taken out through slicing operations as effective feature layers and returned to the model as the input of the model. The characteristics of Mobilenetv3's depthwise separable convolution, residual structure with linear bottleneck, algorithm to obtain the optimal number of convolution kernels and channels, introduction of channel attention structure, and use of the new h-swish activation function are utilized to reduce network parameters without losing model accuracy. Changes are made according to the height, width, and number of channels of the original three effective feature layers. The changed height, width, and number of channels are:

[0024]

[0025] As a further optimization scheme of a real-time pedestrian detection method based on an improved FCOS algorithm provided by the present invention, in the step S6, a depth-separable convolution (DW) convolution is used to replace the ordinary convolution in the feature pyramid FPN and Head parts. The depth-separable convolution first performs channel-by-channel convolution and then point-by-point convolution. Through this operation, the amount of parameters can be reduced without changing the effect. The amount of parameters is 1 / 3 of the ordinary convolution, and can replace the 3x3 ordinary convolution. Based on this, the conv_dw function of the present invention is used to find the ordinary convolution with a kernel_size convolution kernel size of 3 in the FPN and Head structures of the improved FCOS model for replacement, thereby compressing the parameter amount of the entire model and lightweighting the model.

[0026] As a further optimization scheme of a real-time pedestrian detection method based on an improved FCOS algorithm provided by the present invention, in the step S7, an additional C2 is added as an effective feature layer based on C3, and a P2 layer is formed through convolution to perform multi-scale feature fusion in the FPN structure. In the backbone feature extraction network Mobilentv3, feature layers with 104, 104, and 24 high and wide channels are taken out as C2 through an additional slicing operation, and C2 is passed through a convolution layer with a convolution kernel kernel_size of 1x1 and a step size of 1 to obtain P2, and then P2 is upsampled through an upsamplelike function, and finally a convolution with a convolution kernel size of 3, a step size of 1, and a padding of 1 is formed to form a feature layer P2 as the bottom feature layer that retains the details of small targets, thereby improving the detection effect of small targets, and finally sharing the Head with other layers for result prediction.

[0027] As a further optimization scheme of a real-time pedestrian detection method based on an improved FCOS algorithm provided by the present invention, in S8, a lightweight ECA attention mechanism is added to the improved FCOS network. ECA can be regarded as an improved version of the SE module. The input feature map is subjected to global average pooling, and the feature map using the 1x1 convolution layer in the fully connected layer is removed to change from a matrix of [h, w, c] to a vector of [1, 1, c]. Then, the adaptive one-dimensional convolution kernel size kernel_size is calculated according to the number of channels of the feature map, and kernel_size is used in the one-dimensional convolution to obtain a weight for each channel of the feature map. Finally, the normalized weights are multiplied channel by channel with the original input feature map to generate a weighted feature map. It avoids dimensionality reduction, effectively captures cross-channel interactions, and only involves a small number of parameters to achieve good results. The training set is input into the improved model for training, and the model is allowed to concentrate on extracting the detailed features of the occluded areas of pedestrians through ECA.

[0028] As a further optimization scheme of a real-time pedestrian detection method based on an improved FCOS algorithm provided by the present invention, in step S9, the optimal weight of model training is obtained through the validation set, and the test set is tested to obtain the prediction result of the model. The maximum number of iterations is set to 300, the model freeze training Freeze_Eopch is set to 50, the batch_size is set to 8, the initial learning rate is 0.01, the learning rate is adaptively adjusted, and the weight is saved every 5 epochs. After 300 iterative trainings, the loss and accuracy of the model gradually stabilize, and the best weight file is saved according to the verification error of the validation set. The model evaluation indicators of the model test include Average Precision, Recall, and Precision.

[0029] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention proposes a real-time pedestrian detection method based on an improved FCOS algorithm, which uses a lightweight backbone network as a feature extraction network, lightweights the model without reducing the accuracy, compresses the parameters of the network model, and adjusts the feature pyramid and the head network to further compress the network parameters, so that the entire model can be transplanted to a lightweight device to run, and realize real-time detection of pedestrians. The introduction of a lightweight ECA attention mechanism and feature fusion adjustment of the effective feature layer of the backbone makes the network more robust, can effectively handle the situation of occlusion and small targets, and realize accurate and real-time detection of pedestrians; the present invention can effectively improve the accuracy and real-time performance of pedestrian detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0031] Figure 1 This is a detailed diagram of the entire model after the backbone network is modified and the small target underlying feature layer is added in the present invention.

[0032] Figure 2 This is a schematic diagram of the ECA attention residual module in the present invention.

[0033] Figure 3 This is a diagram of the final fused model framework after fusing lightweight attention ECA in the present invention.

[0034] Figure 4 The overall flow chart after the entire model is constructed in the present invention.

[0035] Figure 5 This is a prediction diagram of the effect of real-time detection of occluded targets after the model is improved.

[0036] Figure 6This is a prediction diagram of the effect of real-time detection of small targets after the model is improved. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. Of course, the specific embodiments described here are only used to explain the present invention and are not used to limit the present invention.

[0038] Example 1

[0039] See also Figures 1 to 6 The technical solution provided in this embodiment is a real-time pedestrian detection method based on an improved FCOS algorithm. The overall process of the method is shown in the attached Figure 1 As shown: It mainly includes the following steps:

[0040] S1: Get real-time pedestrian detection video from the dashcam, with a video resolution of 1920×1080, 30 frames / second, and collect images once per second, and preprocess the data set into images. Read a frame of the video, call cv2.cvtColor() in the Open CV function library to convert the format to RGB, and then convert it into an Image image that meets the opencv display format for detection.

[0041] S2: Perform data enhancement on the preprocessed image, simulate occlusion and add small target scene images. This step specifically includes the following steps:

[0042] S2.1: Convert all images into RGB images to prevent grayscale images from reporting errors during prediction. Add gray bars to the images to achieve distortion-free resizing. Scale the images to 640×640. This step is implemented by calling the function resize_image().

[0043] S2.2: Add batch_size dimension to the original image and normalize the preprocessed image. This step is implemented by calling preprocess_input();

[0044] S2.3: Simulate the scene of obstructed pedestrians. Randomly select samples from the training samples and perform weighted summation. The xml tag files of the selected samples are also weighted summation, and a new image is generated by superimposing and mixing them at the pixel level in a certain proportion. In the specific implementation, random data enhancement is performed during training, but not during verification. The number of groups of generated pictures is set to Out_Num, and the size of the generated pictures is unified to 640x640. The labels of the original data set are traversed and assigned in turn. XML and jpg are generated in a loop according to the size of the output group number Out_Num. Finally, the enhanced images are merged. This step is implemented using the get_random_data_with_MixUp algorithm.

[0045] S2.4: Add small target scenes, randomly select 4 pictures from the training samples, and splice them by random scaling, random cropping, and random arrangement in a certain proportion. This indirectly adds small targets and is equivalent to reading 4 pictures at a time, increasing batch_size. In the specific implementation, use the get_random_data_with_Mosaic algorithm to generate the number of picture groups according to the defined Out_Num, traverse the labels and assign them, obtain 4 images and labels according to the number of groups, and generate xml and jpg in a loop, and finally merge the mosaic enhanced image.

[0046] S3: Divide the enhanced data into training set, validation set and test set. Call the voc_annotation file, set the ratio of training set to test set to 9:1 for the created data set, set trainval_percent to 0.9 to specify the ratio of training set + validation set to test set, set train_percent to 0.9 to specify the ratio of training set to validation set in the training set plus validation set, randomly divide the data set, randomly divide the training set into training set and validation set according to the ratio of 9:1, and process the labels of the data set to generate 2007_train.txt and 2007_val.txt files for training. The txt file content contains the absolute path of the image, the image number, and the specific coordinates of the target label.

[0047] S4: Build the FCOS network model.

[0048] S5: Replace the backbone feature extraction network Resnet50 of the FCOS network model with the lightweight backbone network Mobilenetv3 to reduce the model structure; the specific operations are as follows: introduce the Mobilenetv3 network, extract the feature maps with 40, 112, and 160 channels through slicing operations as effective feature layers and return them to the model as the input of the model; change the height, width, and number of channels of the original three effective feature layers. The changed height, width, and number of channels are:

[0049]

[0050] S6: According to the structure in Movilenetv3, the depth-separable convolution (DW) convolution is used to replace the ordinary convolution in the feature pyramid FPN and Head to further reduce the number of parameters; define the conv_dw(filter_in, filter_out, stride=1) function, find the ordinary convolution layer with a convolution kernel kernel_size of 3x3 in the FPN and Head structures of the network model to replace it, thereby further reducing the number of parameters.

[0051] S7: Pedestrians are generally small and medium-sized targets, and their details are mainly retained in the low-level feature layers. In the backbone feature extraction network, an additional C2 is added on the basis of C3 as an effective feature layer. After convolution, the P2 layer is formed to perform multi-scale feature fusion in the FPN structure.

[0052] S7.1: In the backbone feature extraction network Mobilentv3, the feature layers with height and width channels of 104, 104, and 24 are extracted through slicing operations as the bottom effective feature layer C2, which mainly retains the details of small targets.

[0053] S7.2: C2 passes through a convolution layer with a kernel size of 1x1 and a stride of 1 to obtain P2, and then P2 is upsampled by the upsamplelike function, and finally passes through a convolution layer with a kernel size of 3, a stride of 1, and a padding of 1 to form a feature layer P2 as the bottom feature layer that retains the details of small targets, improves the detection effect of small targets, and finally shares the Head with other layers for result prediction. The results of this step are shown in the attached figure. Figure 1 shown.

[0054] S8: Based on the idea of ​​attention mechanism, a lightweight ECA attention mechanism is added to the improved FCOS network to obtain the fused FCOS model. The training set is input into the improved model for training, which specifically includes the following steps:

[0055] S8.1: Introduce a lightweight ECA attention method to pool the input feature map through global average pooling

[0056] AdaptiveAvgPool2d(1), which transforms from a matrix of [h,w,c] to a vector of [1,1,c], is implemented by a fast one-dimensional convolution of size k, where the kernel size k represents the coverage of local cross-channel interactions, that is, how many areas participate in the attention prediction of a channel. In order to avoid manually adjusting k through cross-validation, an adaptive method is developed to determine k, where the coverage of cross-channel interactions (i.e., kernel size k) is proportional to the channel dimension. The adaptive function for determining k is:

[0057] Where γ=2,b=1

[0058] The adaptively obtained k is applied to the one-dimensional convolution to obtain a weight for each channel of the feature map. Finally, the weight is normalized by sigmoid and multiplied by the original input feature map channel by channel to obtain the weighted feature map. Figure 2 shown.

[0059] S8.2: First, use the defined eca_block class to create the attention instances of the effective feature layers C2, C3, C4, and C5 of the backbone network. Then, create the feature layers P2, P3, P4, P5 and upsampled unsample_attention instances after feature fusion. Finally, define the classification branch attention and regression branch attention mechanisms of Fcos_Head. Apply the attention module to the different effective feature layers extracted by the backbone network, the upsampling and downsampling areas of the feature pyramid feature fusion, and the various branches of the head. Finally, input the training set into the fused and improved model for training, input the weight parameters of the extracted features into the improved model, and the improved model learns the extracted features to obtain the final fused feature map, as shown in the formula:

[0060]

[0061] in, are the parameters of the weights of different layers, It is the output of different feature layers. Through the fusion of semantic information of different feature layers, including the fusion of underlying detail contours, edges, colors and shapes, features are extracted using different scales, and finally the head branch is used for classification regression prediction. The results of this step are shown in the attached figure. Figure 3 shown.

[0062] S9: Input the validation set into the improved FCOS network model for validation. The optimal weight of model training is obtained through the validation set. The test set is tested to obtain the prediction results of the model. Set the maximum number of iterations to 300, set the model freeze training Freeze_Eopch to 50, set the batch_size to 8, and the initial learning rate to 0.01. The classification loss Lcls uses the binary cross entropy loss with focal_loss bce_focal_loss, the positioning loss Lreg uses giou_loss, and the center loss uses the binary cross entropy loss. The optimizer options are Adam and SGD. Adam is selected here, and the learning rate is adjusted adaptively. The learning rate reduction method uses cos. The weight is saved every 5 epochs. The model is evaluated with the validation set during training and evaluated every 5 epochs. After 300 iterative training, the loss and accuracy of the model gradually stabilize. Save the best weight file according to the validation error of the validation set. The model evaluation indicators of the model test include average precision AveragePrecision, recall rate Recall, and precision rate Precision; the whole process is as shown in the attached Figure 4 shown.

[0063] S10: The real-time pedestrian detection method based on the improved FCOS algorithm proposed in this embodiment can accurately and real-time detect pedestrian targets, has strong robustness and generalization ability for complex scenes with occlusion and small targets, and can accurately identify the specific location of pedestrians, as shown in the attached figure. Figure 5 , Attachment Figure 6 As shown. According to the evaluation indicators of the model, the average accuracy can reach 95.81%, the recall rate can reach 91.52%, and the precision rate can reach 95.03%. In terms of real-time performance, this embodiment reaches 54.65fps, which meets the real-time detection requirements of autonomous driving. The total parameter size is 10.663M, which is lighter and easier to transplant than the original model.

[0064] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A real-time pedestrian detection method based on an improved FCOS algorithm, characterized in that: The following steps are involved: S1: Obtain pedestrian detection video, extract fixed frames, and convert them into images for preprocessing; S2: Perform data enhancement on the preprocessed images, simulate occlusion and add small target scene images to expand the dataset; S3: Divide the expanded dataset into training set, validation set and test set; S4: Construct FCOS network model; S5: Replace the main feature extraction network Resnet50 of the FCOS network model with the lightweight backbone network Mobilenetv3 to lightweight the model structure; S6: According to the structure of Mobilenetv3, the 3x3 ordinary convolution is replaced by the depthwise separable convolution in the feature pyramid FPN and Head part to further compress the model parameters; S7: Pedestrians are small and medium-sized targets, and their details are retained in the low-level feature layer. In the backbone feature extraction network, an additional C2 is added on the basis of C3 as an effective feature layer, and the P2 layer is formed through convolution to perform multi-scale feature fusion in the FPN structure; S8: Based on the idea of ​​attention mechanism, a lightweight ECA attention mechanism is added to the improved FCOS network to obtain the fused FCOS model, and the training set is input into the improved model for training; S9: Input the validation set into the improved FCOS network model for verification, obtain the optimal weights for model training through the validation set, test the test set, obtain the prediction results of the model, and finally obtain effective and real-time detection of pedestrians.

2. The real-time pedestrian detection method based on the improved FCOS algorithm according to claim 1 is characterized in that: In step S1, a pedestrian detection video data set is obtained, and fixed frames are extracted for preprocessing and converted into images, which specifically includes the following steps: S11: Use the public dataset Caltech video dataset. The Caltech dataset is a video shot by a car camera, which is preprocessed and converted into images. It contains the label folder Annotations, the picture folder JPEGImages, and the data partition folder ImageSets. S12: Use the img script to convert the seq sequence file into a jpg file and put it in the same directory folder JPERImages to save the file converted from the sequence frame; S13: Use the vbb2voc function to convert the label file in vbb format into an xml file and place it in the same folder Annotations. The xml file contains the image name, image path, image label name, and target location coordinates; S14: There is no person target in the picture. Use the select file to filter the jpg images according to the xml file so that the jpg files and the xml files correspond one to one. Save the jpg containing the xml file in the same directory and rename it to keep it consistent for subsequent operations. Because there are too many pictures generated, the content is repeated and single, and the training volume is too large, call the delete file to select every 8 frames to sample the labeled pictures, rearrange the names, and complete the data set conversion.

3. The real-time pedestrian detection method based on the improved FCOS algorithm according to claim 1 is characterized in that: The step S2 includes the following steps: S21: Fuse each pixel of the input image in proportion, and also perform proportional fusion on the output result. Randomly select several samples from the training samples for random weighted fusion. Perform weighted fusion on the labels of the samples, and then calculate the loss between the predicted result and the weighted summed label to simulate the scene where pedestrians are blocked. S22: randomly select pictures, and splice the selected pictures by scaling, randomly cropping, and randomly arranging them.

4. The real-time pedestrian detection method based on the improved FCOS algorithm according to claim 1 is characterized in that: In step S3, the voc_annotation function is used to specify the ratio of the training set to the test set of the created data set as 9:1, trainval_percent is set to 0.9 to specify the ratio of the training set + validation set to the test set, and train_percent is set to 0.9 to specify the ratio of the training set to the validation set in the training set plus the validation set. The data set is randomly divided into training sets and validation sets in a ratio of 9:

1.

5. The real-time pedestrian detection method based on the improved FCOS algorithm according to claim 1 is characterized in that: The step S4 includes the following steps: S41, backbone feature extraction network Backbone, used to extract features from images, using the extracted effective features as feature layers, and the model selects feature layers C3, C4, and C5; S42, enhanced feature network FPN, is used to fuse the effective feature layers of different sizes to adapt to targets of different sizes. The model forms P3, P4, P5, P6, and P7 feature layers through up and down convolution sampling; S43, prediction network Head, is used to classify and regress the final features and predict the final result. The head includes classification branch, regression branch and centrality branch.

6. The real-time pedestrian detection method based on the improved FCOS algorithm according to claim 1 is characterized in that: The step S5 includes the following steps: S51: Introduce the Mobilenetv3 network, extract the feature maps with 40, 112, and 160 channels through slicing operations as the effective feature layers and return them to the model as the input of the model; S52: Utilize the depthwise separable convolution of Mobilenetv3, the residual structure of the linear bottleneck, the algorithm to obtain the optimal number of convolution kernels and channels, introduce the channel attention structure, and use the new h-swish activation function to reduce network parameters without losing model accuracy.

7. The real-time pedestrian detection method based on the improved FCOS algorithm according to claim 1 is characterized in that: In step S6, the 3x3 ordinary convolution is replaced by the depthwise separable convolution (DW) convolution in the feature pyramid FPN and Head part, which specifically includes the following steps: S61: First perform channel-by-channel convolution, and then perform point-by-point convolution. This operation reduces the number of parameters without changing the effect. The number of parameters is 1 / 3 of the ordinary convolution, replacing the 3x3 ordinary convolution. S62: Use the conv_dw function to replace the ordinary convolution with kernel_size convolution kernel size of 3 in the FPN and Head structures of the improved FCOS model, compress the parameters of the entire model, and lightweight the model.

8. The real-time pedestrian detection method based on the improved FCOS algorithm according to claim 1 is characterized in that: In the step S7, an additional C2 is added on the basis of C3 as an effective feature layer, and the P2 layer is formed through convolution to perform multi-scale feature fusion in the FPN structure. In the backbone feature extraction network Mobilentv3, an additional slicing operation is performed to extract the feature layers with height and width channels of 104, 104, and 24 as C2. C2 is passed through a convolution kernel to obtain P2, and then P2 is upsampled. Finally, a convolution with a convolution kernel size of 3, a step size of 1, and a padding of 1 is passed through a convolution to form a feature layer P2 as the underlying feature layer for retaining small target details.

9. The real-time pedestrian detection method based on the improved FCOS algorithm according to claim 1 is characterized in that: In step S8, a lightweight ECA attention mechanism is added to the improved FCOS network, including the following steps: S81: The input feature map is subjected to global average pooling, and the fully connected layer is removed. The feature map using the 1x1 convolution layer is changed from a matrix of [h,w,c] to a vector of [1,1,c]. Then, the adaptive one-dimensional convolution kernel size kernel_size is calculated according to the number of channels of the feature map. The kernel_size is used in the one-dimensional convolution to obtain a weight for each channel of the feature map. Finally, the normalized weight is multiplied by the original input feature map channel by channel to generate a weighted feature map. S82: Input the training set into the improved model for training, and let the model concentrate on extracting the detailed features of the pedestrian occluded area through ECA.

10. The real-time pedestrian detection method based on the improved FCOS algorithm according to claim 1 is characterized in that: In step S9, the optimal weight of model training is obtained through the validation set, and the test set is tested to obtain the prediction result of the model, which specifically includes the following steps: S91: Set the maximum number of iterations to 300, Freeze_Eopch to 50, batch_size to 8, initial learning rate to 0.01, adaptively adjust learning rate, save weights every 5 epochs, and after 300 iterations of training, the loss and accuracy of the model gradually stabilize. Save the best weight file based on the verification error of the verification set. S92: The model evaluation indicators of model testing include average accuracy, recall rate, and precision rate.

Images