Dense pedestrian detection method and device based on YOLOv7 and computer equipment

Abstract

The invention discloses a dense pedestrian detection method and device based on YOLOv7 and computer equipment. The method comprises the following steps: acquiring pedestrian image data in a complex and dense scene, and preprocessing the acquired image data; inputting the preprocessed image data into a pre-trained dense pedestrian detection model; the dense pedestrian detection model adopts a YOLOv7 network as a basic model and comprises a backbone network and a head network; the backbone network comprises a convolution module and an FNS-ELAN module; the head network comprises a feature enhancement module RFEM and a network structure fused with multi-scale features; and based on the feature map calculated by the model, performing final pedestrian detection and positioning by using an output layer, and generating a detection result containing a detection frame, a category and confidence. According to the method, the precision, the speed and the robustness of dense pedestrian detection are remarkably improved, and the application requirements of relatively high requirements on real-time performance and accuracy in a complex scene are met.

Description

Technical Field

[0001] The present invention relates to the field of computer vision application technology, and more specifically to a dense pedestrian detection method, device and computer equipment based on YOLOv7. Background Art

[0002] At present, pedestrian detection is an important research direction in the field of computer vision, and has important research value in promoting the development of intelligent monitoring, autonomous driving, robot navigation and other fields. However, dense scenes refer to situations where the distance between pedestrians is small, they frequently occlude each other, and the background is complex and changeable. In such an environment, pedestrians are densely distributed and their movement patterns are changeable, which makes the feature information complex. At the same time, due to the mutual occlusion between pedestrians and the mixing with the complex background, the interference of irrelevant information in the detection is aggravated, making the pedestrian detection task face greater challenges. At present, pedestrian detection technology faces the following problems:

[0003] ① The detected pedestrian targets are crowded and blocked with each other, and there are few features in the visible area, which makes it difficult to extract network features;

[0004] ② In densely populated scenes, pedestrian detection is subject to background interference, and the network model is difficult to correctly classify and locate pedestrians, resulting in missed detections and false detections.

[0005] At present, the existing practices for the above problems are:

[0006] 1. To address the problem of difficult feature extraction, on the one hand, the accuracy of target detection is improved by accurately locating various parts of the human body through the aggregation loss function, but the impact of density on target detection is not fully considered. On the other hand, the non-maximum suppression threshold is adaptively adjusted by predicting the density of different areas to retain low-confidence candidate boxes, which significantly improves the model performance. In addition, multiple camera perspectives are considered to alleviate the impact of occlusion on pedestrian detection results, but the improvement in detection accuracy is limited.

[0007] 2. To address the problem of false detection and missed detection caused by background interference, existing technologies provide more sufficient feature information for target detection by designing a scale-aware hierarchical detection network or replacing a new backbone network with a pre-trained model. In addition, there is also a variational Bayesian optimization algorithm that uses automatic encoding to model the density in a probabilistic way to improve detector performance. However, these improved methods tend to ignore some shallow feature information, resulting in increased computational effort and limited effect in crowded pedestrian scenes, so performance still needs to be further improved. Summary of the invention

[0008] In view of this, the present invention provides a dense pedestrian detection method, device and computer equipment based on YOLOv7, which can solve the problems faced by the above-mentioned pedestrian detection, such as crowded and occluded pedestrian targets, difficulty in feature extraction and false detection and missed detection caused by background interference.

[0009] In order to achieve the above object, the present invention adopts the following technical solution:

[0010] In a first aspect, an embodiment of the present invention provides a dense pedestrian detection method based on YOLOv7, comprising the following steps:

[0011] S10, collecting pedestrian image data in complex and dense scenes, and preprocessing the acquired image data;

[0012] S20, inputting the preprocessed image data into a pre-trained dense pedestrian detection model; the dense pedestrian detection model uses the YOLOv7 network as a basic model, including: a backbone network and a head network; the backbone network includes: a convolution module and a FNS-ELAN module; the head network includes: a feature enhancement module RFEM and a network structure that integrates multi-scale features;

[0013] S30. Based on the feature map calculated by the model, the output layer is used to perform final pedestrian detection and positioning, and a detection result including a detection box, category and confidence is generated.

[0014] Furthermore, in the backbone network:

[0015] The convolution module, which consists of four consecutive convolutional layers, is used for preliminary feature extraction;

[0016] The FNS-ELAN module, including a combination of various convolutional layers, enhances the differential amplification effect between the fine-grained features of the shallow and deep layers through skip connections and weighted operations, and generates a feature map that amplifies the differences.

[0017] Furthermore, the FNS-ELAN module includes three parts: an efficient structure for controlling long and short paths, a feature differentiation amplification module, and a SimAM attention module;

[0018] Among them, three types of convolutional layers are set on the efficient structure of controlling long and short paths, 1×1 and 3×3 ordinary convolutions, 7×7 depth convolutions and depth-separable convolutions;

[0019] The output of the third convolutional layer is fed into the feature differentiation amplification module, which weights the shallow features with the deep features through skip connections, thereby amplifying the small differences between the target features.

[0020] The output of the feature differentiation amplification module is concatenated with the output results of other different convolutional layers, and the output features are sent to the SimAM attention module;

[0021] The energy function is used to generate the three-dimensional weight of each feature, and the energy value is used to judge the importance of the feature map, so that the visible area is given a higher weight value;

[0022] The sigmoid function is used to enhance the output features, and finally the Hadamard product is calculated using the input features and the attention weights to obtain the enhanced attention feature map.

[0023] Furthermore, the data processing flow of the FNS-ELAN module is as follows:

[0024] The feature map extracted by four consecutive convolutional layers is used as the input F of the FNS-ELAN module in ;

[0025] After three convolutional layers, the output F is obtained 1 , as the input of the feature differentiation amplification module;

[0026] After the feature differentiation amplification module, the output F is obtained 2 , F 2 The output after convolution is concatenated with the output of other convolutional layers to obtain the output feature map F 3 , and then enter the SimAM attention module, redistribute the weights according to the importance of the feature map to obtain the output F 4 , after the last layer of convolution, the backbone network output F is obtained out , the whole calculation process is shown in formulas (1 to 5):

[0027] F 1 =Conv 3 (Conv 3 (Conv 1 (F in ))) (1)

[0028] F 2 =FDA(F 1 ) (2)

[0029]

[0030] F 4 =SimAM(F 3 ) (4)

[0031] F out =Conv 1 (F 4 ) (5)

[0032] Among them, FDA stands for Feature Differentiation Amplification Module; Convj (·) represents the convolution operation, where j represents the convolution kernel size; SimAM represents the SimAM attention module.

[0033] Furthermore, the data processing flow of the feature differentiation amplification module is as follows:

[0034] (1)F 1 As the input of the feature differentiation amplification module, the calculation after the 7×7 depth convolution layer and the depth separable convolution layer is shown in formula (6):

[0035]

[0036] Among them, DLG means after a layer of deep convolution, layer normalization and GiLU activation function; D s LG means after a layer of depth-wise separable convolution, layer normalization and GiLU activation function; represents weighted sum;

[0037] (2) Output F a After two 1×1 ordinary convolutions, the input F 1 Perform weighted processing to obtain the output feature map F 2 , the calculation process is shown in formula (7):

[0038]

[0039] Among them, σ represents the GiLU activation function.

[0040] Furthermore, the SimAM attention module outputs the feature map F of the efficient structure for controlling the long and short paths. 3 As input, the mean and variance of its tensor are calculated in the spatial dimension, and then the minimum energy value is calculated. The activation value is normalized by the sigmoid function, and finally the attention weight is multiplied by the input tensor to obtain the output feature map F 4 , the entire calculation process is shown in formulas (8~11):

[0041] μ=mean(F 3 ) (8)

[0042]

[0043] Among them, μ means calculating the mean in the spatial dimension, mean means calculating the mean function in the spatial dimension; σ 2 represents variance; i represents the channel index; N = H × W represents the total number of neurons on the channel; e tRepresents the minimum energy value, where the lower the energy value, the greater the difference from the surrounding and the higher the importance; sigmoid represents the activation function, where λ represents the added regular term.

[0044] Furthermore, the head network receives output features from FNS-ELAN modules at different layers, adjusts the channels of different output features through 1×1 convolution, and then concatenates the features processed by the feature enhancement module RFEM with the input features; secondly, a cross-layer connection is added to concatenate the second input features after adjusting the number of channels in a top-down path to retain more contextual information.

[0045] Furthermore, the data processing flow of the feature enhancement module RFEM is as follows:

[0046] The input feature map is compressed through a convolutional layer;

[0047] The upsampling kernel size of each position is learned through the encoding layer and normalized using the Softmax function;

[0048] The upsampled feature map is expanded into a feature sequence, and the upsampled kernel is point-multiplied with the expanded feature sequence to obtain a final feature map with higher spatial resolution.

[0049] In a second aspect, an embodiment of the present invention further provides a dense pedestrian detection device based on YOLOv7, comprising:

[0050] The acquisition module is used to collect pedestrian image data in complex and dense scenes and pre-process the acquired image data;

[0051] An input module is used to input the preprocessed image data into a pre-trained dense pedestrian detection model; the dense pedestrian detection model uses the YOLOv7 network as a basic model, including: a backbone network and a head network; the backbone network includes: a convolution module and a FNS-ELAN module; the head network includes: a feature enhancement module RFEM and a network structure that integrates multi-scale features;

[0052] The detection module is used to perform final pedestrian detection and positioning based on the feature map calculated by the model, and generate detection results including detection boxes, categories and confidence levels using the output layer.

[0053] In a third aspect, an embodiment of the present invention further provides a computer device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0054] Memory, used to store computer programs;

[0055] The processor, when used to execute the program stored in the memory, can implement a dense pedestrian detection method based on YOLOv7 as described in any one of the first aspects.

[0056] The description of the second to third aspects of the present invention can refer to the detailed description of the first aspect; and the beneficial effects of the description of the second to third aspects can refer to the analysis of the beneficial effects of the first aspect, which will not be repeated here.

[0057] It can be seen from the above technical solution that compared with the prior art, it has the following technical advantages:

[0058] The present invention adopts the YOLOv7 network as the basic model. Aiming at the problem that pedestrians block each other in crowded and complex scenes, which makes it difficult to extract features, a backbone feature extraction network with enhanced feature differentiation is constructed. The FNS-ELAN module is used to make the model produce stronger fine-grained features, and then the weights are redistributed according to the importance of the features, thereby amplifying the differences between the features and enhancing the model's attention to the visible area; secondly, in view of the problem that the detection accuracy rate decreases due to background interference, a network structure and feature enhancement module that fuses multi-scale features are designed, so that the input features first pass through the feature enhancement module to obtain more context information, and then the learnable weight factors in the BIFPN network are used to select more feature information layers by themselves. By using cross-layer connections, the model can extract and fuse information at multiple scales, thereby improving the accuracy of the model for targets of different scales. Therefore, the present invention can be effectively applied to dense pedestrian detection tasks and has better effects. The method significantly improves the accuracy, speed and robustness of dense pedestrian detection, and meets the application requirements with high real-time and accuracy requirements in complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0060] Figure 1 A flowchart of a dense pedestrian detection method based on YOLOv7 provided by the present invention;

[0061] Figure 2 A network structure diagram of a dense pedestrian detection model provided by the present invention;

[0062] Figure 3 The structural diagram of the FNS-ELAN module provided by the present invention;

[0063] Figure 4 A structural diagram of a feature differentiation amplification module provided by the present invention;

[0064] Figure 5 The RFEM structure diagram of the feature enhancement module provided by the present invention;

[0065] Figure 6 A block diagram of a dense pedestrian detection device based on YOLOv7 provided by the present invention;

[0066] Figure 7 This is a structural block diagram of the computer device provided by the present invention. DETAILED DESCRIPTION

[0067] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] Embodiment 1:

[0069] The embodiment of the present invention discloses a dense pedestrian detection method based on YOLOv7, referring to Figure 1 As shown, the following steps are included:

[0070] S10. Collect pedestrian image data in complex and dense scenes, and pre-process the acquired image data. In this step, for example, pedestrian image data in complex and dense scenes can be acquired through a camera or other image acquisition device. Dense scenes usually refer to situations where pedestrians are close to each other, there are occlusions or complex backgrounds. The acquired image data needs to be pre-processed, which includes image resizing, noise removal, color correction and other operations to ensure image quality and consistency, thereby improving the input effect of subsequent models.

[0071] S20. Input the preprocessed image data into a pre-trained dense pedestrian detection model; the dense pedestrian detection model uses the YOLOv7 network as a basic model, including: a backbone network and a head network; the backbone network includes: a convolution module and a FNS-ELAN module; the head network includes: a feature enhancement module RFEM and a network structure that integrates multi-scale features.

[0072] The preprocessed image data is fed into a pre-trained dense pedestrian detection model. The model is based on the YOLOv7 network and consists of two parts:

[0073] Backbone network: responsible for the extraction of preliminary features, including the basic convolution module and the FNS-ELAN module for enhancing fine-grained features. The main function of the backbone network is to extract representative features from the input image, especially to enhance and differentiate fine-grained features.

[0074] Head network: It is used for further processing and fusion of features. It includes the feature enhancement module RFEM and the network structure for fusion of multi-scale features. The head network is designed to fuse and enhance feature information at different scales to provide support for the final detection results.

[0075] In this step, after the input image enters the backbone network, it first passes through continuous convolution modules for preliminary feature extraction, and then uses the cross-FNS-ELAN module to generate stronger fine-grained features. Then, the weights are redistributed according to the importance of the features to generate feature maps with amplified differences. Feature maps of different scales are sent to the head network, where the output feature maps of the last layer and the third to last layer in the backbone network are processed by the feature enhancement module and then cross-layer connected to fuse the high-dimensional feature maps with the low-dimensional feature maps to retain the feature information contained in different scales to the maximum extent. Finally, the RepConv module is used to adjust the number of channels of features of different scales and then perform detection to obtain the final detection result.

[0076] S30. Based on the feature map calculated by the model, the output layer is used to perform final pedestrian detection and positioning, and a detection result including a detection box, category and confidence is generated.

[0077] In this step, the model performs reasoning and calculation based on the feature maps extracted and processed above, and finally generates pedestrian detection results in the output layer. The detection results usually include the location (detection box), category (such as pedestrian, cyclist, etc.) and confidence (the credibility of the detection result) of the pedestrian. With this information, the system can accurately identify and locate pedestrians in complex and dense scenes.

[0078] This method significantly improves the accuracy, speed and robustness of dense pedestrian detection, meeting the application requirements with high real-time and accuracy in complex scenes.

[0079] The following focuses on the construction of the dense pedestrian detection model and data processing in the above step S20 for detailed description.

[0080] like Figure 2 As shown in the figure, the network structure of the dense pedestrian detection model is adopted. The YOLOv7 network is used as the basic model. The construction process includes the following steps:

[0081] Step 1: In the feature extraction stage, a backbone feature extraction network is constructed to enhance feature differentiation and increase the model's attention to visible area features.

[0082] Step 2: Through the FNS-ELAN module in the backbone feature extraction network, the model generates stronger fine-grained features, and then redistributes the weights according to the importance of the features, thereby amplifying the differences between the features.

[0083] Step 3: In the feature fusion stage, a network structure and feature enhancement module that integrates multi-scale features are designed. The features extracted by the backbone network first pass through the feature enhancement module to obtain more contextual information, and then the learnable weight factors in the BIFPN network are used to select the layer that accepts more feature information.

[0084] Step 4: Add cross-layer connections so that the model can extract and fuse information at multiple scales, improving the network's ability to detect objects of different scales.

[0085] The data used for training the dense pedestrian detection model comes from the wider-person public pedestrian detection dataset, which contains 13,382 images and a total of 400,000 human targets with different degrees of occlusion, including five types of annotations: pedestrians, partially visible people, cyclists, crowds, and ignored areas. The training set is pre-divided into 8,000 images, 1,000 images for validation, and 4,382 images for testing. The images come from a variety of complex scenes with close arrangement and crowded scenes. During the training process, the first four categories are all used as pedestrian annotations, and the training set and validation set of 9,000 labeled images in the dataset are selected for comparative experiments. Among them, the input images are all adjusted to 640×640 pixels. After training, a trained dense pedestrian detection model is finally obtained, which is used to detect pedestrians in complex and dense scenes.

[0086] like Figure 3 As shown in the figure, the FNS-ELAN module in the backbone feature extraction network includes three parts: an efficient structure for controlling long and short paths, a feature differentiation amplification module and a parameter-free attention SimAM. On the efficient structure for controlling long and short paths, three types of convolutional layers are set, 1×1 and 3×3 ordinary convolutions, 7×7 depth convolutions and depth-separable convolutions. The output of the third convolutional layer is sent to the feature differentiation amplification module, and the shallow features and deep features are weighted through jump connections to amplify the small differences between the target features. The output of the feature differentiation amplification module is concatenated with the output results of other different convolutional layers, and the output features are sent to the SimAM attention. The energy function is used to generate the three-dimensional weight of each feature, and the energy value is used to judge the importance of the feature map, so that the visible area is given a higher weight value. The output features are enhanced using the sigmoid function, and finally the input features and the attention weights are used to calculate the Hadamard product to obtain an enhanced attention feature map.

[0087] The process of the FNS-ELAN module is described as follows: the feature map extracted by four consecutive convolutional layers is used as the input F of the FNS-ELAN module in After three convolutional layers, the output F is obtained 1 , as the input of the feature differentiation amplification module, after passing through the feature differentiation amplification module, the output F 2 , F 2 The output after convolution is concatenated with the output of other convolutional layers to obtain the output feature map F 3 , and then enter the SimAM attention module, redistribute the weights according to the importance of the feature map to obtain the output F 4 , after the last layer of convolution, the backbone network output F is obtained out The entire calculation process is shown in formulas (1 to 5):

[0088] F 1 =Conv 3 (Conv 3 (Conv 1 (F in ))) (1)

[0089] F 2 =FDA(F 1 ) (2)

[0090]

[0091] F 4 =SimAM(F 3 ) (4)

[0092] F out =Conv 1 (F 4 ) (5)

[0093] Among them, FDA stands for Feature Differentiation Amplification Module; Conv j (·) represents the convolution operation, where j represents the convolution kernel size; SimAM represents the SimAM attention module.

[0094] like Figure 4 As shown in Figure 2, the process of the feature differentiation amplification module in the FNS-ELAN network structure is described as follows: 1 As the input of the feature differentiation amplification module, the output F is obtained after a 7×7 deep convolution layer and a deep separable convolution layer. a ; will output F a After two 1×1 ordinary convolutions, the input F 1 Perform weighted processing to obtain the output feature map F 2, the calculation process is shown in formula (6) (7):

[0095]

[0096] Among them, DLG means after a layer of deep convolution, layer normalization and GiLU activation function; D s LG means after a layer of depth-wise separable convolution, layer normalization and GiLU activation function; represents weighted summation; σ represents the GiLU activation function.

[0097] The output feature map F of the parameter-free attention SimAM for efficient structure control of long and short paths 3 As input, the mean and variance of the tensor are calculated in the spatial dimension, and then the minimum energy value is calculated. The activation value is normalized by the sigmoid function, and finally the attention weight is multiplied by the input tensor to obtain the output feature map F 4 , the entire calculation process is shown in formulas (8~11):

[0098] μ=mean(F 3 ) (8)

[0099]

[0100] Among them, μ means calculating the mean in the spatial dimension, mean means calculating the mean in the spatial dimension; σ 2 represents variance; i represents the channel index; N = H × W represents the total number of neurons on the channel; sigmoid represents the activation function; e t Represents the minimum energy value, where the lower the energy value, the greater the difference from the surrounding and the higher the importance, and λ represents the added regularization term.

[0101] The head network combines the idea of ​​the BIFPN network, including the feature enhancement module RFEM and a network structure that integrates multi-scale features. The head network receives output features from FNS-ELAN modules at different layers, adjusts the channels of different output features through 1×1 convolution, and then concatenates the features processed by the feature enhancement module with the input features; secondly, cross-layer connections are added to concatenate the second input features after adjusting the number of channels in a top-down path to retain more contextual information.

[0102] The feature enhancement module RFEM enlarges the receptive field by learning a large upsampling kernel, and then performs weighted fusion of adjacent pixels to better utilize contextual information. Figure 5As shown in the figure, the specific process can be described as follows: the input feature map is first compressed through a convolutional layer, and then the upsampling kernel size at each position is learned through the encoding layer, and normalized using the Softmax function. Next, the upsampled feature map is expanded into a feature sequence, and the upsampling kernel is multiplied by the expanded feature sequence to obtain a feature map with a higher spatial resolution.

[0103] The specific implementation modes of the present invention are described in detail below.

[0104] The environment of the present invention is: the operating system is Ubuntu 5.19.0-50-generic; the deep learning framework is python3.8, pytorch1.10.0, Cuda11.0; the processor is Intel(R) Core(TM) i5-8250U CPU@1.60GHz1.80 GHz; the graphics card is Nvidia Ge-Force RTX 3090.

[0105] The number of training iterations was set to 150, the Adam optimizer was used, the batch size was set to 16, and the initial learning rate was set to 0.01.

[0106] In order to evaluate the performance of the model in this method, the average precision mAP@0.5 with a confidence threshold of 50, the accuracy P (Precision) and the recall R (Recall) are used as evaluation indicators. The specific calculation formula is as follows:

[0107]

[0108] TP represents the number of correctly predicted positive samples, FP represents the number of incorrectly predicted positive samples, FN represents the number of incorrectly predicted negative samples, AP represents the average accuracy of each specific target, and mAP represents the mean of all categories, that is, the accuracy of the overall model, which is an important indicator that can reflect the global performance of the model. The higher the mAP value, the more accurate the model detection result. N represents the total number of categories, and i represents the category index.

[0109] Experimental results analysis

[0110] Table 1

[0111]

[0112]

[0113] Table 1 shows the results of each module in the Wider-Person experiment on the same dataset. From the perspective of various evaluation indicators, the modules proposed in the present invention have an improvement effect on the performance of the model, and compared with the basic model YOLOv7, the accuracy has been effectively improved. Experimental data show that the accuracy, recall and mean average precision (mAP) of the basic model YOLOv7 are 0.803, 0.805 and 0.798 respectively. After adding the FNS-ELAN module to the backbone network, the accuracy, recall and mean average precision (mAP) are increased to 0.829, 0.831 and 0.825 respectively. The improvement of the evaluation indicators represents the improvement of the overall performance of the model, indicating that the FNS-ELAN module increases the model's attention to the features of the visible area; the fusion of multi-scale features in the head network and the addition of the RFEM module increase the accuracy, recall and mean average precision (mAP) to 0.818, 0.825 and 0.814 respectively, indicating that the module expands the receptive field and obtains more contextual information. At the same time, it shows that the fusion of multi-scale features improves the accuracy of the model. Finally, the basic model YOLOv7, adding the FNS-ELAN module in the backbone network, adding the multi-scale feature fusion and RFEM module in the head network, the precision, recall rate and mean average precision mAP are improved to 0.833, 0.835 and 0.829 respectively. Compared with the above three methods, the effect of the present invention is the best.

[0114] The dense pedestrian detection method based on YOLOv7 constructed by the present invention increases the model's attention to the features of the visible area by enhancing the backbone feature extraction network with feature differentiation, thereby laying a good foundation for the detection and positioning of subsequent targets. The features extracted by the backbone network are then input into a network structure and a feature enhancement module designed to fuse multi-scale features, so that the input features first pass through the feature enhancement module to obtain more contextual information, and then select more feature information layers by themselves through the learnable weight factors in the BIFPN network. By utilizing cross-layer connections, the model can extract and fuse information at multiple scales, thereby improving the accuracy of the model for targets of different scales. Experimental results show that the present invention can meet the detection tasks in dense and crowded crowds and further improve the detection performance of the model.

[0115] The dense pedestrian detection method based on YOLOv7 provided by the present invention:

[0116] By using the YOLOv7 network as the basic model and combining it with specific optimization modules, such as the FNS-ELAN module and the feature enhancement module RFEM, the detection difficulty caused by pedestrian mutual occlusion and background interference in complex and dense scenes is effectively solved. The model performs well in extracting and amplifying fine-grained features, enhancing the recognition ability of densely occluded pedestrians.

[0117] Improved the detection accuracy of the model:

[0118] Due to the optimized design of the backbone network and the head network, the model can more accurately extract and fuse multi-scale features, thereby improving the detection accuracy of pedestrians of different scales. In particular, through the use of the BIFPN network structure in the feature fusion stage, the model can more flexibly handle targets of different scales and reduce the probability of missed detection and false detection.

[0119] Improved the computational efficiency of the model:

[0120] While maintaining high detection accuracy, the model achieves efficient feature processing by introducing feature enhancement modules and parameter-free attention mechanism (SimAM). This optimization enables the model to quickly process large amounts of data in complex scenarios, improves overall detection speed and computational efficiency, and is suitable for real-time applications.

[0121] With strong versatility and adaptability:

[0122] The method of the present invention can be applied to a variety of complex scenes, not just a specific environment. Through the optimization design of the preprocessing steps and the model, the method has strong adaptability to pedestrian detection tasks in different scenes and can work stably under various complex backgrounds and density conditions.

[0123] Reduced model false positive rate:

[0124] Through the strategies of feature enhancement and multi-scale feature fusion, the model effectively reduces the false detection rate caused by background interference or pedestrian occlusion, ensuring the accuracy and reliability of the detection results.

[0125] Embodiment 2:

[0126] Based on the same inventive concept, Figure 6 As shown, the present invention also provides a dense pedestrian detection device based on YOLOv7, comprising:

[0127] The acquisition module is used to collect pedestrian image data in complex and dense scenes and pre-process the acquired image data;

[0128] An input module is used to input the preprocessed image data into a pre-trained dense pedestrian detection model; the dense pedestrian detection model uses the YOLOv7 network as a basic model, including: a backbone network and a head network; the backbone network includes: a convolution module and a FNS-ELAN module; the head network includes: a feature enhancement module RFEM and a network structure that integrates multi-scale features;

[0129] The detection module is used to perform final pedestrian detection and positioning based on the feature map calculated by the model, and generate detection results including detection boxes, categories and confidence levels using the output layer.

[0130] The dense pedestrian detection device based on YOLOv7 has significant advantages in detection efficiency, accuracy and real-time performance, and is suitable for various complex scenarios that require efficient and accurate pedestrian detection.

[0131] Embodiment 3:

[0132] Based on the same inventive concept, the present invention also provides a computer device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0133] Memory, used to store computer programs;

[0134] The processor is used to implement an agent-based autonomous learning method as described in any one of the embodiments 1 when executing the program stored in the memory.

[0135] like Figure 4 As shown, the electronic device may include: a processor 71, a communication interface 72, a memory 73 and a communication bus 74, wherein the processor 71, the communication interface 72 and the memory 73 communicate with each other through the communication bus 44. The processor 71 may call the logic instructions in the memory 73 to execute a dense pedestrian detection method based on YOLOv7, the method comprising:

[0136] S10, collecting pedestrian image data in complex and dense scenes, and preprocessing the acquired image data;

[0137] S20, inputting the preprocessed image data into a pre-trained dense pedestrian detection model; the dense pedestrian detection model uses the YOLOv7 network as a basic model, including: a backbone network and a head network; the backbone network includes: a convolution module and a FNS-ELAN module; the head network includes: a feature enhancement module RFEM and a network structure that integrates multi-scale features;

[0138] S30. Based on the feature map calculated by the model, the output layer is used to perform final pedestrian detection and positioning, and a detection result including a detection box, category and confidence is generated.

[0139] In addition, the logic instructions in the above-mentioned memory 73 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0140] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program codes.

[0141] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0142] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0143] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0144] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A dense pedestrian detection method based on YOLOv7, characterized in that: The following steps are involved: S10, collecting pedestrian image data in complex and dense scenes, and preprocessing the acquired image data; S20, inputting the preprocessed image data into a pre-trained dense pedestrian detection model; The dense pedestrian detection model adopts the YOLOv7 network as the basic model, including: a backbone network and a head network; the backbone network includes: a convolution module and an FNS-ELAN module; the head network includes: a feature enhancement module RFEM and a network structure that integrates multi-scale features; S30. Based on the feature map calculated by the model, the output layer is used to perform final pedestrian detection and positioning, and a detection result including a detection box, category and confidence is generated.

2. The dense pedestrian detection method based on YOLOv7 according to claim 1, characterized in that: In the backbone network: The convolution module, which consists of four consecutive convolutional layers, is used for preliminary feature extraction; The FNS-ELAN module, including a combination of various convolutional layers, enhances the differential amplification effect between the fine-grained features of the shallow and deep layers through skip connections and weighted operations, and generates a feature map that amplifies the differences.

3. The dense pedestrian detection method based on YOLOv7 according to claim 2, characterized in that: The FNS-ELAN module consists of three parts: an efficient structure for controlling long and short paths, a feature differentiation amplification module, and a SimAM attention module; Among them, three types of convolutional layers are set on the efficient structure of controlling long and short paths, 1×1 and 3×3 ordinary convolutions, 7×7 depth convolutions and depth-separable convolutions; The output of the third convolutional layer is fed into the feature differentiation amplification module, which weights the shallow features with the deep features through skip connections, thereby amplifying the small differences between the target features. The output of the feature differentiation amplification module is concatenated with the output results of other different convolutional layers, and the output features are sent to the SimAM attention module; The energy function is used to generate the three-dimensional weight of each feature, and the energy value is used to judge the importance of the feature map, so that the visible area is given a higher weight value; The sigmoid function is used to enhance the output features, and finally the Hadamard product is calculated using the input features and the attention weights to obtain the enhanced attention feature map.

4. The dense pedestrian detection method based on YOLOv7 according to claim 3, characterized in that: The data processing flow of the FNS-ELAN module is as follows: The feature map extracted by four consecutive convolutional layers is used as the input F of the FNS-ELAN module in ; After three convolutional layers, the output F1 is obtained as the input of the feature differentiation amplification module; After the feature differentiation amplification module, the output F2 is obtained. The output of F2 after convolution is spliced ​​with the output of other convolution layers to obtain the output feature map F3, which then enters the SimAM attention module to redistribute the weights according to the importance of the feature map to obtain the output F4. After the last layer of convolution, the backbone network output F is obtained. out , the whole calculation process is shown in formulas (1 to 5): F1=Conv3(Conv3(Conv1(F in ))) (1) F2=FDA(F1) (2) F4=SimAM(F3) (4) F out =Conv1(F4) (5) Among them, FDA stands for Feature Differentiation Amplification Module; Conv j (·) represents the convolution operation, where j represents the convolution kernel size; SimAM represents the SimAM attention module.

5. The dense pedestrian detection method based on YOLOv7 according to claim 4, characterized in that: The data processing flow of the feature differentiation amplification module is as follows: (1) F1 is used as the input of the feature differentiation amplification module and is calculated through the 7×7 depth convolution layer and the depth-wise separable convolution layer as shown in formula (6): Among them, DLG means after a layer of deep convolution, layer normalization and GiLU activation function; D s LG means after a layer of depth-wise separable convolution, layer normalization and GiLU activation function; represents weighted sum; (2) Output F a After two 1×1 ordinary convolutions, the weighted processing is performed with the input F1 to obtain the output feature map F2. The calculation process is shown in formula (7): Among them, σ represents the GiLU activation function.

6. The dense pedestrian detection method based on YOLOv7 according to claim 4, characterized in that: The SimAM attention module takes the output feature map F3 of the efficient structure controlling the long and short paths as input, calculates the mean and variance of its tensor in the spatial dimension, and then calculates the minimum energy value. The activation value is normalized by the sigmoid function, and finally the attention weight is multiplied by the input tensor to obtain the output feature map F4. The entire calculation process is shown in formulas (8 to 11): μ=mean(F3) (8) Among them, μ means calculating the mean in the spatial dimension, mean means calculating the mean function in the spatial dimension; σ 2 represents variance; i represents the channel index; N = H × W represents the total number of neurons on the channel; sigmoid represents the activation function; e t Represents the minimum energy value, where the lower the energy value, the greater the difference from the surrounding and the higher the importance, and λ represents the added regularization term.

7. The dense pedestrian detection method based on YOLOv7 according to claim 4, characterized in that: The head network receives output features from different layers of FNS-ELAN modules, adjusts the different output features through 1×1 convolution channels, and then concatenates the features processed by the feature enhancement module RFEM with the input features; Secondly, cross-layer connections are added to concatenate the second input features after adjusting the number of channels in the top-down path to retain more contextual information.

8. The dense pedestrian detection method based on YOLOv7 according to claim 7, characterized in that: The data processing flow of the feature enhancement module RFEM is as follows: The input feature map is compressed through a convolutional layer; The upsampling kernel size of each position is learned through the encoding layer and normalized using the Softmax function; The upsampled feature map is expanded into a feature sequence, and the upsampled kernel is point-multiplied with the expanded feature sequence to obtain a final feature map with higher spatial resolution.

9. A dense pedestrian detection device based on YOLOv7, characterized in that: include: The acquisition module is used to collect pedestrian image data in complex and dense scenes and pre-process the acquired image data; An input module, used to input the preprocessed image data into the pre-trained dense pedestrian detection model; The dense pedestrian detection model adopts the YOLOv7 network as the basic model, including: a backbone network and a head network; the backbone network includes: a convolution module and an FNS-ELAN module; the head network includes: a feature enhancement module RFEM and a network structure that integrates multi-scale features; The detection module is used to perform final pedestrian detection and positioning based on the feature map calculated by the model, and generate detection results including detection boxes, categories and confidence levels using the output layer.

10. A computer device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement a dense pedestrian detection method based on YOLOv7 as described in any one of claims 1 to 8 when executing a program stored in the memory.

Images