Pedestrian detection method and device based on channel attention mechanism

By introducing the fusion processing of the channel attention mechanism and the object detection module in pedestrian detection, a multi-scale predicted attention map is generated, which solves the insufficient identification of the existing technology in complex scenarios and occlusion situations, and achieves high-accuracy pedestrian detection.

CN118942119APending Publication Date: 2024-11-12FUDAN UNIVERSITY
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202411042570.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The existing pedestrian detection methods do not perform well when dealing with complex scenes, posture changes and partial occlusion, and are not robust and real-time, making it difficult to adapt to the needs of complex scenes.

Method used

The pedestrian detection method based on the channel attention mechanism is adopted, and the feature extraction module is used to perform deep feature extraction, and the predicted attention maps of multiple different scales are generated by combining the channel attention module, and the object detection module is used for fusion processing to generate pedestrian appearance feature vectors to Position pedestrian location.

Benefits of technology

Accurate identification of pedestrians in video sequences is achieved, the robustness of detection and the ability to adapt to complex scenarios is improved, and the detection accuracy rate reaches 91.3%, which is suitable for different scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118942119A_ABST
    Figure CN118942119A_ABST
Patent Text Reader

Abstract

The invention provides a pedestrian detection method and device based on a channel attention mechanism, and the method is characterized in that the method comprises the following steps: S1, carrying out the preprocessing of a video sequence, and obtaining a preprocessed image; s2, inputting the preprocessed image into a pedestrian detection model to obtain a pedestrian appearance feature vector; s3, the pedestrian appearance feature vector is calculated, a pedestrian detection result is obtained, and a pedestrian detection model comprises a feature extraction module used for performing depth feature extraction on the preprocessed image to obtain a feature map; the channel attention module is used for processing the feature map through a channel attention mechanism to obtain a predicted attention map; and the target detection module is used for performing up-sampling and convolution processing on the predicted attention map to obtain a pedestrian appearance feature vector, and the method can accurately identify pedestrians in the video sequence.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a pedestrian detection method, and in particular to a pedestrian detection method and device based on a channel attention mechanism. Background Art

[0002] Early pedestrian detection methods mainly used Haar features and cascade classifiers, among which the Viola-Jones algorithm is a representative. This method quickly identifies pedestrians in images by training a cascade of weak classifiers. However, these methods perform poorly when dealing with complex scenes, posture changes, and partial occlusion.

[0003] With the rise of deep learning, convolutional neural networks have achieved remarkable success in pedestrian detection tasks. By using deep convolutional networks, the system can automatically learn features in images, thereby improving the detection ability of pedestrians of different scales, postures, and occlusions.

[0004] Pedestrian detection has also benefited from the development of target detection, such as Faster R-CNN, YOLO, and SSD frameworks. These frameworks achieve faster and more accurate target detection by introducing the region proposal network RPN or adopting a single network structure. Through data enhancement technology, more training data can be synthesized to improve the generalization ability of the model. Transfer learning uses models pre-trained on other tasks to accelerate and improve the performance of pedestrian detection models.

[0005] However, existing methods still face higher demands in terms of robustness, real-time performance, and adaptability to complex scenarios. Summary of the invention

[0006] The present invention is made to solve the above-mentioned problems, and its purpose is to provide a pedestrian detection method and device based on channel attention mechanism.

[0007] The present invention provides a pedestrian detection method based on a channel attention mechanism, which is used to obtain pedestrian detection results of video sequences and has the following characteristics, including the following steps: step S1, preprocessing the video sequence to obtain a preprocessed image; step S2, inputting the preprocessed image into a pedestrian detection model to obtain a pedestrian appearance feature vector; step S3, calculating the pedestrian appearance feature vector to obtain a pedestrian detection result, wherein the pedestrian detection model includes: a feature extraction module, which is used to perform deep feature extraction on the preprocessed image to obtain a feature map; a channel attention module, which is used to process the feature map through the channel attention mechanism to obtain a predicted attention map; and a target detection module, which is used to upsample and convolve the predicted attention map to obtain a pedestrian appearance feature vector.

[0008] The pedestrian detection method based on the channel attention mechanism provided by the present invention may also have the following features: wherein, in step S1, preprocessing includes removing noise, adjusting image brightness and contrast, normalization processing, and resizing.

[0009] The pedestrian detection method based on the channel attention mechanism provided by the present invention may also have the following features: wherein, in the feature extraction module, the ResNet-50 network is used as a multi-scale feature extractor.

[0010] In the pedestrian detection method based on the channel attention mechanism provided by the present invention, it can also have the following characteristics: wherein, the channel attention module includes a first convolution unit, a first upsampling layer, a second convolution unit, a second upsampling layer, a third convolution unit, a third upsampling layer and a fourth convolution unit connected in sequence, the first upsampling layer, the second upsampling layer and the third upsampling layer are all used to upsample the input to twice, the first convolution unit, the second convolution unit, the third convolution unit and the fourth convolution unit all include a first convolution layer, a Relu activation function, a second convolution layer and a sigmoid activation function connected in sequence, and the first convolution unit, the second convolution unit, the third convolution unit and the fourth convolution unit respectively output predicted attention maps of corresponding sizes.

[0011] In the pedestrian detection method based on the channel attention mechanism provided by the present invention, it can also have the following characteristics: the convolution kernel size of the first convolution layer of the first convolution unit, the second convolution layer of the first convolution unit, the first convolution layer of the second convolution unit, and the second convolution layer of the second convolution unit are all 3×3, and the number of channels is 512, the convolution kernel size of the first convolution layer and the second convolution layer of the third convolution unit are both 3×3, and the number of channels is 256, and the convolution kernel size of the first convolution layer and the second convolution layer of the fourth convolution unit are both 3×3, and the number of channels is 128.

[0012] In the pedestrian detection method based on the channel attention mechanism provided by the present invention, it can also have the following characteristics: wherein, the target detection module includes a fifth convolution unit, a fourth upsampling layer, a sixth convolution unit, a fifth upsampling layer, a seventh convolution unit, a sixth upsampling layer and an eighth convolution unit connected in sequence, the fourth upsampling layer, the fifth upsampling layer and the sixth upsampling layer are all used to upsample the input to twice, the fifth convolution unit, the sixth convolution unit, the seventh convolution unit and the eighth convolution unit all include a concat layer, a third convolution layer and a Relu activation function connected in sequence, the concat layer is used to connect the input of the convolution unit with a predicted attention map of the same size, and use the connection result as the input of the connected third convolution layer.

[0013] In the pedestrian detection method based on the channel attention mechanism provided by the present invention, it can also have the following characteristics: the convolution kernel size of the third convolution layer of the fifth convolution unit is 3×3, and the number of channels is 512, the convolution kernel size of the sixth convolution unit is 3×3, and the number of channels is 512, the convolution kernel size of the seventh convolution unit is 3×3, and the number of channels is 256, and the convolution kernel size of the eighth convolution unit is 3×3, and the number of channels is 128.

[0014] In the pedestrian detection method based on the channel attention mechanism provided by the present invention, it can also have the following characteristics: wherein, the training of the pedestrian detection model includes the following steps: step T1, constructing a training data set based on existing image data; step T2, selecting a training image from the training data set and inputting it into the pedestrian detection model to obtain a pedestrian appearance feature vector and a predicted attention map; step T3, calculating the pedestrian detection result based on the pedestrian appearance feature vector; step T4, calculating the loss based on the pedestrian detection result and the predicted attention map and performing weighted addition to obtain the total loss; step T5, updating the parameters of the pedestrian detection model through back propagation based on the total loss; step T6, repeating steps T1 to T5 until the training termination condition is reached, thereby obtaining a trained pedestrian detection model.

[0015] The present invention also provides a pedestrian detection device based on a channel attention mechanism, which has the following characteristics: a preprocessing unit, which is used to preprocess a video sequence to obtain a preprocessed image; a detection unit, which includes a pedestrian detection model, which is used to input the preprocessed image into the pedestrian detection model to obtain a pedestrian appearance feature vector; a pedestrian detection result output unit, which is used to calculate the pedestrian appearance feature vector to obtain a pedestrian detection result, wherein the pedestrian detection model includes: a feature extraction module, which is used to perform deep feature extraction on the preprocessed image to obtain a feature map; a channel attention module, which is used to process the feature map through the channel attention mechanism to obtain a predicted attention map; and a target detection module, which is used to upsample and convolve the predicted attention map to obtain a pedestrian appearance feature vector.

[0016] Functions and Effects of the Invention

[0017] According to the pedestrian detection method and device based on the channel attention mechanism involved in the present invention, a channel attention module is used to generate a plurality of predicted attention maps of different scales, and then a target detection module is used to fuse the multi-scale predicted attention maps, thereby being able to effectively locate the position containing pedestrians in the picture. Therefore, the pedestrian detection method and device based on the channel attention mechanism of the present invention can accurately identify pedestrians in video sequences. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a block diagram of a pedestrian detection device according to an embodiment of the present invention;

[0019] Figure 2 is a block diagram of a pedestrian detection model in an embodiment of the present invention;

[0020] Figure 3 is a schematic diagram of a process for training a pedestrian detection model in an embodiment of the present invention;

[0021] Figure 4 It is a flow chart of a pedestrian detection method based on a channel attention mechanism in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the following embodiments and the accompanying drawings specifically illustrate the pedestrian detection method and device based on the channel attention mechanism of the present invention.

[0023] In this embodiment, a pedestrian detection device based on a channel attention mechanism is provided, hereinafter referred to as a pedestrian detection device, which is used to obtain pedestrian detection results of a video sequence.

[0024] Figure 1 4 is a block diagram of a pedestrian detection device in an embodiment of the present invention.

[0025] like Figure 1 As shown, the pedestrian detection device 100 includes a pre-processing unit 11, a detection unit 12 and a pedestrian detection result output unit 13.

[0026] The preprocessing unit 11 is used to preprocess the video sequence to obtain a preprocessed image.

[0027] The preprocessing includes removing noise, adjusting image brightness and contrast, normalizing, and resizing. In this embodiment, normalizing is to scale the pixel values ​​of the image to a specific range, such as [0, 1] or [-1, 1].

[0028] The detection unit 12 includes a pedestrian detection model, which is used to input the preprocessed image into the pedestrian detection model to obtain a pedestrian appearance feature vector.

[0029] Figure 2 is a block diagram of a pedestrian detection model in an embodiment of the present invention.

[0030] like Figure 2 As shown, the pedestrian detection model 200 includes a feature extraction module 21, a channel attention module 22 and a target detection module 23.

[0031] The feature extraction module 21 is used to perform deep feature extraction on the preprocessed image to obtain a feature map. In the feature extraction module 21, a ResNet-50 network is used as a multi-scale feature extractor to ensure sufficient network depth with relatively low computational overhead.

[0032] The channel attention module 22 is used to process the feature map through the channel attention mechanism to obtain a predicted attention map.

[0033] Among them, the channel attention module 22 includes a first convolution unit, a first upsampling layer, a second convolution unit, a second upsampling layer, a third convolution unit, a third upsampling layer and a fourth convolution unit connected in sequence.

[0034] The first upsampling layer, the second upsampling layer, and the third upsampling layer are all used to upsample the input to two times.

[0035] The first convolution unit, the second convolution unit, the third convolution unit and the fourth convolution unit each include a first convolution layer, a Relu activation function, a second convolution layer and a sigmoid activation function connected in sequence.

[0036] Among them, the convolution kernel size of the first convolution layer of the first convolution unit, the second convolution layer of the first convolution unit, the first convolution layer of the second convolution unit, and the second convolution layer of the second convolution unit are all 3×3, and the number of channels is 512. The convolution kernel size of the first convolution layer and the second convolution layer of the third convolution unit is 3×3, and the number of channels is 256. The convolution kernel size of the first convolution layer and the second convolution layer of the fourth convolution unit is 3×3, and the number of channels is 128.

[0037] The first convolution unit, the second convolution unit, the third convolution unit, and the fourth convolution unit respectively output predicted attention maps of corresponding sizes, that is, there are four predicted attention maps in total.

[0038] The target detection module 23 is used to upsample and convolve the predicted attention map to obtain a pedestrian appearance feature vector.

[0039] Among them, the target detection module 23 includes a fifth convolution unit, a fourth upsampling layer, a sixth convolution unit, a fifth upsampling layer, a seventh convolution unit, a sixth upsampling layer and an eighth convolution unit connected in sequence.

[0040] The fourth upsampling layer, the fifth upsampling layer, and the sixth upsampling layer are all used to upsample the input to twice.

[0041] The fifth convolution unit, the sixth convolution unit, the seventh convolution unit and the eighth convolution unit each include a concat layer, a third convolution layer and a Relu activation function connected in sequence.

[0042] Among them, the convolution kernel size of the third convolution layer of the fifth convolution unit is 3×3, and the number of channels is 512. The convolution kernel size of the sixth convolution unit is 3×3, and the number of channels is 512. The convolution kernel size of the seventh convolution unit is 3×3, and the number of channels is 256. The convolution kernel size of the eighth convolution unit is 3×3, and the number of channels is 128.

[0043] The concat layer is used to connect the input of the convolution unit with the predicted attention map of the same size, and use the connection result as the input of the connected third convolution layer.

[0044] In this embodiment, the sliding step sizes of the convolutional layers of the channel attention module 22 and the target detection module 23 are both 1.

[0045] Figure 3 It is a schematic diagram of the process of training a pedestrian detection model in an embodiment of the present invention.

[0046] like Figure 3 As shown, the training process of the pedestrian detection model 200 includes the following steps:

[0047] Step T1, construct a training data set based on existing image data.

[0048] In this embodiment, images from the existing CityPersons dataset are used as existing images to construct a training dataset. CityPersons is a large-scale dataset for pedestrian detection, which mainly focuses on pedestrian detection tasks in urban scenes. The CityPersons dataset contains a large number of images labeled as pedestrians, with a total of nearly 5,000 training images and more test images. These images come from urban environments and have a variety of scenes, including urban streets, intersections, sidewalks, etc., covering a variety of postures and movements of pedestrians in different backgrounds. Compared with some other datasets, CityPersons has relatively dense annotations for each pedestrian instance. This means that the bounding box of each pedestrian is annotated, and more detailed attribute information is provided, such as the degree of occlusion, the direction of the pedestrian, etc. In addition, in this embodiment, the existing images are further augmented, including random rotation, flipping, cropping, etc., which helps to expand the training dataset and improve the generalization ability of the model.

[0049] Step T2, select a training image from the training data set and input it into the pedestrian detection model 200 to obtain the pedestrian appearance feature vector and the predicted attention map.

[0050] Step T3, obtaining pedestrian detection results based on pedestrian appearance feature vector calculations.

[0051] Step T4, calculate the loss based on the pedestrian detection result and the predicted attention map and perform weighted addition to obtain the total loss. In this embodiment, the weight is learnable.

[0052] Step T5, based on the total loss, update the parameters of the pedestrian detection model 200 through back propagation.

[0053] Step T6, repeating steps T1 to T5 until the training termination condition is reached, thereby obtaining a trained pedestrian detection model 200.

[0054] In this embodiment, through the above training, the detection accuracy of the pedestrian detection model 200 reaches 91.3%. Compared with the existing pedestrian detection model based on the channel attention mechanism, the pedestrian detection model 200 can extract feature vectors with richer semantic information, more accurately use the location information and quantity information of pedestrians, thereby learning richer features, thereby obtaining better detection accuracy, and is suitable for different scenarios.

[0055] The pedestrian detection result output unit 13 is used to calculate the pedestrian appearance feature vector to obtain the pedestrian detection result.

[0056] The following describes the process of using the pedestrian detection device 100 to perform a pedestrian detection method based on a channel attention mechanism in conjunction with the accompanying drawings.

[0057] Figure 4 It is a flow chart of a pedestrian detection method based on a channel attention mechanism in an embodiment of the present invention.

[0058] like Figure 4 As shown in the figure, the pedestrian detection method based on the channel attention mechanism includes the following steps:

[0059] Step S1, using the preprocessing unit 11 to preprocess the video sequence to obtain a preprocessed image.

[0060] Step S2: Using the detection unit 12 to input the pre-processed image into the pedestrian detection model 200 to obtain a pedestrian appearance feature vector.

[0061] Step S3, using the pedestrian detection result output unit 13 to calculate the pedestrian appearance feature vector to obtain the pedestrian detection result.

[0062] Functions and Effects of the Embodiments

[0063] According to the pedestrian detection method and device based on the channel attention mechanism involved in this embodiment, a channel attention module is used to generate multiple predicted attention maps of different scales, and then a target detection module is used to fuse the multi-scale predicted attention maps, thereby effectively locating the position containing pedestrians in the picture. In short, this method can accurately identify pedestrians in video sequences.

[0064] Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A pedestrian detection method based on a channel attention mechanism, used to obtain pedestrian detection results of a video sequence, characterized in that: The following steps are involved: Step S1, preprocessing the video sequence to obtain a preprocessed image; Step S2, inputting the preprocessed image into a pedestrian detection model to obtain the pedestrian appearance feature vector; Step S3, calculating the pedestrian appearance feature vector to obtain the pedestrian detection result, Wherein, the pedestrian detection model includes: A feature extraction module, used to perform deep feature extraction on the preprocessed image to obtain a feature map; A channel attention module, used for processing the feature map through a channel attention mechanism to obtain a predicted attention map; The target detection module is used to upsample and convolve the predicted attention map to obtain a pedestrian appearance feature vector.

2. The pedestrian detection method based on channel attention mechanism according to claim 1, characterized in that: in, In the step S1, the preprocessing includes removing noise, adjusting image brightness and contrast, normalizing, and resizing.

3. The pedestrian detection method based on channel attention mechanism according to claim 1, characterized in that: in, In the feature extraction module, the ResNet-50 network is used as a multi-scale feature extractor.

4. The pedestrian detection method based on channel attention mechanism according to claim 1, characterized in that: in, The channel attention module includes a first convolution unit, a first upsampling layer, a second convolution unit, a second upsampling layer, a third convolution unit, a third upsampling layer and a fourth convolution unit connected in sequence, The first upsampling layer, the second upsampling layer and the third upsampling layer are all used to upsample the input to twice. The first convolution unit, the second convolution unit, the third convolution unit and the fourth convolution unit each include a first convolution layer, a Relu activation function, a second convolution layer and a sigmoid activation function connected in sequence, The first convolution unit, the second convolution unit, the third convolution unit and the fourth convolution unit respectively output the predicted attention map of corresponding size.

5. The pedestrian detection method based on channel attention mechanism according to claim 4, characterized in that: in, The convolution kernel sizes of the first convolution layer of the first convolution unit, the second convolution layer of the first convolution unit, the first convolution layer of the second convolution unit, and the second convolution layer of the second convolution unit are all 3×3, and the number of channels is 512. The convolution kernel sizes of the first convolution layer and the second convolution layer of the third convolution unit are both 3×3, and the number of channels is 256. The convolution kernel sizes of the first convolution layer and the second convolution layer of the fourth convolution unit are both 3×3, and the number of channels is both 128.

6. The pedestrian detection method based on channel attention mechanism according to claim 1, characterized in that: in, The target detection module includes a fifth convolution unit, a fourth upsampling layer, a sixth convolution unit, a fifth upsampling layer, a seventh convolution unit, a sixth upsampling layer and an eighth convolution unit connected in sequence, The fourth upsampling layer, the fifth upsampling layer and the sixth upsampling layer are all used to upsample the input to twice, The fifth convolution unit, the sixth convolution unit, the seventh convolution unit and the eighth convolution unit each include a concat layer, a third convolution layer and a Relu activation function connected in sequence, The concat layer is used to connect the input of the convolution unit with the predicted attention map of the same size, and use the connection result as the input of the connected third convolution layer.

7. The pedestrian detection method based on channel attention mechanism according to claim 6, characterized in that: in, The convolution kernel size of the third convolution layer of the fifth convolution unit is 3×3, and the number of channels is 512. The convolution kernel size of the sixth convolution unit is 3×3, and the number of channels is 512. The convolution kernel size of the seventh convolution unit is 3×3, and the number of channels is 256. The convolution kernel size of the eighth convolution unit is 3×3, and the number of channels is 128.

8. The pedestrian detection method based on channel attention mechanism according to claim 1, Features: The training of the pedestrian detection model includes the following steps: Step T1, constructing a training data set based on existing image data; Step T2, selecting a training image from the training data set and inputting it into the pedestrian detection model to obtain the pedestrian appearance feature vector and the predicted attention map; Step T3, obtaining the pedestrian detection result according to the pedestrian appearance feature vector calculation; Step T4, calculating the loss according to the pedestrian detection result and the predicted attention map and performing weighted addition to obtain a total loss; Step T5, updating the parameters of the pedestrian detection model through back propagation according to the total loss; Step T6, repeatedly executing steps T1 to T5 until the training termination condition is reached, thereby obtaining the trained pedestrian detection model.

9. A pedestrian detection device based on a channel attention mechanism, used to obtain pedestrian detection results of a video sequence, characterized in that: include: A preprocessing unit, used for preprocessing the video sequence to obtain a preprocessed image; A detection unit, comprising a pedestrian detection model, configured to input the preprocessed image into the pedestrian detection model to obtain a pedestrian appearance feature vector; A pedestrian detection result output unit is used to calculate the pedestrian appearance feature vector to obtain the pedestrian detection result. Wherein, the pedestrian detection model includes: A feature extraction module, used to perform deep feature extraction on the preprocessed image to obtain a feature map; A channel attention module, used for processing the feature map through a channel attention mechanism to obtain a predicted attention map; The target detection module is used to upsample and convolve the predicted attention map to obtain a pedestrian appearance feature vector.

Citation Information

Patent Citations

  • Crowd density estimation method and system based on multi-scale attention mechanism

    CN114511818A