A Pedestrian Detection Method Based on Convolutional Neural Network

By constructing a pedestrian detection method of convolutional neural network, the problems of low accuracy and slow speed of traditional pedestrian detection methods are solved, and high-precision and moderate detection speed are achieved, which are suitable for multiple application fields.

CN113989844BActive Publication Date: 2024-12-03NANJING UNIV OF SCI & TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111273933.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-29
Publication Date
2024-12-03
Estimated Expiration
2041-10-29

AI Technical Summary

Technical Problem

The traditional pedestrian detection method has low detection accuracy, and the detection speed based on the two-stage method is too slow and cannot be applied to the actual detection environment.

Method used

The pedestrian detection method based on convolutional neural network is adopted, and the pedestrian regression network is constructed by building a feature extraction network, a feature extraction branch network and a pedestrian regression network, and pedestrian detection is used using deep learning to train the model to improve detection accuracy and speed.

Benefits of technology

It realizes high-precision pedestrian detection, moderate detection speed, can meet the needs of the actual detection environment, and is suitable for fields such as autonomous driving, security systems and robots.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113989844B_ABST
    Figure CN113989844B_ABST
Patent Text Reader

Abstract

The present invention discloses a pedestrian detection method based on a convolutional neural network, which relates to the field of deep learning in computer vision. The method uses a convolutional neural network to detect pedestrians, and can dynamically select training branches according to the sizes of the detection targets to obtain a more accurate description of pedestrian features, without the need to manually design pedestrian features. The method has a high detection accuracy and a high detection speed, which saves time and effort. The method can be applied to multiple fields such as automatic driving, assisted driving systems, security systems, and robots. Moreover, because the method is based on a deep learning method, different pedestrian data sets can be obtained for training according to different application scenarios, so that the method can be applied to a variety of different fields and maintain a high detection accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of target detection in computer vision, and in particular to a pedestrian detection method based on convolutional neural network. Background Art

[0002] Pedestrian detection technology is to determine whether there are pedestrians in a given image and mark the location of the pedestrians, generally using a rectangular frame for marking. Pedestrian detection technology can be combined with pedestrian tracking, pedestrian re-identification and other technologies, and applied to automotive unmanned driving systems (ADAS), intelligent robots, intelligent video surveillance, human behavior analysis, passenger flow counting systems and intelligent transportation.

[0003] Traditional pedestrian detection methods include background difference method, frame difference method, optical flow method, template matching method and machine learning-based methods. Such methods have low detection accuracy due to changes in pedestrian postures and the complex environment in which pedestrians are located. Therefore, pedestrian detection methods based on deep learning have emerged. The detection accuracy of pedestrian detection methods based on deep learning is much higher than that of other traditional methods, and their application fields are wide. At the same time, target detection methods based on deep learning are roughly divided into single-stage method and two-stage method. The single-stage method has a fast detection speed, but the detection accuracy is slightly lower than that of the two-stage method. Although the two-stage method has a higher detection accuracy, it cannot be applied to the actual detection environment due to its too low detection speed. Therefore, the present invention provides a pedestrian detection method that can be applied to the actual detection environment. Summary of the invention

[0004] The purpose of the present invention is to provide a pedestrian detection method based on a convolutional neural network to solve the problems of low detection accuracy of traditional methods in the field of pedestrian detection and slow detection speed based on a two-stage method.

[0005] The technical solution to achieve the purpose of the present invention is: a pedestrian detection method based on a convolutional neural network, the method comprising the following steps:

[0006] Step A, collecting and preprocessing a pedestrian image dataset, and dividing the dataset into a training set and a test set according to a preset ratio;

[0007] Step B, constructing a convolutional neural network model and initializing the parameters in the model;

[0008] Step C, marking the size of the object in the training set image, and using the training set to train the convolutional neural network model;

[0009] Step D: Use the trained deep convolutional neural network model to detect pedestrians and obtain the pedestrian detection frame at the accurate position.

[0010] Furthermore, in step B, a convolutional neural network model is constructed and the parameters in the model are initialized. The specific process includes:

[0011] Step B1, constructing a feature extraction network for extracting features from pedestrian images, and initializing its parameters;

[0012] Step B2, constructing a feature extraction branch network for processing images of targets of different sizes respectively, and initializing its parameters;

[0013] Step B3: construct a pedestrian regression network and initialize its parameters.

[0014] Further, the feature extraction network in step B1 specifically includes: a first convolutional layer, a maximum pooling layer, three identical first residual modules, four identical second residual modules, six identical third residual modules, and three identical fourth residual modules connected in sequence, and the input of the network is connected to the output of the first residual module; the convolution kernel size of the first convolutional layer is 7×7, the number of channels is 64, and the step size is 2, and the convolution kernel of the maximum pooling layer is 3×3 and the step size is 2; the first residual module includes a convolution layer with a convolution kernel of 1×1 and 64 channels, a convolution layer with a convolution kernel of 3×3 and 64 channels, and a convolution layer with a convolution kernel of 1×1 and 256 channels connected in sequence; The second residual module includes a convolution layer with a convolution kernel of 1×1 and 128 channels, a convolution layer with a convolution kernel of 3×3 and 128 channels, and a convolution layer with a convolution kernel of 1×1 and 512 channels, which are connected in sequence; the third residual module includes a convolution layer with a convolution kernel of 1×1 and 256 channels, a convolution layer with a convolution kernel of 3×3 and 256 channels, and a convolution layer with a convolution kernel of 1×1 and 1024 channels, which are connected in sequence; the fourth residual module includes a convolution layer with a convolution kernel of 1×1 and 512 channels, a convolution layer with a convolution kernel of 3×3 and 512 channels, and a convolution layer with a convolution kernel of 1×1 and 2048 channels, which are connected in sequence;

[0015] Initializing the parameters of the feature extraction network specifically includes:

[0016] (1) Set the residual module parameters, convolution layer parameters and pooling layer parameters. The forward propagation formula of the residual module is shown in formula (1):

[0017]

[0018] Among them, x L represents the output of the residual module, x l represents the input of the residual module, represents the convolution operation on the input, F(·) represents the convolution function, and Wi Represents the parameters to be trained of the convolution structure;

[0019] The back propagation formula of the residual module is shown in formula (2):

[0020]

[0021] The forward propagation formula of the convolutional layer is shown in formula (3):

[0022]

[0023] Where f represents the activation function, l represents the number of layers, m and n represent the mapping numbers of the feature maps on the previous layer and the current layer. represents the nth feature map of the current lth layer, represents the mth feature map of the l-1th layer, The convolution kernel representing the correlation between the nth feature map of the lth layer and the mth feature map of the l-1th layer, represents the bias corresponding to the nth feature map of the lth layer, M n Represents a subset of the input feature map, * represents a convolution operation;

[0024] The reverse error propagation formula of the convolutional layer is shown in formula (4):

[0025]

[0026] In the formula, represents the sensitivity of the nth neural node in the lth layer, represents the weight of the downsampling layer, represents dot product, up() represents upsampling operation, f T (·) means to transpose the function f. in, and They represent the weight and partial derivative corresponding to the nth feature map of the lth layer, Represents the nth feature map of the l-1th layer;

[0027] The bias of the convolution layer and the back propagation formula of the convolution kernel are shown in equations (5) and (6) respectively:

[0028]

[0029] In the formula, E represents the loss function, for For each small area when calculating the convolution, u and v represent the sensitivity respectively. The position of the elements in ;

[0030] For the forward propagation of the pooling layer, the average of the four adjacent pixels of the feature map of the previous layer is directly calculated. For the backward propagation of the pooling layer, when the next layer of the pooling layer is a convolutional layer, the reverse error propagation formula is shown in formula (7):

[0031]

[0032] In the formula, M represents the set of input features, Represents the convolution kernel related to the nth feature map of the l+1th layer and the mth feature map of the lth layer, represents the sensitivity of the nth neural node in the l+1th layer, Represents the sensitivity of the mth neural node in the lth layer; when the next layer of the pooling layer is not a convolutional layer, the reverse error propagation of the pooling layer is the same as the error propagation of the BP network;

[0033] (2) Use Gaussian initialization to initialize the feature extraction network.

[0034] Furthermore, the input of the feature extraction branch network is the output of the feature extraction network, including three branches, each of which includes a 1×1 convolution layer, a 3×3 convolution layer, and a 1×1 convolution layer connected in sequence, the input of the branch is connected to the output, and each branch shares weights with other branches. The difference between the three branches is that the expansion rates of the dilated convolution are different, which are set to 1, 2, and 3 respectively;

[0035] Initializing parameters of the feature extraction branch network specifically includes:

[0036] (1) Set the parameters of the dilated convolution. The forward propagation formula and the back propagation formula of the dilated convolution are the same as those of the convolution layer, as shown in equations (5) and (6). The calculation formula of the receptive field of the dilated convolution is shown in equation (8):

[0037]

[0038] In the formula, r n-1 is the receptive field of the previous layer, s i is the step size of the i-th convolution layer, and k is the size of the convolution kernel;

[0039] (2) Use Gaussian initialization to initialize the feature extraction branch network.

[0040] Furthermore, the input of the pedestrian regression network is the result of the output of the feature extraction branch network passing through a convolution layer, including three branches, which are respectively used to predict the position of the pedestrian center point, the size of the pedestrian, and the bias of the pedestrian center point; the three branches each include a convolution layer with a convolution kernel of 3×3, a padding of 1, and a stride of 1, and a convolution layer with a convolution kernel of 1×1, which are connected in sequence;

[0041] Initializing the parameters of the CCB regression network includes:

[0042] Gaussian initialization is used to initialize the pedestrian regression network.

[0043] Furthermore, in step C, the convolutional neural network model is trained using the training set, specifically:

[0044] (1) The images in the training set are input into the feature extraction network;

[0045] (2) Then, the output of the feature extraction network is sent to the three branches of the feature extraction branch network according to the following principles:

[0046] Will The image is sent to the branch with a dilation rate of 1;

[0047] Will The image is sent to the branch with a dilation rate of 2;

[0048] Will The image is sent to the branch with a dilation rate of 3;

[0049] Among them, l i 、u i are the preset minimum and maximum sizes of the target, respectively, w and h represent the width and height of the target marked in the image, respectively;

[0050] (3) The output of the feature extraction branch network is input into the pedestrian regression network after a convolution layer, and the position and size of the pedestrian in the image are output;

[0051] (4) Using (9), (10), (11) and (12) to obtain the network loss, use back propagation to train the parameters in the convolutional neural network model, and repeat the training until the difference between the network output and the actual label is less than the set value, then the network training is completed;

[0052]

[0053] Formula (9) is the loss function of the center point, where α and β are 2 and 4 respectively, N represents the number of key points in the image, that is, the number of objects to be detected, x, y and z represent the position and category of the pixel in the image respectively, and Y is the true value. Predicted value;

[0054]

[0055] Formula (10) is the loss function of the target position, p represents the coordinate of the true value of the annotation, represents the predicted target center position, Output feature map for the network bias;

[0056]

[0057] Formula (11) is the loss function of the target size, represents the predicted pedestrian size, N represents the number of targets in the detected image, and s k represents the true size of the target k;

[0058] L det =L k +λ size L size +λ off L off (12)

[0059] Formula (12) is the loss function of the entire network. The parameters of the network are trained according to this formula, where λ size , off 0.1 and 1 respectively.

[0060] Compared with the prior art, the present invention has the following significant advantages: 1) It uses deep learning to construct a pedestrian detection network with high detection accuracy, is insensitive to changes in the actual detection environment, has good robustness, and can be applied in actual production environments; 2) Since the convolutional neural network is directly used in the network to regress the pedestrian position and size information, and the branch training method is used at the same time, the detection speed and detection accuracy can well meet the detection requirements in the project.

[0061] The present invention is further described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 FIG. 4 is a structural diagram of a residual module in an embodiment.

[0063] Figure 2 FIG. 4 is a diagram of a feature extraction network structure in one embodiment.

[0064] Figure 3 FIG. 4 is a diagram of a feature extraction branch network structure in one embodiment.

[0065] Figure 4 The figure is a flowchart of a pedestrian detection method based on a convolutional neural network in one embodiment. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0067] In one embodiment, in combination Figure 4 , a pedestrian detection method based on convolutional neural network is provided, the method comprising the following steps:

[0068] Step A, collecting and preprocessing a pedestrian image dataset, and dividing the dataset into a training set and a test set according to a preset ratio;

[0069] Here, the pedestrian image dataset can be obtained by collecting public pedestrian detection image data (such as Caltech pedestrian dataset, EuroCity Persons-ECP dataset, CVC pedestrian dataset, etc.) and existing road checkpoint surveillance videos;

[0070] Step B, constructing a convolutional neural network model and initializing the parameters in the model;

[0071] Step C, marking the size of the object in the training set image, and using the training set to train the convolutional neural network model;

[0072] Step D: Use the trained deep convolutional neural network model to detect pedestrians and obtain the pedestrian detection frame at the accurate position.

[0073] Furthermore, in one embodiment, the preprocessing in step A includes Gaussian filtering and normalization processing, and the specific process includes:

[0074] Step A2a, the size of the images in the pedestrian image dataset is uniformly set to 512x512, and Gaussian filtering is performed on the images. The formula used is:

[0075]

[0076] In the formula, (x, y) is the coordinate of the point in the image, and σ is the standard deviation. This operation is to eliminate irrelevant interference in the image to make the final detection more accurate.

[0077] In step A2b, each pixel in the image is normalized using the formula:

[0078]

[0079] Where (i, j) represents the position of the pixel in the image, x ij represents the pixel value at position (i, j), x min 、x max Represent the minimum and maximum values ​​of pixels in the image respectively.

[0080] Furthermore, in one of the embodiments, in step A, the entire pedestrian image dataset is divided into training data and test data in a ratio of 4:1, because this ratio can better generalize the model parameters and make the trained network more robust.

[0081] Furthermore, in one embodiment, the step B of constructing a convolutional neural network model and initializing the parameters in the model specifically includes:

[0082] Step B1, constructing a feature extraction network for extracting features from pedestrian images, and initializing its parameters;

[0083] Step B2, constructing a feature extraction branch network for processing images of targets of different sizes respectively, and initializing its parameters;

[0084] Step B3: construct a pedestrian regression network and initialize its parameters.

[0085] Further, in one embodiment, in combination Figure 1 and Figure 2 , the feature extraction network in step B1 is composed of multiple residual modules connected. The so-called residual module connects the input and output based on the series connection of multiple convolutional layers. Specifically, it includes the following connected in sequence: the first convolutional layer, the maximum pooling layer, three identical first residual modules, four identical second residual modules, six identical third residual modules, and three identical fourth residual modules. The input of the network is connected to the output of the first residual module; the convolution kernel size of the first convolutional layer is 7×7, the number of channels is 64, and the step size is 2; the convolution kernel of the maximum pooling layer is 3×3, and the step size is 2; the first residual module includes a convolution layer with a convolution kernel of 1×1 and 64 channels, a convolution layer with a convolution kernel of 3×3 and 64 channels, and a convolution layer with a convolution kernel of 1×1 and 256 channels connected in sequence; the second residual module includes a convolution layer with a convolution kernel of 1×1 and 64 channels, a convolution layer with a convolution kernel of 3×3 and 64 channels, and a convolution layer with a convolution kernel of 1×1 and 256 channels connected in sequence; The convolution layer has a convolution kernel of 1×1 and a channel number of 128, a convolution kernel of 3×3 and a channel number of 128, and a convolution kernel of 1×1 and a channel number of 512; the third residual module includes a convolution layer with a convolution kernel of 1×1 and a channel number of 256, a convolution layer with a convolution kernel of 3×3 and a channel number of 256, and a convolution layer with a convolution kernel of 1×1 and a channel number of 1024 connected in sequence; the fourth residual module includes a convolution layer with a convolution kernel of 1×1 and a channel number of 512, a convolution layer with a convolution kernel of 3×3 and a channel number of 512, and a convolution layer with a convolution kernel of 1×1 and a channel number of 2048 connected in sequence; specifically as shown in Table 1 below:

[0086] Table 1 Feature extraction network structure

[0087]

[0088] Initializing the parameters of the feature extraction network specifically includes:

[0089] (1) Set the residual module parameters, convolution layer parameters and pooling layer parameters. The forward propagation formula of the residual module is shown in formula (15):

[0090]

[0091] Among them, x L represents the output of the residual module, x l represents the input of the residual module, represents the convolution operation on the input, F(·) represents the convolution function, and W i Represents the parameters to be trained of the convolution structure;

[0092] The back propagation formula of the residual module is shown in formula (16):

[0093]

[0094] The forward propagation formula of the convolutional layer is shown in formula (17):

[0095]

[0096] Where f represents the activation function, l represents the number of layers, m and n represent the mapping numbers of the feature maps on the previous layer and the current layer. represents the nth feature map of the current lth layer, represents the mth feature map of the l-1th layer, The convolution kernel representing the correlation between the nth feature map of the lth layer and the mth feature map of the l-1th layer, represents the bias corresponding to the nth feature map of the lth layer, M n Represents a subset of the input feature map, * represents a convolution operation;

[0097] The reverse error propagation formula of the convolutional layer is shown in formula (18):

[0098]

[0099] In the formula, represents the sensitivity of the nth neural node in the lth layer, represents the weight of the downsampling layer, represents dot product, up() represents upsampling operation, f T (·) means to transpose the function f. in, and They represent the weight and partial derivative corresponding to the nth feature map of the lth layer, Represents the nth feature map of the l-1th layer;

[0100] The bias of the convolution layer and the back propagation formula of the convolution kernel are shown in equations (19) and (20) respectively:

[0101]

[0102] In the formula, E represents the loss function, for For each small area when calculating the convolution, u and v represent the sensitivity respectively. The position of the elements in ;

[0103] For the forward propagation of the pooling layer, the average of the four adjacent pixels of the feature map of the previous layer is directly calculated. For the backward propagation of the pooling layer, when the next layer of the pooling layer is the convolutional layer, the reverse error propagation formula is shown in formula (21):

[0104]

[0105] In the formula, M represents the set of input features, Represents the convolution kernel related to the nth feature map of the l+1th layer and the mth feature map of the lth layer, represents the sensitivity of the nth neural node in the l+1th layer, Represents the sensitivity of the mth neural node in the lth layer; when the next layer of the pooling layer is not a convolutional layer, the reverse error propagation of the pooling layer is the same as the error propagation of the BP network;

[0106] (2) Use Gaussian initialization to initialize the feature extraction network.

[0107] This network is just an example. In actual implementation, the number of network layers can be appropriately increased or decreased, or even modules such as inception can be added to improve network performance.

[0108] Further, in one embodiment, in combination Figure 3 , the input of the feature extraction branch network is the output of the feature extraction network, including three branches, each of which includes a 1×1 convolution layer, a 3×3 convolution layer, and a 1×1 convolution layer connected in sequence, the input of the branch is connected to the output, and each branch shares weights with other branches. The difference between the three branches is that the expansion rates of the dilated convolution are different, which are set to 1, 2, and 3 respectively; the introduction to the expansion rate is as follows: in a general convolution kernel, adjacent and continuous pixels of the feature map are taken as elements and multiplied by the convolution kernel. At this time, the dilation rate is 1, but for a convolution with a dilated convolution greater than 1, the two sampling points are separated by d-1 feature pixels, where d is the dilation rate;

[0109] Initializing parameters of the feature extraction branch network specifically includes:

[0110] (1) Set the parameters of the dilated convolution. The forward propagation formula and the back propagation formula of the dilated convolution are the same as those of the convolution layer, as shown in equations (19) and (20). The calculation formula of the receptive field of the dilated convolution is shown in equation (22):

[0111]

[0112] In the formula, r n-1 is the receptive field of the previous layer, s i is the step size of the i-th convolution layer, and k is the size of the convolution kernel;

[0113] (2) Use Gaussian initialization to initialize the feature extraction branch network, that is, randomly sample from a Gaussian distribution with a mean of 0 and a variance of 1.

[0114] In this example, one such feature extraction branch network is used. In a specific implementation scheme, multiple such branch networks can be used according to actual use.

[0115] Furthermore, in one of the embodiments, the input of the pedestrian regression network is the result of the output of the feature extraction branch network passing through a convolution layer, including three branches connected in parallel, which are respectively used to predict the position of the pedestrian center point, the size of the pedestrian, and the bias of the pedestrian center point; the three branches each include a convolution layer with a convolution kernel of 3×3, a padding of 1, and a stride of 1, and a convolution layer with a convolution kernel of 1×1 (dimensionality reduction);

[0116] Initializing the parameters of the CCB regression network includes:

[0117] Gaussian initialization is used to initialize the pedestrian regression network.

[0118] Further, in one embodiment, in step C, the convolutional neural network model is trained using the training set, specifically:

[0119] (1) The images in the training set are input into the feature extraction network;

[0120] (2) Then, the output of the feature extraction network is sent to the three branches of the feature extraction branch network according to the following principles:

[0121] Will The image is sent to the branch with a dilation rate of 1;

[0122] Will The image is sent to the branch with a dilation rate of 2;

[0123] Will The image is sent to the branch with a dilation rate of 3;

[0124] Among them, l i 、u i are the preset minimum and maximum sizes of the target, respectively, w and h represent the width and height of the target marked in the image, respectively;

[0125] (3) The output of the feature extraction branch network is input into the pedestrian regression network after a convolution layer, and the position and size of the pedestrian in the image are output;

[0126] (4) Using (23), (24), (25) and (26) to obtain the network loss, use back propagation to train the parameters in the convolutional neural network model, and repeat the training until the difference between the network output and the actual label is less than the set value, then the network training is completed;

[0127]

[0128] Formula (23) is the loss function of the center point, where α and β are 2 and 4 respectively, N represents the number of key points in the image, that is, the number of targets to be detected, x, y and z represent the position and category of the pixel in the image respectively, and Y is the true value. Predicted value;

[0129]

[0130] Formula (24) is the loss function of the target position, p represents the coordinate of the annotated true value, represents the predicted target center position, Output feature map for the network bias;

[0131]

[0132] Formula (25) is the loss function of the target size, represents the predicted pedestrian size, N represents the number of targets in the detected image, and s k represents the true size of the target k;

[0133] L det =L k +λ size L size +λ off L off (26)

[0134] Formula (26) is the loss function of the entire network. The parameters of the network are trained according to this formula, where λ size , off 0.1 and 1 respectively.

[0135] In the training process of the above convolutional network, one iteration process includes: fitting the detection of pedestrian targets through back propagation and gradient descent algorithms, reducing the error of the entire convolutional neural network by reducing the error of the detection target position, bias and category, and then updating the weights in the model through forward propagation. After 10,000 iterations or the error between the output of the neural network and the true target is less than the set value, the training is terminated.

[0136] In summary, the method of the present invention uses a convolutional neural network to detect pedestrians. It can dynamically select training branches according to the sizes of the detection targets to obtain a more accurate description of pedestrian features without manually designing pedestrian features. It has a high detection accuracy and a fast detection speed, which saves time and effort. It can be applied to multiple fields such as autonomous driving, assisted driving systems, security systems, and robots. Moreover, because it is based on a deep learning method, different pedestrian data sets can be obtained for training according to different application scenarios, so that it can be applied to a variety of different fields and maintain a high detection accuracy.

[0137] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A pedestrian detection method based on convolutional neural network, characterized in that: The method comprises the following steps: Step A, collecting and preprocessing a pedestrian image dataset, and dividing the dataset into a training set and a test set according to a preset ratio; Step B: construct a convolutional neural network model and initialize the parameters in the model; the specific process includes: Step B1, constructing a feature extraction network for extracting features from pedestrian images, and initializing its parameters; the feature extraction network specifically includes: a first convolutional layer, a maximum pooling layer, three identical first residual modules, four identical second residual modules, six identical third residual modules, and three identical fourth residual modules connected in sequence, and the input of the network is connected to the output of the first residual module; the convolution kernel size of the first convolutional layer is 7×7, the number of channels is 64, and the step size is 2, and the convolution kernel of the maximum pooling layer is 3×3 and the step size is 2; the first residual module includes a convolution layer with a convolution kernel of 1×1 and a number of channels of 64, a convolution layer with a convolution kernel of 3×3 and a number of channels of 64, and a convolution kernel of 1× 1, a convolution layer with 256 channels; the second residual module includes a convolution layer with a convolution kernel of 1×1 and 128 channels, a convolution layer with a convolution kernel of 3×3 and 128 channels, and a convolution layer with a convolution kernel of 1×1 and 512 channels connected in sequence; the third residual module includes a convolution layer with a convolution kernel of 1×1 and 256 channels, a convolution layer with a convolution kernel of 3×3 and 256 channels, and a convolution layer with a convolution kernel of 1×1 and 1024 channels connected in sequence; the fourth residual module includes a convolution layer with a convolution kernel of 1×1 and 512 channels, a convolution layer with a convolution kernel of 3×3 and 512 channels, and a convolution layer with a convolution kernel of 1×1 and 2048 channels connected in sequence; Initializing the parameters of the feature extraction network specifically includes: (1) Set the residual module parameters, convolution layer parameters and pooling layer parameters. The forward propagation formula of the residual module is shown in formula (1): Among them, x L represents the output of the residual module, x l represents the input of the residual module, represents the convolution operation on the input, F(·) represents the convolution function, and W i Represents the parameters to be trained of the convolution structure; The back propagation formula of the residual module is shown in formula (2): The forward propagation formula of the convolutional layer is shown in formula (3): In the formula, f represents the activation function, l represents the number of layers, m and n represent the mapping numbers of the feature maps on the previous layer and the current layer. represents the nth feature map of the current lth layer, represents the mth feature map of the l-1th layer, The convolution kernel representing the correlation between the nth feature map of the lth layer and the mth feature map of the l-1th layer, represents the bias corresponding to the nth feature map of the lth layer, M n Represents a subset of the input feature map, * represents a convolution operation; The reverse error propagation formula of the convolutional layer is shown in formula (4): In the formula, represents the sensitivity of the nth neural node in the lth layer, represents the weight of the downsampling layer, represents dot product, up() represents upsampling operation, f T (·) means to transpose the function f. in, and They represent the weight and partial derivative corresponding to the nth feature map of the lth layer, Represents the nth feature map of the l-1th layer; The bias of the convolution layer and the back propagation formula of the convolution kernel are shown in equations (5) and (6) respectively: In the formula, E represents the loss function, for For each small area when calculating the convolution, u and v represent the sensitivity respectively. The position of the elements in ; For the forward propagation of the pooling layer, the average of the four adjacent pixels of the feature map of the previous layer is directly calculated. For the backward propagation of the pooling layer, when the next layer of the pooling layer is a convolutional layer, the reverse error propagation formula is shown in formula (7): In the formula, M represents the set of input features, Represents the convolution kernel related to the nth feature map of the l+1th layer and the mth feature map of the lth layer, represents the sensitivity of the nth neural node in the l+1th layer, Represents the sensitivity of the mth neural node in the lth layer; when the next layer of the pooling layer is not a convolutional layer, the reverse error propagation of the pooling layer is the same as the error propagation of the BP network; (2) Use Gaussian initialization to initialize the feature extraction network; Step B2, constructing a feature extraction branch network for processing images of targets of different sizes respectively, and initializing its parameters; Step B3, construct a pedestrian regression network and initialize its parameters; Step C, marking the size of the object in the training set image, and using the training set to train the convolutional neural network model; Step D: Use the trained deep convolutional neural network model to detect pedestrians and obtain the pedestrian detection frame at the accurate position.

2. The pedestrian detection method based on convolutional neural network according to claim 1, characterized in that: The preprocessing in step A includes Gaussian filtering and normalization processing, and the specific process includes: Step A1a, the size of the images in the pedestrian image dataset is uniformly set to 512x512, and Gaussian filtering is performed on the images. The formula used is: Where (x, y) is the coordinate of the point in the image, and σ is the standard deviation; Step A1b, normalize each pixel in the image using the formula: Where (i, j) represents the position of the pixel in the image, x ij represents the pixel value at position (i, j), x min 、x max Represent the minimum and maximum values ​​of pixels in the image respectively.

3. The pedestrian detection method based on convolutional neural network according to claim 1, characterized in that: The input of the feature extraction branch network is the output of the feature extraction network, including three parallel branches, each of which includes a 1×1 convolution layer, a 3×3 convolution layer, and a 1×1 convolution layer connected in sequence, and the input and output of the branch are connected. At the same time, each branch shares weights with other branches. The difference between the three branches is that the expansion rates of the dilated convolution are different, which are set to 1, 2, and 3 respectively; Initializing parameters of the feature extraction branch network specifically includes: (1) Set the parameters of the dilated convolution. The forward propagation formula and the back propagation formula of the dilated convolution are the same as those of the convolution layer, as shown in equations (5) and (6). The calculation formula of the receptive field of the dilated convolution is shown in equation (10): In the formula, r n-1 is the receptive field of the previous layer, s i is the step size of the i-th convolution layer, and k is the size of the convolution kernel; (2) Use Gaussian initialization to initialize the feature extraction branch network.

4. The pedestrian detection method based on convolutional neural network according to claim 1, characterized in that: The input of the pedestrian regression network is the result of the output of the feature extraction branch network passing through a convolution layer, including three parallel branches, which are respectively used to predict the position of the pedestrian center point, the size of the pedestrian, and the bias of the pedestrian center point; the three branches all include a convolution layer with a convolution kernel of 3×3, a padding of 1, and a stride of 1, and a convolution layer with a convolution kernel of 1×1. Initializing the parameters of the CCB regression network includes: Gaussian initialization is used to initialize the pedestrian regression network.

5. The pedestrian detection method based on convolutional neural network according to claim 1, characterized in that: In step C, the convolutional neural network model is trained using the training set, specifically: (1) The images in the training set are input into the feature extraction network; (2) Then, the output of the feature extraction network is sent to the three branches of the feature extraction branch network according to the following principles: Will The image is sent to the branch with a dilation rate of 1; Will The image is sent to the branch with a dilation rate of 2; Will The image is sent to the branch with a dilation rate of 3; Among them, l i 、u i are the minimum and maximum sizes of the preset target, respectively, w and h represent the width and height of the target marked in the image, respectively; (3) The output of the feature extraction branch network is input into the pedestrian regression network after a convolution layer, and the position and size of the pedestrian in the image are output; (4) Using (11), (12), (13) and (14) to obtain the network loss, use back propagation to train the parameters in the convolutional neural network model, and repeat the training until the difference between the network output and the actual label is less than the set value, then the network training is completed; Formula (11) is the loss function of the center point, where α and β are 2 and 4 respectively, N represents the number of key points in the image, that is, the number of targets to be detected, x, y and z represent the position and category of the pixel in the image respectively, and Y is the true value. Predicted value; Formula (12) is the loss function of the target position, p represents the coordinate of the true value of the annotation, represents the predicted target center position, Output feature map for the network bias; Formula (13) is the loss function of the target size, represents the predicted pedestrian size, N represents the number of targets in the detected image, and s k represents the true size of the target k; L det =L k +λ size L size +λ off L off (14) Formula (14) is the loss function of the entire network. The parameters of the network are trained according to this formula, where λ size , off 0.1 and 1 respectively.