A pedestrian detection method, electronic equipment and readable storage medium

Abstract

The application is suitable for the technical field of computers, and provides a pedestrian detection method, an electronic device and a readable storage medium. The method comprises the following steps: determining a first image based on a to-be-detected image, and the first image comprises a target pedestrian; inputting the first image into a human body detection model to obtain a detection result; the human body detection model at least comprises a feature extraction layer, and the feature extraction layer is used for extracting features of the first image; the feature extraction layer is composed of a first feature extraction layer, a second feature extraction layer, a third feature extraction layer and a fourth feature extraction layer; the detection result comprises one or more of the following: a first detection box, a second detection box and a third detection box, the first detection box is used for identifying a head region of the target pedestrian, the second detection box is used for identifying an upper body region of the target pedestrian, and the third detection box is used for identifying a lower body region of the target pedestrian. Therefore, each part of the pedestrian can be quickly identified, and the accuracy of pedestrian detection is improved.

Description

Technical Field

[0001] This application belongs to the field of target detection technology, and in particular relates to a pedestrian detection method, electronic device and readable storage medium. Background Technology

[0002] In computer applications (such as artificial intelligence systems, vehicle driver assistance systems, intelligent robots, human behavior analysis, and intelligent transportation), electronic devices typically detect pedestrians in images or video sequences, enabling them to perform subsequent operations based on the pedestrians' locations. For example, in the field of intelligent robots, intelligent robots can detect pedestrians in images, allowing electronic devices to accurately avoid pedestrians based on their positions.

[0003] Currently, pedestrian detection algorithms based on deep convolutional neural networks (CNNs) are commonly used for training and classification. However, these algorithms have long processing times for images or video sequences, resulting in delayed detection responses and lengthy lead times for obtaining detection results. Summary of the Invention

[0004] This application provides a pedestrian detection method, an electronic device, and a readable storage medium, which can accelerate the inference speed of the detection algorithm and obtain detection results more quickly.

[0005] In a first aspect, embodiments of this application provide a pedestrian detection method, which includes:

[0006] A first image is determined based on the image to be detected, and the first image includes the target pedestrian;

[0007] The first image is input into the human detection model to obtain the detection result. The human detection model is trained based on the training sample set. The human detection model includes at least a feature extraction layer, which is used to extract features from the first image. The feature extraction layer consists of a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, and a fourth feature extraction layer.

[0008] The first feature extraction layer is used to output the first feature map, the second feature extraction layer is used to output the second feature map, the third feature extraction layer is used to output the third feature map, and the fourth feature extraction layer is used to output the fourth feature map. The sizes of the first feature map, the second feature map, the third feature map, and the fourth feature map decrease sequentially, and the number of the first feature map, the second feature map, the third feature map, and the fourth feature map increases sequentially.

[0009] The detection results include one or more of the following: a first detection box, a second detection box, and a third detection box. The first detection box is used to identify the head area of ​​the target pedestrian, the second detection box is used to identify the upper body area of ​​the target pedestrian, and the third detection box is used to identify the lower body area of ​​the target pedestrian.

[0010] This application embodiment determines a first image based on the image to be detected, enabling the extraction of an image including the target pedestrian from the image to be detected. This allows the human detection model to perform subsequent detection processing on the first image of a specified size, accelerating inference speed. Inputting the first image into the human detection model yields detection results. Feature extraction from the first image can be achieved through four feature extraction layers in the human detection model, accelerating feature extraction. Furthermore, increasing the number of feature maps makes the detection results of the target pedestrian more accurate. Specifically, the four feature extraction layers can accurately extract the head region, upper body region, and lower body region of the target pedestrian. Therefore, pedestrian detection speed can be accelerated, and accurate detection results can be obtained.

[0011] In one possible implementation, the first feature extraction layer includes a first convolution operator, a second convolution operator, an activation function, and a concatenation and fusion operator; the first feature map is obtained through the following method:

[0012] The fifth feature map is convolved using the first convolution operator to obtain the convolved fifth feature map, and the fifth feature map after convolution is nonlinearly mapped using an activation function to obtain the first intermediate map; the fifth feature map is the feature map input to the first feature extraction layer;

[0013] The first intermediate image is convolved using the second convolution operator to obtain the first intermediate image after convolution, and the second intermediate image is obtained by nonlinear mapping of the first intermediate image after convolution using the activation function.

[0014] The fifth feature map and the second intermediate map are fused using the splicing and fusion operator to obtain the third intermediate map;

[0015] The third intermediate image is convolved using the first convolution operator to obtain the third intermediate image after convolution, and the fourth intermediate image is obtained by nonlinearly mapping the third intermediate image after convolution using the activation function.

[0016] The fourth intermediate image is convolved using the second convolution operator to obtain the convolved fourth intermediate image, and the fifth intermediate image is obtained by nonlinearly mapping the convolved fourth intermediate image using the activation function.

[0017] The first feature map is obtained by using the splicing and fusion operator to perform feature fusion on the third and fifth intermediate images.

[0018] In one possible implementation, the human detection model further includes an input layer, a convolutional layer, a max-pooling layer, a transition layer, an upsampling layer, a concatenation and fusion layer, and an output layer. The input layer is used to input a first image. The convolutional layer is used to perform convolution operations on the first image output by the input layer to obtain a convolutional feature map. The max-pooling layer is used to perform dimensionality reduction processing on the feature map output by the convolutional layer to obtain a dimensionality-reduced feature map. The transition layer is used to reduce the number of feature maps output by the convolutional layer. The upsampling layer is used to aggregate the feature maps output by the convolutional layer to obtain an aggregated feature map. The concatenation and fusion layer is used to fuse the feature maps output by different convolutional layers to obtain a fused feature map. The output layer is used to output the detection result.

[0019] In one possible implementation, the training sample set includes a first sample image set and a second sample image set; the first sample image set is a collection of images after annotating the body parts and coordinate information of each body part for each of the multiple pedestrians, and the body parts of the pedestrians include one or more of the following: head, upper body, and lower body; the human detection model is obtained using the following training method:

[0020] Obtain the first sample image set;

[0021] The first sample image set is subjected to data augmentation processing to obtain the second sample image set. The data augmentation processing includes one or more of the following processing methods: symmetry processing, scaling processing and translation processing.

[0022] The first and second sample image sets are input into the original human detection model for training to obtain the human detection model.

[0023] In one possible implementation, the pedestrian detection method further includes:

[0024] The first image is input into the human detection model to obtain multiple head prediction boxes, multiple upper body prediction boxes, and multiple lower body prediction boxes;

[0025] Multiple head prediction boxes are removed to obtain a first detection box; multiple upper body prediction boxes are removed to obtain a second detection box; and multiple lower body prediction boxes are removed to obtain a third detection box. The detection results include the first detection box, the second detection box, and the third detection box.

[0026] In one possible implementation, the pedestrian detection method further includes: determining whether the confidence level of each head prediction box in the multiple head prediction boxes is less than a first threshold, wherein the first threshold is a pre-set prediction box confidence value;

[0027] If so, then the head prediction boxes with confidence scores less than the first threshold in the multiple head prediction boxes are removed to obtain the first remaining prediction boxes, which include one or more head prediction boxes.

[0028] Determine whether the intersection-union ratio of each head prediction box in the first remaining prediction box is greater than the second threshold, which is a pre-set intersection-union ratio value for prediction boxes;

[0029] If so, the header prediction boxes with an intersection-union ratio greater than the second threshold in the first remaining prediction boxes are removed to obtain the first detection box.

[0030] In one possible implementation, the pedestrian detection method further includes: acquiring an image to be detected, the image to be detected including multiple pedestrians, the multiple pedestrians including the target pedestrian, and each pedestrian having corresponding coordinate information in the image to be detected;

[0031] Based on the coordinates of the target pedestrian, the first image is extracted from the image to be detected.

[0032] In one possible implementation, the pedestrian detection method further includes the following steps before inputting the first image into the human detection model:

[0033] The size of the first image is adjusted to the preset size to obtain the first image after adjustment. The preset size refers to the size of the image input into the human detection model that is set in advance.

[0034] The resized first image is subjected to mean removal and / or normalization to obtain the preprocessed first image;

[0035] The process of inputting the first image into the human detection model includes:

[0036] The preprocessed first image is input into the human detection model.

[0037] Secondly, embodiments of this application provide a pedestrian detection device for performing the method in the first aspect or any possible implementation thereof. Specifically, the device may include modules for performing the pedestrian detection method in the first aspect or any possible implementation thereof.

[0038] Thirdly, embodiments of this application provide an electronic device including a memory and a processor. The memory stores instructions; the processor executes the instructions stored in the memory, causing the electronic device to perform the pedestrian detection method in the first aspect or any possible implementation thereof.

[0039] Fourthly, a readable storage medium is provided, which stores instructions that, when executed on a computer, cause the computer to perform the pedestrian detection method of the first aspect or any possible implementation thereof.

[0040] Fifthly, a computer program product containing instructions is provided, which, when executed on a mobile device, causes the mobile device to perform the pedestrian detection method of the first aspect or any possible implementation thereof.

[0041] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a flowchart illustrating a pedestrian detection method provided in an embodiment of this application;

[0044] Figure 2 This is a schematic diagram of the structure of a human body detection model provided in an embodiment of this application;

[0045] Figure 3 This is a schematic diagram of the first type of detection result in the embodiments of this application;

[0046] Figure 4 This is a schematic diagram of the second type of detection result in the embodiments of this application;

[0047] Figure 5 This is a schematic diagram of the third detection result in the embodiments of this application;

[0048] Figure 6 This is a schematic diagram of the structure of a human body detection model provided in an embodiment of this application;

[0049] Figure 7 This is a flowchart illustrating the training process of a human detection model provided in an embodiment of this application;

[0050] Figure 8 This is a training block diagram of a human detection model provided in an embodiment of this application;

[0051] Figure 9 This is a schematic block diagram of a pedestrian detection device provided in an embodiment of this application. Detailed Implementation

[0052] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0053] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0054] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0055] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0056] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0057] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0058] The pedestrian detection method of this application is applicable to scenarios requiring pedestrian monitoring and identification. This method, through four feature extraction layers, can more quickly detect the head, upper body, and lower body regions of pedestrians, and the obtained results for these regions are more accurate.

[0059] The pedestrian detection method of this application embodiment can be applied to electronic devices. The electronic device includes a memory, a processor (such as a CPU), and a computer program stored in the memory and executable on the processor. The processor executes the computer program to implement the pedestrian detection method of this application embodiment.

[0060] It should be understood that electronic devices can be smartphones, tablets, desktop computers, laptops, handheld devices, servers, etc. This application does not impose any limitations on the specific type of electronic device.

[0061] It should be noted that the pedestrian detection in this application refers to a method of using computer vision technology to determine whether there are pedestrians in an image or video sequence, and to accurately locate pedestrians in the image or accurately locate pedestrians in the sequence frame.

[0062] It should also be noted that, in the pedestrian detection method of this application embodiment, the electronic device's display screen will show a prominent prompt before collecting pedestrian data, thereby reminding pedestrians that their personal information is about to be collected. Furthermore, the collection of pedestrian video or image data is conducted with the pedestrian's consent, and the collected data is used solely for detection and will not be used for any other purpose.

[0063] Currently, pedestrian detection algorithms contain a large number of feature extraction layers, which results in a slow speed for extracting pedestrian features and a long time to obtain detection results.

[0064] Please see Figure 1 , Figure 1 A flowchart illustrating a pedestrian detection method provided in an embodiment of this application is shown. Figure 1 As shown, the pedestrian detection method of this application includes:

[0065] S101, determine the first image based on the image to be detected.

[0066] Optionally, the image to be detected includes multiple pedestrians. Each pedestrian has corresponding coordinate information in the image to be detected. For example, the image to be detected can be a raw image captured by an image acquisition device such as a camera. The image acquisition device can be a camera of model CV500. Alternatively, the image to be detected can also be a video sequence extracted from any surveillance video, and this application does not impose any restrictions on this.

[0067] Optionally, the first image includes the target pedestrian. The target pedestrian is one of a group of pedestrians. For example, the first image may be the region of interest (ROI) of the target pedestrian. The ROI refers to a rectangular region or mask of the target pedestrian defined in the image to be detected.

[0068] In some embodiments, the size of the first image is larger than the preset size of the human detection model. The preset size refers to the size of the image input into the human detection model beforehand. The preset size includes a preset width (W) and a preset height (H), which can be set and changed by those skilled in the art according to actual detection needs. For example, the preset width is set to 128 pixels and the preset height is set to 256 pixels.

[0069] Electronic devices can acquire images to be detected from image acquisition devices. Alternatively, electronic devices can extract video sequences from any segment of surveillance video stored within the device as images to be detected. Thus, electronic devices can acquire images to be detected.

[0070] After acquiring the image to be detected, the electronic device can determine the first image, or in other words, the region of interest of the target pedestrian, based on the image. This provides a data foundation for subsequent pedestrian detection.

[0071] Alternatively, in some embodiments, the first image can be determined by the following methods:

[0072] Electronic devices can acquire images to be detected. After acquiring the images, the electronic devices can determine the coordinate information of multiple pedestrians from the images.

[0073] Electronic devices can determine the coordinates of a target pedestrian from the coordinates of multiple pedestrians. Therefore, the electronic device can extract a first image from the image to be detected based on the target pedestrian's coordinates.

[0074] Since the size of the first image may not match the size of the image input into the human detection model, i.e., the preset size, the size of the first image needs to be adjusted (resized) after obtaining the first image.

[0075] Before inputting the first image into the human detection model, the electronic device can adjust the size of the first image to a preset size, obtaining a resized first image. The electronic device can also perform mean-reduction and / or normalization processing on the resized first image to obtain a preprocessed first image. This makes data processing in the human detection model more convenient and speeds up the detection process.

[0076] In some embodiments, the electronic device can subtract the mean of 0 from the pixel values ​​of each of the red (R), green (G), and blue (B) channels to obtain a mean-reduced first image. The electronic device can then perform a standard normalization operation on the mean-reduced first image, or in other words, divide the pixel values ​​of the mean-reduced first image by 255, thereby obtaining a preprocessed first image. Based on this preprocessed first image, the electronic device can then detect target pedestrians.

[0077] S102, input the first image into the human detection model to obtain the detection result.

[0078] Optionally, the human detection model is trained based on a training sample set. The training sample set includes a first sample image set and a second sample image set. The first sample image set is a collection of images after annotating the body parts and coordinate information of each body part of each pedestrian among multiple pedestrians. The body parts of the pedestrians include one or more of the following: head, upper body, and lower body.

[0079] Optionally, the human detection model includes at least an input layer, an output layer, and a feature extraction layer, with the order of the layers as follows: Figure 2 As shown in the diagram. The feature extraction layer is used to extract features from the first image. For example... Figure 2 As shown, the feature extraction layer consists of a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, and a fourth feature extraction layer from top to bottom. For example, the feature extraction layer can employ dense blocks. Dense blocks are deep convolutional neural networks used for image classification.

[0080] In this embodiment, by employing dense blocks, the feature maps output by any layer in the human detection model can be accessed during feature extraction. This allows for feature fusion between the feature maps output by any previous layer and the current output feature map, ensuring that the feature maps are fully utilized within the dense blocks. This improves feature extraction performance and enhances the expressive power of the human detection model. Furthermore, dense blocks can reduce the number of parameters in the original human detection model while maintaining network depth, thereby improving training efficiency.

[0081] In some embodiments, a first feature extraction layer is used to output a first feature map. A second feature extraction layer is used to output a second feature map. A third feature extraction layer is used to output a third feature map. A fourth feature extraction layer is used to output a fourth feature map. The sizes of the first, second, third, and fourth feature maps decrease sequentially, and the number of each feature map increases sequentially. This allows for the use of fewer feature extraction layers, accelerating the detection speed of the human detection model and achieving better feature extraction results.

[0082] In some embodiments, the first feature extraction layer includes a first convolution operator, a second convolution operator, an activation function, and a concatenation and fusion operator. The first and second convolution operators perform convolution operations to extract features of the target pedestrian. The kernel sizes of the first and second convolution operators are different. The stride sizes of the first and second convolution operators are also different. Furthermore, the padding values ​​of the first and second convolution operators are different.

[0083] In some embodiments, the activation function is used to perform nonlinear mapping on the result of the first convolution operator or the result of the second convolution operator. For example, the activation function can be a leaky function. The leaky function works correctly on different types of electronic devices, thus enabling the human detection model to better match different types of electronic devices.

[0084] In some embodiments, the concat operator is used to fuse multiple feature maps of the same size, thereby enabling the construction of more complex feature representations.

[0085] It should be understood that the number and order of the various operators (such as the first convolution operator, activation function, etc.) in the first feature extraction layer can be set according to the actual situation, and this application does not limit this.

[0086] It should be noted that the other feature extraction layers besides the first feature extraction layer (the second, third, and fourth feature extraction layers) also include the first convolution operator, the second convolution operator, the activation function, and the concatenation and fusion operator. The difference lies in the number of each operator in each feature extraction layer. Therefore, the description of the second, third, and fourth feature extraction layers can be referenced from that of the first feature extraction layer, and will not be repeated here.

[0087] In some embodiments, the detection result includes one or more of the following: a first detection box, a second detection box, and a third detection box. The first detection box is used to identify the head region of the target pedestrian. The second detection box is used to identify the upper body region of the target pedestrian. The third detection box is used to identify the lower body region of the target pedestrian.

[0088] It should be noted that the first, second, and third detection boxes represent different parts of the target pedestrian. In actual detection, there may be multiple detection boxes that do not overlap, or there may be two detection boxes that partially overlap. For example, ... Figure 3 As shown, the first, second, and third detection boxes do not overlap. Figure 4 As shown, the first and second detection boxes partially overlap. Figure 5 As shown, the second and third detection boxes partially overlap. It should be understood that... Figures 3 to 5 The dashed lines in the text are merely illustrative annotations, intended to visually illustrate different scenarios of the test results, and do not represent the actual output of the test results.

[0089] It should also be noted that the first detection box, the second detection box, and the third detection box can be as follows: Figures 3 to 5 The rectangular frame shown can also be any other shape of detection box, and this application does not limit it.

[0090] In some embodiments, the detection result may further include one or more of the following: the confidence level of the first detection box, the coordinates of the first detection box, the class probability of the first detection box, the confidence level of the second detection box, the coordinates of the second detection box, the class probability of the second detection box, the confidence level of the third detection box, the coordinates of the third detection box, and the class probability of the third detection box.

[0091] Taking the first detection box as an example, when the first detection box is a rectangle, the head position of the target pedestrian can be the coordinates of the center point of the first detection box, or the coordinates of the upper left corner and the lower right corner of the first detection box. This application does not limit this.

[0092] After obtaining the preprocessed first image, the electronic device can input the first image into the human detection model and output the detection result. In other words, the electronic device can input the first image into the human detection model for forward propagation, and the network structure in the human detection model processes the first image sequentially and outputs the detection result of the target pedestrian.

[0093] It should be understood that the above detection method is illustrated with one target pedestrian as an example. When there are multiple target pedestrians, the electronic device can crop the first image and input the cropped image of each target pedestrian into the human detection model for detection to obtain the detection result of each target pedestrian.

[0094] The pedestrian detection method of this application determines a first image based on the image to be detected, and can extract an image including the target pedestrian from the image to be detected. This allows the human detection model to perform subsequent detection processing on the first image of a specified size, accelerating the inference speed. Inputting the first image into the human detection model yields detection results. Feature extraction from the first image can be achieved through four feature extraction layers in the human detection model, which speeds up feature extraction. Furthermore, increasing the number of feature maps makes the detection results of the target pedestrian more accurate. That is, the head region, upper body region, and lower body region of the target pedestrian can be accurately extracted through the four feature extraction layers. Therefore, the pedestrian detection speed can be accelerated, and accurate detection results can be obtained.

[0095] Based on the above description, the following details one possible implementation method for obtaining detection results by the electronic device in the embodiments of this application.

[0096] Considering that the detection results may include one or more components, to more clearly illustrate the specific process by which an electronic device obtains the detection results, the following explanation uses an example where the detection results include a first detection frame, a second detection frame, and a third detection frame.

[0097] In some embodiments, the electronic device may input a first image into a human detection model to obtain multiple head prediction boxes, multiple upper body prediction boxes, and multiple lower body prediction boxes.

[0098] Multiple head prediction bounding boxes are used to indicate the regions in the first image where the head of a target pedestrian may be present. Multiple upper body prediction bounding boxes are used to indicate the regions in the first image where the upper body of a target pedestrian may be present. Multiple lower body prediction bounding boxes are used to indicate the regions in the first image where the lower body of a target pedestrian may be present.

[0099] In other embodiments, the electronic device may input a first image into a human detection model to obtain the confidence and class probability of each head prediction box in a plurality of head prediction boxes, the confidence and class probability of each upper body prediction box in a plurality of upper body prediction boxes, and the confidence and class probability of each lower body prediction box in a plurality of lower body prediction boxes.

[0100] The category probability of the predicted bounding box (including the head predicted bounding box, the upper body predicted bounding box, and the lower body predicted bounding box) refers to the probability that the predicted bounding box belongs to any body part of the pedestrian.

[0101] After obtaining the predicted bounding boxes, the electronic device can use algorithms from the human detection model to remove overlapping boxes within the predicted bounding boxes (or perform post-processing) to ensure accurate detection results. Optionally, the electronic device can use the non-maximum suppression (NMS) algorithm from the human detection model to remove overlapping boxes within the predicted bounding boxes.

[0102] In some embodiments, the electronic device can remove multiple head prediction boxes to obtain a first detection box. The electronic device can remove multiple upper body prediction boxes to obtain a second detection box. And, the electronic device can remove multiple lower body prediction boxes to obtain a third detection box. This ensures that the detection results output by the human detection model include a unique first detection box, a unique second detection box, and a unique third detection box, avoiding false detections and guaranteeing the accuracy of the detection results obtained by the electronic device.

[0103] Electronic devices can employ various methods to remove overlapping boxes within the predicted bounding box. The following section uses the header predicted bounding box as an example to illustrate the removal process in detail.

[0104] In some embodiments, the electronic device can arrange the confidence scores of each head prediction box in descending order to obtain a confidence score list.

[0105] The electronic device can determine whether the confidence level of each head prediction box in multiple head prediction boxes is less than a first threshold.

[0106] Optionally, the first threshold is a pre-set confidence value for the prediction box. For example, the first threshold can be 0.5. It should be understood that the first threshold can be determined by experimental testing or theoretical calculation, and the specific value of the first threshold is not limited in the embodiments of this application.

[0107] If so, the electronic device removes head prediction boxes with a confidence level less than a first threshold from the multiple head prediction boxes to obtain the first remaining prediction boxes. This allows for the initial removal of foreground and background boxes, thereby reducing the possibility of false detections of pedestrians.

[0108] After obtaining the first remaining predicted bounding box, the electronic device can determine whether the intersection-union ratio (IU) of each head predicted bounding box in the first remaining predicted bounding box is greater than a second threshold. This allows for the continued removal of overlapping bounding boxes from the first remaining predicted bounding box.

[0109] Optionally, the second threshold is a pre-set intersection-union ratio of the predicted bounding boxes. For example, the second threshold can be 0.3. It should be understood that the second threshold can be determined by experimental testing or theoretical calculation, and the specific value of the second threshold is not limited in the embodiments of this application.

[0110] If so, the electronic device removes the head prediction boxes whose intersection-union ratio (IU) is greater than the second threshold from the first remaining prediction boxes to obtain the first detection box.

[0111] When the first remaining prediction box includes multiple head prediction boxes, in order to obtain a unique first detection box, the electronic device may repeat the above steps to perform the elimination process until a unique first detection box is obtained.

[0112] based on Figure 2 The following description, combined with... Figure 6 This paper details a feasible implementation method of the human body detection model of this application.

[0113] Please see Figure 6 , Figure 6 A schematic block diagram of the human body detection model in an embodiment of this application is shown.

[0114] like Figure 6 As shown, the human body detection model of this application includes, from top to bottom:

[0115] The system consists of an input layer, a first convolutional layer, a first max pooling layer, a first feature extraction layer, a first transition layer, a second feature extraction layer, a second transition layer, a third feature extraction layer, a third transition layer, a second max pooling layer, a fourth feature extraction layer, a second convolutional layer, an upsampling layer, a splicing and fusion layer, a third convolutional layer, a fourth convolutional layer, and an output layer.

[0116] In some embodiments, the input layer is used to input a first image. Convolutional layers (including a first convolutional layer, a second convolutional layer, a third convolutional layer, and a fourth convolutional layer) are used to perform convolution operations on the first image output by the output layer to obtain a convolutional feature map. Max pooling layers (including a first max pooling layer and a second max pooling layer) are used to perform dimensionality reduction processing on the feature maps output by the convolutional layers to obtain dimensionality-reduced feature maps, thereby reducing the dimensionality of the feature maps output by the convolutional layers and improving computational efficiency. Transition layers (including a first transition layer, a second transition layer, and a third transition layer) are used to reduce the number of feature maps output by the convolutional layers, thereby reducing the model complexity of the human detection model. Upsampling layers are used to aggregate the feature maps output by the convolutional layers to obtain aggregated feature maps, which can ensure the integrity of contextual information and reduce the number of feature maps, thereby reducing computational complexity. Concatenation and fusion layers are used to fuse the feature maps output by different convolutional layers to obtain fused feature maps, thereby increasing the dimensionality and diversity of features and improving the performance of the human detection model. The output layer is used to output the detection results.

[0117] In some embodiments, the first feature extraction layer includes two first convolution operators, two second convolution operators, two concatenation and fusion operators, and four activation functions. The second feature extraction layer includes four first convolution operators, four second convolution operators, four concatenation and fusion operators, and eight activation functions. The third feature extraction layer includes two first convolution operators, two second convolution operators, two concatenation and fusion operators, and four activation functions. The fourth feature extraction layer includes two first convolution operators, two second convolution operators, two concatenation and fusion operators, and four activation functions. It should be noted that the order of the above feature extraction layers and the order of the operators within each feature extraction layer are as follows: Figure 6 As shown.

[0118] In some embodiments, the first transition layer includes a first convolution operator, an activation function, and a max pooling operator. The second transition layer includes a first convolution operator, an activation function, and a max pooling operator. The third transition layer includes two first convolution operators, two activation functions, and a max pooling operator. It should be noted that the order of the transition layers and the order of the operators within each transition layer are as follows: Figure 6 As shown.

[0119] Based on the above description, the detection process of the human body detection model will be illustrated below with specific parameters to make the pedestrian detection method in this application embodiment clearer.

[0120] Assume the width of the first image is 128 pixels and the height is 256 pixels. The first image is an RGB image, therefore, the number of input channels (channel, C) of the first image is 3.

[0121] The kernel size of the first convolutional layer is 3*3, the stride of the first convolutional layer is 2, and the padding value of the first convolutional layer is 1.

[0122] The pooling window size of the first max pooling layer is 2*2, the stride of the first max pooling layer is 2, and the padding value is 0.

[0123] The first convolution operator has a kernel size of 1x1. Its stride is 1. Its padding is 0. The second convolution operator has a kernel size of 3x3. Its stride is 1. Its padding is 1.

[0124] The stride of the upsampling layer is 2. Furthermore, the parameters of the max pooling operator (including the pooling window size, stride, and padding) are the same as those of the first max pooling layer. The parameters of the second max pooling layer are the same as those of the first max pooling layer. The parameters of the second convolutional layer (including the kernel size, stride, and padding) are the same as those of the first convolutional operator. The parameters of the third convolutional layer are the same as those of the first convolutional operator. The parameters of the fourth convolutional layer are the same as those of the first convolutional operator, and will not be described further here.

[0125] Based on the above parameters, the pedestrian detection method in this application embodiment includes:

[0126] Step 11: Input a first image with a width of 128 pixels, a height of 256 pixels, and 3 input channels into the input layer of the human detection model, as shown below. Figure 6 The image shown is 128*256*3. Therefore, by inputting an image of a specified size, the inference time of the human detection model can be reduced while maintaining its accuracy.

[0127] Step 12: The first convolutional layer performs a convolution operation on the first image, outputting a feature vector with a width of 64 pixels, a height of 128 pixels, and a number of 24. Figure 1 ,like Figure 6 The figure shown is 64*128*24.

[0128] Step 13, the first max pooling layer on features Figure 1 After dimensionality reduction, the output features are 32 pixels wide, 64 pixels high, and number 24. Figure 2 (i.e., the fifth feature map), such as Figure 6 The figure shown is 32*64*24. The fifth feature map is the feature map input to the first feature extraction layer.

[0129] Step 14, the first feature extraction layer extracts features. Figure 2 Perform feature extraction and output 40 features with a width of 32 pixels, a height of 64 pixels, and a number of features. Figure 3 (i.e., the first feature map), such as Figure 6 The value shown is 32*64*40.

[0130] Specifically, in the first feature extraction layer, the first convolution operator is used to extract features. Figure 2 Perform convolution operations to obtain the features after convolution. Figure 2 And the features after convolution operation using activation functions. Figure 2 A non-linear mapping is performed to obtain a first intermediate image with a width of 32 pixels, a height of 64 pixels, and a number of 24, as shown below. Figure 6The image shown is 32*64*24. The first intermediate image is convolved using the second convolution operator to obtain the convolved first intermediate image. Then, an activation function is used to perform a non-linear mapping on the convolved first intermediate image to obtain a second intermediate image with a width of 32 pixels, a height of 64 pixels, and a number of elements of 8. Figure 6 The figure shown is 32*64*8. The splicing and fusion operator is used to combine the features. Figure 2 That is, the fifth feature map and the second intermediate map are fused to obtain a third intermediate map with a width of 32 pixels, a height of 64 pixels, and a number of 32, as shown below. Figure 6 The image shown is 32*64*32. Then, a convolution operation is performed on the third intermediate image again to obtain the convolved third intermediate image. An activation function is then used to perform a non-linear mapping on the convolved third intermediate image to obtain a fourth intermediate image with a width of 32 pixels, a height of 64 pixels, and a number of 24 pixels, as shown below. Figure 6 The image shown is 32*64*24. The second convolution operator is used to convolve the fourth intermediate image, resulting in the convolved fourth intermediate image. Then, an activation function is used to perform a non-linear mapping on the convolved fourth intermediate image, resulting in a fifth intermediate image with a width of 32 pixels, a height of 64 pixels, and 8 pixels in number, as shown. Figure 6 The image shows 32*64*8. Finally, the third and fifth intermediate images are fused using a splicing and fusion operator to obtain the features. Figure 3 That is, the first feature map.

[0131] Step 15, the first transition layer for features Figure 3 The process outputs a feature with a width of 16 pixels, a height of 32 pixels, and a number of features of 32. Figure 4 ,like Figure 6 The figure shown is 16*32*32.

[0132] Specifically, in the first transition layer, the features are processed using the first convolution operator and activation function. Figure 3 After processing, a middle element with a width of 32 pixels, a height of 64 pixels, and a quantity of 32 is obtained. Figure 1 ,like Figure 6 The diagram shows 32*64*32. Then, the max-pooling operator is used to process the intermediate... Figure 1 Processing is performed to obtain features. Figure 4 .

[0133] Step 16, the second feature extraction layer extracts features. Figure 4 Feature extraction is performed, outputting 64 features with a width of 16 pixels, a height of 32 pixels, and a number of features. Figure 5 (i.e., the second feature map), such as Figure 6 The figure shown is 16*32*64.

[0134] It should be noted that the specific implementation process in the second feature extraction layer can refer to the specific implementation process in the first feature extraction layer, and the output result is as follows. Figure 6 As shown, it will not be elaborated further here.

[0135] Step 17, the second transition layer for features Figure 5 The process outputs 48 features with a width of 8 pixels, a height of 16 pixels, and a total number of features. Figure 6 ,like Figure 6 The figure shown is 8*16*48.

[0136] It should be noted that the specific implementation process in the second transition layer can refer to the specific implementation process in the first transition layer, and the output results are as follows: Figure 6 As shown, it will not be elaborated further here.

[0137] Step 18, the third feature extraction layer extracts features. Figure 6 Perform feature extraction and output 64 features with a width of 8 pixels, a height of 16 pixels, and a number of features. Figure 7 (i.e., the third feature map), such as Figure 6 The figure shown is 8*16*64.

[0138] It should be noted that the specific implementation process in the third feature extraction layer can refer to the specific implementation process in the first feature extraction layer, and the output result is as follows. Figure 6 As shown, it will not be elaborated further here.

[0139] Step 19, the third transition layer for features Figure 7 The process outputs 64 features with a width of 4 pixels, a height of 8 pixels, and a total number of features. Figure 8 ,like Figure 6 The figure shown is 4*8*64.

[0140] It should be noted that the specific implementation process in the third transition layer can refer to the specific implementation process in the first transition layer, and the output results are as follows: Figure 6 As shown, it will not be elaborated further here.

[0141] Step 20, the second max pooling layer on features Figure 8 Dimensionality reduction is performed, outputting features with a width of 2 pixels, a height of 4 pixels, and a number of 64. Figure 9 ,like Figure 6 The figure shown is 2*4*64.

[0142] Step 21, the fourth feature extraction layer extracts features. Figure 9 Feature extraction is performed, outputting feature map 10 (i.e., the fourth feature map) with a width of 2 pixels, a height of 4 pixels, and a number of 80, as shown below. Figure 6 The figure shows 2*4*80.

[0143] It should be noted that the specific implementation process in the fourth feature extraction layer can refer to the specific implementation process in the first feature extraction layer, and the output result is as follows. Figure 6 As shown, it will not be elaborated further here.

[0144] Step 22: The second convolutional layer performs a convolution operation on feature map 10, outputting feature map 11 with a width of 2 pixels, a height of 4 pixels, and a number of 32, as shown below. Figure 6 The figure shows 2*4*32.

[0145] Step 23: The upsampling layer processes feature map 11, outputting feature map 12 with a width of 4 pixels, a height of 8 pixels, and a number of 32, as shown below. Figure 6 The figure shown is 4*8*32.

[0146] Step 24, splicing and fusion layer for features Figure 8 Feature fusion is performed with feature map 12, outputting feature map 13 with a width of 4 pixels, a height of 8 pixels, and a number of 96, as shown below. Figure 6 The figure shown is 4*8*96.

[0147] Step 25: The third convolutional layer performs a convolution operation on feature map 13, outputting feature map 14 with a width of 4 pixels, a height of 8 pixels, and a number of 64, as shown below. Figure 6 The figure shown is 4*8*64.

[0148] Step 26: The fourth convolutional layer performs a convolution operation on feature map 14, outputting feature map 15 with a width of 4 pixels, a height of 8 pixels, and a number of 48, as shown below. Figure 6 The figure shown is 4*8*48.

[0149] Step 27: The output layer processes feature map 15, outputting detection results with a width of 4 pixels, a height of 8 pixels, and a quantity of 48, as shown below. Figure 6 The figure shown is 4*8*48.

[0150] Specifically, the output layer can use a non-maximum suppression algorithm for elimination, thereby ensuring that the detection results contain a unique first detection box, a unique second detection box, and a unique third detection box.

[0151] In summary, compared to current detection methods, the pedestrian detection method in this embodiment uses a human detection model containing only four feature extraction layers, resulting in fewer feature extraction layers and thus enabling faster detection of target pedestrians. Furthermore, the number of feature maps (or output channels) output by the first, second, and fourth feature extraction layers gradually increases. A key difference is that the number of feature maps output by the third feature extraction layer is the same as that output by the second feature extraction layer. This embodiment utilizes multiple feature extraction layers to increase the number of output feature maps, allowing for the extraction of more features from the first image and resulting in more accurate detection results from the human detection model.

[0152] In addition, the human detection model only includes one output layer, eliminating the need to adjust the size of the detection result to match the size of the input first image. This reduces the computational process, further accelerating the inference speed of the human detection model and resulting in higher detection efficiency.

[0153] It should be understood that Figure 6 The human detection model shown is merely an example. Other network structures may also be used for the human detection model in this application, and this application does not limit them.

[0154] Considering that the aforementioned human detection model needs to be trained before actual use, in order to improve the detection accuracy of the human detection model.

[0155] The following is combined Figure 7 and Figure 8 This document details one possible implementation of training a human detection model in the embodiments of this application.

[0156] Please see Figure 7 , Figure 7 A flowchart illustrating the training process of the human detection model in an embodiment of this application is shown. Figure 7 As shown, the specific training process of the human detection model in this embodiment is as follows:

[0157] S201, Obtain the first sample image set.

[0158] Electronic devices can obtain a set of first sample images labeled by annotation software from other electronic devices.

[0159] Optionally, in some embodiments, the annotation software obtains the coordinates of the four vertices of at least one parallelogram and the true category label by drawing a parallelogram for at least one part of each pedestrian in the image and labeling the true category of each pedestrian. For example, the annotation software may be Labelme software.

[0160] It should be noted that manual annotation is required for any omissions in the standard software to ensure the accuracy of the training data and thus guarantee the high detection accuracy of the resulting human detection model.

[0161] After obtaining the first set of sample images, data augmentation can be performed on the first set of sample images to increase the training sample set, thereby improving the training effect of the original human detection model.

[0162] S202, perform data augmentation on the first sample image set to obtain the second sample image set.

[0163] Optionally, data augmentation processing includes one or more of the following methods: symmetry processing, scaling processing, and translation processing. This enriches the training sample set, making the trained human detection model more robust.

[0164] The electronic device can perform data augmentation on the first sample image set to obtain the second sample image set. Furthermore, since the first sample image set has already been labeled, performing symmetry processing, scaling, and translation on the first sample image set will not change the labeling results. Therefore, there is no need to label the second sample image set again, which enriches the training sample set and accelerates the training speed.

[0165] S203, input the first sample image set and the second sample image set into the original human detection model for training to obtain the human detection model.

[0166] like Figure 8 As shown, after obtaining the first sample image set and the second sample image set, the electronic device can input the first sample image set and the second sample image set into the original human detection model to train the original detection model and obtain the trained human detection model.

[0167] Optionally, the original human detection model includes six different sizes of anchor boxes, or prior boxes. These six prior boxes are: a first prior box with a width of 44 pixels and a height of 42 pixels; a second prior box with a width of 64 pixels and a height of 79 pixels; a third prior box with a width of 76 pixels and a height of 102 pixels; a fourth prior box with a width of 113 pixels and a height of 131 pixels; a fifth prior box with a width of 114 pixels and a height of 141 pixels; and a sixth prior box with a width of 117 pixels and a height of 202 pixels. The first and second prior boxes correspond to the pedestrian's head region. The third and fourth prior boxes correspond to the pedestrian's upper body region. The fifth and sixth prior boxes correspond to the pedestrian's lower body region.

[0168] It should be noted that during training, the original human detection model can be trained based on the six prior boxes mentioned above, thereby obtaining the parameters of the loss function and the parameters of each layer in the original human detection model. It should be understood that the loss function can be a commonly used loss function in detection models by those skilled in the art; the specific formula for the loss function will not be elaborated here.

[0169] In summary, electronic devices can complete the training process of the original human detection model to obtain the human detection model.

[0170] Please see Figure 9 , Figure 9 A schematic block diagram of a pedestrian detection device provided in an embodiment of this application is shown. Figure 9 As shown, the pedestrian detection device 400 of this application embodiment may include: a determination module 401 and a detection module 402;

[0171] Determining module 401 is used to determine a first image based on the image to be detected, wherein the first image includes the target pedestrian;

[0172] The detection module 402 is used to input the first image into the human detection model to obtain the detection result; the human detection model is trained based on the training sample set, and the human detection model includes at least a feature extraction layer, wherein the feature extraction layer is used to extract the features of the first image; the feature extraction layer consists of a first feature extraction layer, a second feature extraction layer, a third feature extraction layer and a fourth feature extraction layer;

[0173] The first feature extraction layer is used to output the first feature map, the second feature extraction layer is used to output the second feature map, the third feature extraction layer is used to output the third feature map, and the fourth feature extraction layer is used to output the fourth feature map. The sizes of the first feature map, the second feature map, the third feature map, and the fourth feature map decrease sequentially, and the number of the first feature map, the second feature map, the third feature map, and the fourth feature map increases sequentially.

[0174] The detection results include one or more of the following: a first detection box, a second detection box, and a third detection box. The first detection box is used to identify the head area of ​​the target pedestrian, the second detection box is used to identify the upper body area of ​​the target pedestrian, and the third detection box is used to identify the lower body area of ​​the target pedestrian.

[0175] In some embodiments, the first feature extraction layer includes a first convolution operator, a second convolution operator, an activation function, and a splicing and fusion operator;

[0176] The detection module 402 is further configured to perform a convolution operation on the fifth feature map using a first convolution operator to obtain a fifth feature map after convolution, and to perform a non-linear mapping on the fifth feature map after convolution using an activation function to obtain a first intermediate map; the fifth feature map is the feature map input to the first feature extraction layer; the second convolution operator is used to perform a convolution operation on the first intermediate map to obtain a first intermediate map after convolution, and the activation function is used to perform a non-linear mapping on the first intermediate map after convolution to obtain a second intermediate map; and a concatenation and fusion operator is used to combine the fifth feature map and the second intermediate map. The first intermediate image is obtained by feature fusion. The third intermediate image is then convolved with the first convolution operator to obtain the convolved third intermediate image. The convolutioned third intermediate image is then nonlinearly mapped with an activation function to obtain the fourth intermediate image. The fourth intermediate image is then convolved with the second convolution operator to obtain the convolved fourth intermediate image. The convolutioned fourth intermediate image is then nonlinearly mapped with an activation function to obtain the fifth intermediate image. The first feature map is obtained by feature fusion of the third and fifth intermediate images using a concatenation and fusion operator.

[0177] In some embodiments, the human detection model further includes an input layer, a convolutional layer, a max-pooling layer, a transition layer, an upsampling layer, a concatenation and fusion layer, and an output layer; the input layer is used to input a first image; the convolutional layer is used to perform convolution operations on the first image output by the input layer to obtain a convolutional feature map; the max-pooling layer is used to perform dimensionality reduction processing on the feature map output by the convolutional layer to obtain a dimensionality-reduced feature map; the transition layer is used to reduce the number of feature maps output by the convolutional layer; the upsampling layer is used to perform aggregation processing on the feature maps output by the convolutional layer to obtain an aggregated feature map; the concatenation and fusion layer is used to perform feature fusion on the feature maps output by different convolutional layers to obtain a fused feature map; and the output layer is used to output the detection result.

[0178] In some embodiments, the pedestrian detection device 400 may further include a training module 403, and the training sample set includes a first sample image set and a second sample image set; the first sample image set is a collection of images after annotating the body parts and coordinate information of each body part of each of the multiple pedestrians, and the body parts of the pedestrians include one or more of the following: head, upper body and lower body; the training module 403 is used to acquire the first sample image set; perform data augmentation processing on the first sample image set to obtain the second sample image set, and the data augmentation processing includes one or more of the following processing methods: symmetry processing, scaling processing and translation processing; input the first sample image set and the second sample image set into the original human detection model for training to obtain the human detection model.

[0179] In some embodiments, the detection module 402 is further configured to input the first image into a human detection model to obtain multiple head prediction boxes, multiple upper body prediction boxes, and multiple lower body prediction boxes; to perform a removal process on the multiple head prediction boxes to obtain a first detection box; to perform a removal process on the multiple upper body prediction boxes to obtain a second detection box; and to perform a removal process on the multiple lower body prediction boxes to obtain a third detection box; wherein the detection result includes the first detection box, the second detection box, and the third detection box.

[0180] In some embodiments, the detection module 402 is further configured to determine whether the confidence level of each head prediction box in the plurality of head prediction boxes is less than a first threshold, wherein the first threshold is a preset prediction box confidence value.

[0181] If so, then the head prediction boxes with confidence scores less than the first threshold in the multiple head prediction boxes are removed to obtain the first remaining prediction boxes, which include one or more head prediction boxes.

[0182] Determine whether the intersection-union ratio of each head prediction box in the first remaining prediction box is greater than the second threshold, which is a pre-set intersection-union ratio value for prediction boxes;

[0183] If so, the header prediction boxes with an intersection-union ratio greater than the second threshold in the first remaining prediction boxes are removed to obtain the first detection box.

[0184] In some embodiments, the determining module 401 is further configured to acquire an image to be detected, the image to be detected including multiple pedestrians, the multiple pedestrians including a target pedestrian, each pedestrian having corresponding coordinate information in the image to be detected; and to extract a first image from the image to be detected based on the coordinate information of the target pedestrian.

[0185] In some embodiments, before inputting the first image into the human detection model, the determining module 401 is further configured to adjust the size of the first image to a preset size to obtain the first image after adjustment. The preset size refers to the size of the image input into the human detection model that is set in advance. The first image after adjustment is subjected to mean removal processing and / or normalization processing to obtain the preprocessed first image.

[0186] The detection module 402 is also used to input the preprocessed first image into the human detection model.

[0187] It should be understood that the device 400 in this application embodiment can be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. It can also be implemented using software. Figure 1 The pedestrian detection method shown, when implemented through software... Figure 1 In the pedestrian detection method shown, the device 400 and its various modules can also be software modules.

[0188] This application also provides a readable storage medium storing a computer program, which, when executed by a processor, implements the steps described in the above-described method embodiments.

[0189] This application provides a computer program product that, when run on a mobile device, enables the mobile device to implement the steps described in the various method embodiments above.

[0190] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0191] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0192] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the above device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0193] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0194] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0195] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0196] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0197] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A pedestrian detection method, characterized in that, The method includes: A first image is determined based on the image to be detected, and the first image includes the target pedestrian; The first image is input into a human detection model to obtain a detection result; the human detection model is trained based on a training sample set, and the human detection model includes at least a feature extraction layer, wherein the feature extraction layer is used to extract features of the first image; the feature extraction layer consists of a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, and a fourth feature extraction layer; Wherein, the first feature extraction layer is used to output a first feature map, the second feature extraction layer is used to output a second feature map, the third feature extraction layer is used to output a third feature map, and the fourth feature extraction layer is used to output a fourth feature map; the sizes of the first feature map, the second feature map, the third feature map, and the fourth feature map decrease sequentially, and the number of the first feature map, the second feature map, the third feature map, and the fourth feature map increases sequentially; The detection result includes one or more of the following: a first detection box, a second detection box, and a third detection box, wherein the first detection box is used to identify the head region of the target pedestrian, the second detection box is used to identify the upper body region of the target pedestrian, and the third detection box is used to identify the lower body region of the target pedestrian; The first feature extraction layer includes a first convolution operator, a second convolution operator, an activation function, and a concatenation and fusion operator; the first feature map is obtained through the following method: The fifth feature map is convolved using the first convolution operator to obtain the convolved fifth feature map, and the fifth feature map after convolution is nonlinearly mapped using the activation function to obtain a first intermediate map; the fifth feature map is the feature map input to the first feature extraction layer. The first intermediate image is convolved using the second convolution operator to obtain the first intermediate image after convolution, and the first intermediate image after convolution is nonlinearly mapped using the activation function to obtain the second intermediate image. The fifth feature map and the second intermediate map are fused using the splicing and fusion operator to obtain the third intermediate map; The third intermediate image is convolved using the first convolution operator to obtain the third intermediate image after convolution, and the third intermediate image after convolution is nonlinearly mapped using an activation function to obtain the fourth intermediate image. The fourth intermediate image is convolved using the second convolution operator to obtain the fourth intermediate image after convolution, and the fifth intermediate image is obtained by nonlinear mapping of the fourth intermediate image after convolution using the activation function. The first feature map is obtained by fusing the features of the third intermediate map and the fifth intermediate map using the splicing and fusion operator.

2. The method as described in claim 1, characterized in that, The human detection model also includes an input layer, a convolutional layer, a max pooling layer, a transition layer, an upsampling layer, a splicing and fusion layer, and an output layer; The input layer is used to input the first image; the convolutional layer is used to perform convolution operations on the first image output by the input layer to obtain a convolutional feature map; the max pooling layer is used to perform dimensionality reduction processing on the feature map output by the convolutional layer to obtain a dimensionality-reduced feature map; the transition layer is used to reduce the number of feature maps output by the convolutional layer; the upsampling layer is used to perform aggregation processing on the feature maps output by the convolutional layer to obtain an aggregated feature map; the concatenation and fusion layer is used to perform feature fusion on the feature maps output by different convolutional layers to obtain a fused feature map; and the output layer is used to output the detection result.

3. The method as described in claim 1 or 2, characterized in that, The training sample set includes a first sample image set and a second sample image set; the first sample image set is a collection of images after annotating the body parts and coordinate information of each body part of each pedestrian among multiple pedestrians, and the body parts of the pedestrians include one or more of the following: head, upper body and lower body; The human detection model was obtained using the following training method: Obtain the first sample image set; The first sample image set is subjected to data augmentation processing to obtain the second sample image set. The data augmentation processing includes one or more of the following processing methods: symmetry processing, scaling processing, and translation processing. The first sample image set and the second sample image set are input into the original human detection model for training to obtain the human detection model.

4. The method as described in claim 1 or 2, characterized in that, The step of inputting the first image into the human detection model to obtain the detection result includes: The first image is input into the human body detection model to obtain multiple head prediction boxes, multiple upper body prediction boxes, and multiple lower body prediction boxes; The multiple head prediction boxes are removed to obtain the first detection box; the multiple upper body prediction boxes are removed to obtain the second detection box; and the multiple lower body prediction boxes are removed to obtain the third detection box; wherein, the detection result includes the first detection box, the second detection box, and the third detection box.

5. The method as described in claim 4, characterized in that, The step of removing the multiple predicted head boxes to obtain the first detection box includes: Determine whether the confidence level of each head prediction box in the plurality of head prediction boxes is less than a first threshold, wherein the first threshold is a preset prediction box confidence value; If so, the head prediction boxes with a confidence level less than the first threshold in the plurality of head prediction boxes are removed to obtain the first remaining prediction boxes, which include one or more head prediction boxes. Determine whether the intersection-union ratio of each head prediction box in the first remaining prediction box is greater than a second threshold, where the second threshold is a preset prediction box intersection-union ratio value; If so, the head prediction boxes with an intersection-union ratio greater than the second threshold in the first remaining prediction boxes are removed to obtain the first detection box.

6. The method as described in claim 1 or 2, characterized in that, The process of determining the first image based on the image to be detected includes: The image to be detected is obtained, the image to be detected includes multiple pedestrians, the multiple pedestrians include the target pedestrian, and each pedestrian has corresponding coordinate information in the image to be detected; Based on the coordinate information of the target pedestrian, the first image is extracted from the image to be detected.

7. The method as described in claim 6, characterized in that, Before inputting the first image into the human detection model, the method further includes: The size of the first image is adjusted to a preset size to obtain the first image after adjustment. The preset size refers to the size of the image input into the human body detection model that is set in advance. The first image after resizing is subjected to mean removal and / or normalization to obtain the preprocessed first image; The process of inputting the first image into the human detection model includes: The preprocessed first image is input into the human detection model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1-7.

9. A readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.

Images