A head center point auxiliary-based feature comparison pedestrian detection method and system

Abstract

This invention discloses a pedestrian detection method and system based on head center point-assisted feature comparison. The method includes: constructing a new CSP network architecture by adding a head branch, a feature comparison branch, a detection confidence enhancer, and a head-aware NMS module to the original CSP network architecture; constructing a training sample set, where the head center point is used as the positive sample for the head branch, and non-head center points within the head bounding box are used as the negative sample for the head branch; using the training sample set as input to the CSP network for training to obtain a pedestrian detection network model; supervising network learning during the training phase using head branch loss and feature comparison loss; and enhancing the detection results during the training and testing phases using the detection confidence enhancer and the head-aware NMS method. This invention improves the detection accuracy of CSP and reduces the probability of false positives and false negatives by introducing head center point, feature comparison, detection confidence enhancer, and head-aware NMS methods.

Description

Technical Field

[0001] This invention belongs to the field of pedestrian detection technology, specifically relating to a pedestrian detection method and system based on feature comparison assisted by head center point. Background Technology

[0002] Pedestrian detection is an important research area in computer vision. Its goal is to detect whether there are pedestrians in an image or video and return their location information. It is widely used in fields such as autonomous driving, intelligent security and intelligent robots.

[0003] Currently, there are two main approaches to pedestrian detection. The first is based on traditional computer vision methods that utilize hand-designed features, including Histogram of Oriented Gradients (HOG), Aggregate Channel Feature Function (ACF), and Low-Level Channel Feature Function (LDCF). These methods primarily detect low-level features in images, such as pedestrian color, texture, and edges, focusing on local information between pixels. The second approach is based on deep learning, which uses neural networks for pedestrian detection, such as Beta R-CNN, Pedestron, and F2DNet. These methods focus on high-level semantic features and global information between pixels. In recent years, due to the superior performance of neural networks, deep learning-based methods have become the mainstream approach in pedestrian detection.

[0004] Among numerous pedestrian detection methods, CSP (Center and Scale Prediction) has achieved remarkable results. Its main principle is to use a fully convolutional network to directly detect the high-level semantic feature of the pedestrian's center point in the image, and then add the pedestrian's scale to the detection. It achieves highly accurate and efficient detection results using only a simple network structure. However, this method still has some shortcomings. First, it misidentifies some rectangular objects resembling pedestrians on the roadside, such as tree trunks, road signs, and traffic lights, as pedestrians. Second, its detection performance is poor when pedestrians are occluded. These two factors limit CSP's performance in complex environments, causing it to fall far short of human baseline performance. Summary of the Invention

[0005] This invention addresses the problems of false positives and missed detections in existing pedestrian detection methods (CSP) based on pedestrian center points and scale, which suffer from numerous instances of false positives and missed detections when pedestrians are occluded. To reduce the probability of false positives and missed detections, this invention provides a feature comparison pedestrian detection method and system based on head center point assistance. Specifically, the technical solution of this invention improves the original CSP network architecture by adding a head branch, a feature comparison branch, and a detection confidence enhancer. The head branch helps the network better learn pedestrian features, the feature comparison branch helps the network generate higher-quality bounding boxes, and the detection confidence enhancer enhances the detection confidence of the head's location, significantly reducing the overall accuracy of CSP pedestrian detection.

[0006] Therefore, the present invention provides the following technical solution:

[0007] On one hand, the present invention provides a pedestrian detection method based on head center point-assisted feature comparison, comprising the following steps:

[0008] A new CSP network architecture is constructed, which adds a head branch, a feature comparison branch, and a detection confidence enhancer to the original CSP network architecture. The head branch and the feature comparison branch are both convolutional layers, and are set up in parallel with the pedestrian center point branch and the pedestrian scale branch in the original CSP network architecture. The pedestrian center point heatmap and the scale heatmap obtained based on the pedestrian center point branch and the pedestrian scale branch, as well as the head center point heatmap obtained based on the head branch, are used as inputs to the detection confidence enhancer to enhance the confidence of the pedestrian detection results.

[0009] Construct a training sample set, in which the samples are collected pedestrian images and the pedestrians are labeled. Calculate the head bounding box and head center point based on the pedestrian labels, and use the head center point as the positive sample of the head branch, and the non-head center point in the head bounding box as the negative sample of the head branch.

[0010] The pedestrian detection network model is obtained by training the CSP network based on the new CSP network architecture using the training sample set as input.

[0011] The head branch and the feature comparison branch respectively construct the head branch loss and the feature comparison loss. During the training phase, the head branch loss and the feature comparison loss are used to supervise the network learning.

[0012] Pedestrian detection is performed using a trained pedestrian detection network model.

[0013] The technical solution of this invention adds a detection confidence enhancer after the first detection result and the head branch, receiving the outputs of both as inputs and outputting the detection result with enhanced detection confidence. This detection confidence enhancer can improve the detection of occluded pedestrians, especially in cases of occlusion, where the pedestrian's body is often obscured while the head is less frequently obscured.

[0014] Further optionally, the head branch loss is expressed as:

[0015]

[0016]

[0017]

[0018] In the formula, p ij ∈[0,1] is the probability estimated by the pedestrian detection network model, indicating whether a head center exists at position (i,j), y ij ∈{0,1} represents the true label, y ij =1 indicates a positive sample; α ij γ and β are both focus hyperparameters, and β is a hyperparameter; W and H are the width and height of the input image; M is a custom symbol defined for the above formula. ij is the Gaussian mask for the non-head center point (i,j) in the head bounding box, and K represents the number of heads in the image.

[0019] Alternatively, the process of calculating the head center point based on pedestrian annotations is as follows:

[0020] Construct a head frame on the pedestrian frame based on the pedestrian standard;

[0021] Calculate the center point of the head frame as the head center point;

[0022] Next, calculate the Gaussian mask for points outside the head center within the head bounding box to mitigate the ambiguity caused by positive samples being surrounded by negative samples during training; wherein the Gaussian mask is calculated according to the following formula:

[0023]

[0024]

[0025] Where K represents the number of heads in the image, x k ,y k Let w be the coordinates of the k-th head center point where position (i,j) is located. k ,h k Let σ be the width and height of the head frame of the k-th head. wk,σ hk ) represents the variance of the Gaussian mask corresponding to the k-th header, which is proportional to the height and width of the header box; e is the base.

[0026] It should be understood that determining the head center point based on the above method does not require additional head annotation in the dataset; that is, the head center point annotation can be automatically generated based on the existing pedestrian bounding boxes.

[0027] Further, optionally, the feature comparison branch calculates the feature cosine similarity between the predicted bounding box and the ground truth bounding box corresponding to the pedestrian detection result, and applies the feature cosine similarity as a focus weight to the regression box loss, in the following form:

[0028] L sim =μ×L reg

[0029] μ = -(1-sim) γ log(sim)

[0030] Where sim is the feature cosine similarity between the predicted bounding box and the ground truth bounding box, γ is the focus parameter, μ is the focus weight parameter of the feature contrast loss function, and L is the focus weight parameter of the feature contrast loss. reg The regression loss is for the predicted bounding box.

[0031] Further optionally, the enhancement formula of the detection confidence enhancer is expressed as:

[0032]

[0033] Among them, s new To enhance the detection confidence, s ori β is the original detection confidence level, α is the set head center point detection threshold, and α is the detection confidence weighting parameter (e.g., setting it to 3 yields the best results). head The confidence level of the head center point;

[0034] If the confidence level of the head center point in the detection frame is determined to be greater than or equal to the head center point detection threshold β, then the detection confidence level of the pedestrian detection result is enhanced. That is, the detection confidence enhancer is used to enhance the detection score of the pedestrian and obtain a new pedestrian detection result.

[0035] It should be noted that the pedestrian bounding box is calculated from the pedestrian center point heatmap and the scale heatmap. During detection, a detection threshold for the pedestrian center point is set. The network will output the pedestrian center point on the center point heatmap at a position higher than the detection threshold, and output the height of the pedestrian at the corresponding position on the scale heatmap. Then, the position of the pedestrian bounding box is calculated using the relationship of pedestrian width:height = 0.41.

[0036] In addition, the network outputs the head center point for locations exceeding the head center point detection threshold. If a detection box outputs both the pedestrian center point and the head center point, it indicates a very high probability that the location is a pedestrian. Therefore, a detection confidence enhancer is used to increase the detection confidence for pedestrians.

[0037] Specifically, the detection confidence enhancer receives the first detection result and the head center point heatmap as input. It first determines whether there is a head center point with high confidence (e.g., β is set to 0.4) at the position of the head center point in the detection frame (e.g., set to (0.5w, 0.075h), where w and h are the width and height of the detection frame, respectively). If there is, the confidence of the pedestrian detection result is enhanced; otherwise, no enhancement is performed.

[0038] Further optionally, the new CSP network architecture also includes a head-aware NMS module following the detection confidence enhancer;

[0039] The enhanced confidence detection results are input into the head perception NMS module. Then, based on the number of head center points detected within the pedestrian detection box that are greater than the head center point detection threshold, the NMS threshold of each pedestrian is dynamically adjusted. The pedestrian detection results are then updated using the non-maximum suppression NMS algorithm. The updated NMS threshold is expressed as follows:

[0040]

[0041] Among them, NMS new n is the pedestrian NMS threshold. head This represents the number of head center points detected within the current pedestrian detection box whose confidence level is greater than the head center point detection threshold. γ1 is the NMS threshold weighting parameter (a value of 4 achieves the best results).

[0042] Specifically, the head-sensing NMS module receives the enhanced detection results as input, and then dynamically sets an NMS threshold for each pedestrian based on the number of head center points detected within the pedestrian detection box that are greater than the head center point detection threshold. This solves the problem that some pedestrians are incorrectly suppressed by the original NMS algorithm due to pedestrian congestion in crowd scenes.

[0043] Further optionally, the new architecture of the CSP network is as follows: the input image is processed by the backbone network ResNet to form a feature map. The feature map is processed by a 3×3×256 convolution and then connected to four branches. Each of the four branches is composed of a 1×1 convolution, namely the pedestrian center point branch, the pedestrian scale branch, the head branch, and the feature comparison branch. The head center point branch and the pedestrian detection results are connected to the detection confidence enhancer. The detection confidence enhancer is then connected to the NMS module for facial perception to form the final detection result.

[0044] The pedestrian center point branch and pedestrian scale branch generate detection results for pedestrian center point heatmap and scale heatmap;

[0045] The head branch generates a head center point heatmap; then it is connected to a detection confidence enhancer and a head perception NMS module. The pedestrian center point heatmap, the scale heatmap obtained based on the pedestrian center point branch and the pedestrian scale branch, and the head center point heatmap obtained based on the head branch are all inputs to the detection confidence enhancer, and the head perception NMS module is connected after the detection confidence enhancer.

[0046] The initial detection results and the head center point heatmap are combined with the detection confidence enhancer and then processed by the NMS algorithm of head perception to form the final detection result.

[0047] Secondly, the present invention provides a detection system for pedestrian detection based on a head center point-assisted feature comparison method, comprising:

[0048] The network construction module is used to construct a new CSP network architecture. The new CSP network architecture adds a head branch, a feature comparison branch, and a detection confidence enhancer to the original CSP network architecture. The head branch and the feature comparison branch are both convolutional layers and are set up in parallel with the pedestrian center point branch and the pedestrian scale branch in the original CSP network architecture. The pedestrian center point heatmap and the scale heatmap obtained based on the pedestrian center point branch and the pedestrian scale branch, as well as the head center point heatmap obtained based on the head branch, are used as inputs to the detection confidence enhancer to enhance the confidence of the pedestrian detection results.

[0049] The sample construction module is used to construct a training sample set, in which the sample consists of collected pedestrian images and the pedestrians are labeled. The head bounding box and the head center point are calculated based on the pedestrian labels. The head center point is used as the positive sample of the head branch, and the non-head center point in the head bounding box is used as the negative sample of the head branch.

[0050] The training module is used to train a pedestrian detection network model by inputting the training sample set into the CSP network based on the new CSP network architecture.

[0051] The head branch and the feature comparison branch respectively construct the head branch loss and the feature comparison loss. During the training phase, the head branch loss and the feature comparison loss are used to supervise the network learning.

[0052] The detection module is used to detect pedestrians using the trained pedestrian detection network model.

[0053] In three aspects, the present invention provides an electronic terminal that includes at least one or more processors;

[0054] A memory that stores one or more computer programs;

[0055] The processor invokes the computer program to execute:

[0056] Steps of a feature comparison pedestrian detection method based on head center point assistance.

[0057] In four aspects, the present invention provides a readable storage medium storing a computer program, which is invoked by a processor for execution:

[0058] Steps of a feature comparison pedestrian detection method based on head center point assistance.

[0059] Beneficial effects

[0060] Compared with existing methods, the advantages of the present invention are:

[0061] This invention provides a pedestrian detection method and system based on head center point-assisted feature comparison. It introduces a head branch and a feature comparison branch. The head branch assists the network in better learning pedestrian features, while the feature comparison branch helps the network generate higher-quality regression boxes. The addition of the head branch allows the network to focus on feature regions containing the head, reducing the probability of the original CSP network misidentifying objects such as tree trunks and traffic lights as pedestrians, thus eliminating false detections of such objects. Furthermore, this invention includes a detection confidence enhancer to improve pedestrian detection results where the head is present. This is particularly useful in occlusion situations, where the pedestrian's body is often obscured while the head is less frequently obscured; therefore, this detection confidence enhancer can improve the detection of occluded pedestrians.

[0062] The head branch and feature comparison branch set in this invention generate head branch loss and feature comparison loss, which are used to supervise the network learning during training. The final pedestrian bounding box detection result is still generated by the pedestrian center point and pedestrian scale calculation, which is unrelated to these two branches. The detection confidence enhancer is just a linear calculation, which requires very low computational cost. Therefore, the technical solution of this invention can maintain the original inference speed, while making the network pay more attention to the area where head features exist and strengthen the learning of pedestrian features under occlusion, thereby improving the final detection result.

[0063] In a further preferred embodiment of the present invention, a head-aware NMS algorithm is introduced after the confidence enhancer to eliminate the false suppression of crowded pedestrians by the ordinary NMS algorithm. In crowded scenes, pedestrians often occlude each other, creating congestion. The ordinary NMS algorithm simply sets a fixed NMS threshold (usually 0.5), which suppresses many crowded pedestrian detection boxes. However, the head-aware NMS algorithm proposed in this invention dynamically sets an NMS threshold for each pedestrian by judging the number of heads detected within a pedestrian detection box (representing the number of crowded pedestrians), thereby eliminating false detections of crowded pedestrian detection boxes. Moreover, the head-aware NMS algorithm only adds one step of calculating the NMS threshold compared to the original NMS algorithm, and the additional inference time is almost negligible.

[0064] In a further preferred embodiment of the invention, a Gaussian mask is designed to reduce the ambiguity caused by negative samples in the head bounding box to positive samples. This is because it is difficult to precisely specify the center point of a head bounding box, and rigidly specifying positive and negative samples would complicate training. Furthermore, the technical solution of this invention multiplies the loss function of the feature comparison branch by a weight related to feature similarity before the regression loss. This penalizes more regressive bounding boxes with low similarity to the ground truth bounding box (this occurs more frequently under occlusion), thereby strengthening the network's learning of them. Attached Figure Description

[0065] Figure 1 This is a schematic diagram of pedestrian image detection provided in an embodiment of the present invention;

[0066] Figure 2 This is a schematic diagram of the new CSP network architecture provided by the present invention;

[0067] Figure 3 This is a flowchart illustrating the detection method provided in an embodiment of the present invention. Detailed Implementation

[0068] This invention provides a method and system for pedestrian detection based on head center point-assisted feature comparison. This method integrates the head center point into pedestrian detection, adding a head branch, a feature comparison branch, and a detection confidence enhancer to the original CSP network architecture. It utilizes the head branch loss and feature comparison loss of the head branch and feature comparison branch to supervise network learning during the training phase, thereby improving the model's detection accuracy. Furthermore, a head-aware NMS module is introduced after the detection confidence enhancer. The invention will be further described below with reference to embodiments.

[0069] Example 1:

[0070] like Figure 3As shown in the figure, this embodiment provides a pedestrian detection method based on head center point-assisted feature comparison, which includes the following steps:

[0071] Step 1: Build a new CSP network architecture.

[0072] like Figure 2 As shown, the novel CSP network architecture of this invention adds a head branch, a feature comparison branch, and a detection confidence enhancer to the original CSP network architecture. The head branch and feature comparison branch are both Conv 1×1 convolutional layers, and are set up alongside the pedestrian center point branch and pedestrian scale branch in the original CSP network architecture. The pedestrian center point heatmap and scale heatmap obtained based on the pedestrian center point branch and pedestrian scale branch, as well as the head center point heatmap obtained based on the head branch, are used as inputs to the detection confidence enhancer to enhance the confidence of the pedestrian detection results. Furthermore, a head-aware NMS module is set after the detection confidence enhancer to eliminate the erroneous suppression of crowded pedestrians by the ordinary NMS algorithm.

[0073] like Figure 2 As shown, the novel CSP network architecture provided in this embodiment of the invention is as follows: the input image is processed by the backbone network ResNet to form a feature map. The feature map is then processed by a 3×3×256 convolution followed by four branches, each consisting of a 1×1 convolution: a pedestrian center point branch, a pedestrian scale branch, a head branch, and a feature comparison branch. The pedestrian center point branch predicts the center point of the pedestrian in the image, the pedestrian scale branch predicts the height of the pedestrian, the head branch assists the network in better learning pedestrian features, and the feature comparison branch helps the network generate higher-quality regression boxes. The pedestrian center point branch and the pedestrian scale branch generate detection results for the pedestrian center point heatmap and scale heatmap, respectively. The head branch generates a head center point heatmap, which is then connected to a detection confidence enhancer and a head-aware NMS module. The pedestrian center point heatmap, scale heatmap, and head center point heatmap obtained based on the pedestrian center point branch and pedestrian scale branch, respectively, are all inputs to the detection confidence enhancer. The head-aware NMS module is then connected after the detection confidence enhancer.

[0074] In some embodiments, a CSP network is provided with a head branch, a feature comparison branch, and a detection confidence enhancer, but without a head-aware NMS module, which also meets the application requirements of the present invention and falls within the protection scope of the present invention.

[0075] Step 2: Construct a training sample set, in which the samples are collected pedestrian images and the pedestrians are labeled. The head center point is calculated based on the pedestrian labeling, and the head center point is used as the positive sample of the head branch, while the non-head center point in the head box is used as the negative sample of the head branch.

[0076] The training sample sets used in this embodiment are Caltech and Cityperson, which collect pedestrian information in daily life. They only annotate the pedestrians in the images, that is, the coordinate information of the upper left and lower right corners of the pedestrian's full-body bounding box.

[0077] Based on this, the present invention automatically generates the head frame annotation using the ratio of a pedestrian's head to their whole body. For example, in this embodiment, points (0, 0.33w) and (0.67w, 0.15h) are taken on the pedestrian frame as the upper left and lower right vertices of the head frame, thus constructing the head frame. It should be understood that the above values ​​are empirical values. In other feasible embodiments, these values ​​can be adaptively adjusted according to accuracy requirements, such as (0, 0.2w), (0.8w, 0.2h); (0, 0.4w), and (0.8w, 0.1h).

[0078] After obtaining the head bounding box, the position of the head center point is calculated using the information of the head bounding box during the training process. This position is used as the positive sample of the head center point branch, while other positions in the image are negative samples of the head center point branch.

[0079] To mitigate the ambiguity caused by negative samples in the head bounding box to positive samples, since it's difficult to precisely identify the center point of a head, and rigidly specifying positive and negative samples would hinder training, this invention employs a Gaussian mask. Specifically, the Gaussian mask is applied to the portion of the head bounding box excluding the head center point, i.e., the negative sample areas. First, the Gaussian mask value for each negative sample in the head bounding box is calculated, and then this value is used as the weight (in the expression (1-M)). ij The β) is multiplied in the loss of negative samples to reduce the penalty of negative samples around positive samples, reduce the ambiguity of positive samples caused by negative samples in the head box, and make the network easier to train.

[0080] The Gaussian mask is represented as:

[0081]

[0082]

[0083] Among them, M ij For non-head positions (i,j), use a Gaussian mask, where K is the number of heads in the image, and x... k ,y k Let w be the coordinates of the k-th head center point. k ,h k Let σ be the width and height of the head frame of the k-th head. wk ,σ hk ) represents the variance of the Gaussian mask corresponding to the k-th header, which is proportional to the height and width of the header box; e is the base.

[0084] It should be noted that when two header boxes overlap, the Gaussian mask of the overlapping part will have two calculated values, and the larger of the two values ​​will be used as the Gaussian mask value.

[0085] Step 3: Use the training sample set as input to train the CSP network based on the new CSP network architecture to obtain the pedestrian detection network model.

[0086] The technical solution of this invention constructs a head branch loss and a feature comparison loss in the head branch and feature comparison branch, respectively. During the training phase, the head branch loss and the feature comparison loss are used to supervise the network learning.

[0087] The head branch loss in this embodiment is set as follows:

[0088]

[0089]

[0090]

[0091] In the formula, p ij ∈[0,1] is the network estimated probability, indicating whether position (i,j) is the head center, y ij ∈{0,1} represents the true label, y ij =1 indicates a positive sample; α ij Both α and γ are focal hyperparameters; in this embodiment, γ is 2. β is a hyperparameter used to control the degree of penalty; in this embodiment, it is set to 4. For positive sample points, α... ij Set to 1, where W and H are the width and height of the input image; Custom symbols defined for the above formula.

[0092] In this embodiment, the feature comparison branch calculates the feature cosine similarity between the predicted bounding box and the ground truth bounding box, and applies the feature cosine similarity as the focus weight to the regression box loss, in the following form:

[0093] L sim =μ×L reg

[0094] μ = -(1-sim) γ log(sim)

[0095] Where sim is the feature cosine similarity between the predicted bounding box and the ground truth bounding box, γ is the focus parameter, μ is the focus weight parameter of the feature contrast loss function, and L is the focus weight parameter of the feature contrast loss. reg The regression loss is used for predicting the bounding box. This embodiment uses the Smooth L1 loss, specifically:

[0096]

[0097] In the formula, x is the logarithmic value of the regressed pedestrian height.

[0098] In this embodiment, the weights of each part of the total loss function are adjusted so that the loss of each branch contributes differently to achieve the best result. The final weights for the pedestrian center point branch, pedestrian scale branch, head center point branch, and feature comparison branch are determined to be 0.01:1:0.005:0.1. It should be understood that the technical solution of this invention does not restrict the construction form of the total loss function. This invention is based on the loss function and continuously updates the network model parameters during training.

[0099] It should be understood that the head branch and feature comparison branch provided by the technical solution of this invention only participate in the training phase and not in the inference phase. These two branches generate two loss functions respectively, which supervise the network to better learn pedestrian features during training. In the inference phase, the detection results of pedestrian bounding boxes are directly generated using the pedestrian center point branch and the pedestrian scale branch, without the need for the head branch and feature comparison branch to generate detection results.

[0100] Regarding the detection confidence enhancer: The detection confidence enhancer receives the initial detection result and the head center point heatmap as input. It first determines whether a head center point with high confidence (e.g., β is set to 0.4) exists at the position of the head center point within the detection frame (e.g., set to (0.5w, 0.075h), where w and h are the width and height of the detection frame, respectively). If such a head center point exists, the confidence of the pedestrian detection result is enhanced; otherwise, no enhancement is performed. Specifically:

[0101]

[0102] Among them, s new To enhance the detection confidence, s ori β is the original detection confidence level, α is the set head center point detection threshold, and α is the detection confidence weighting parameter (e.g., setting it to 3 yields the best results). head The confidence level of the head center point.

[0103] It should be understood that after the detection confidence is enhanced, the corresponding detection results will be adaptively changed. That is, the detection confidence enhancer set in this invention is used to enhance the detection score of pedestrians. If the detection score of a pedestrian is greater than the detection threshold set by the network, it is considered that a pedestrian has been detected.

[0104] Regarding the head-sensing NMS module: The non-maximum suppression (NMS) algorithm is used to adjust the pedestrian detection results. It should be noted that using the NMS algorithm in object detection is existing technology, and this invention will not describe its implementation process in detail. Unlike existing NMS algorithms that use a fixed threshold, this invention optimizes the NMS threshold, specifically in the head-sensing NMS module after the confidence enhancer. The enhanced detection results are input into the head-sensing NMS module, and the NMS threshold for each pedestrian is dynamically adjusted based on the number of head center points detected within the pedestrian detection box that exceed the head center point detection threshold, thereby updating the detection results. The updated NMS threshold is expressed as follows:

[0105]

[0106] Among them, NMS new n is the pedestrian NMS threshold. head γ1 represents the number of head center points detected within the current pedestrian detection box whose confidence level is greater than the head center point detection threshold, and γ1 is the NMS threshold weighting parameter (a value of 4 achieves the best results). Therefore, using the above algorithm, the enhanced detection results are received as input, and then an NMS threshold is dynamically set for each pedestrian based on the number of head center points detected within the pedestrian detection box that are greater than the head center point detection threshold. This solves the problem of some pedestrians being incorrectly suppressed by the original NMS algorithm in crowded scenes, resulting in more accurate pedestrian detection results.

[0107] Step 4: Perform pedestrian detection using the trained pedestrian detection network model. The input to the pedestrian detection network model is an image, and the output is a heatmap of the pedestrian's center point and a heatmap of the pedestrian's scale.

[0108] Example 2:

[0109] The present invention also provides a detection system for pedestrian detection based on a feature comparison method assisted by head center point, comprising: a network construction module, a sample construction module, a training module, and a detection module.

[0110] The network construction module is used to construct a new CSP network architecture. The new CSP network architecture adds a head branch, a feature comparison branch, and a detection confidence enhancer to the original CSP network architecture. The head branch and the feature comparison branch are both convolutional layers and are set up in parallel with the pedestrian center point branch and the pedestrian scale branch in the original CSP network architecture. The pedestrian center point heatmap and the scale heatmap obtained based on the pedestrian center point branch and the pedestrian scale branch, as well as the head center point heatmap obtained based on the head branch, are used as inputs to the detection confidence enhancer to enhance the confidence of the pedestrian detection results.

[0111] The sample construction module is used to construct a training sample set, in which the samples are collected pedestrian images and the pedestrians are labeled. The head bounding box and the head center point are calculated based on the pedestrian labels. The head center point is used as the positive sample of the head branch, and the non-head center point in the head bounding box is used as the negative sample of the head branch.

[0112] The training module is used to train a pedestrian detection network model using the training sample set input based on the new CSP network architecture; wherein, the head branch and the feature comparison branch respectively construct the head branch loss and the feature comparison loss, and the head branch loss and the feature comparison loss are used to supervise network learning during the training phase; the detection module is used to perform pedestrian detection using the trained pedestrian detection network model.

[0113] For details on the implementation process of each module, please refer to the methods described above; they will not be repeated here. It should be understood that the above division of functional modules is merely a logical functional division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the integrated units described above can be implemented in hardware or as software functional units.

[0114] Example 3:

[0115] The present invention provides an electronic terminal comprising at least one or more processors and a memory storing one or more computer programs; wherein the processor invokes the computer programs to execute the steps of a feature comparison pedestrian detection method based on head center point assistance.

[0116] Specific implementation:

[0117] Step 1: Construct a new CSP network architecture. The CSP network architecture is described in detail in Example 1.

[0118] Step 2: Construct a training sample set, in which the samples are collected pedestrian images and the pedestrians are labeled. The head center point is calculated based on the pedestrian labeling, and the head center point is used as the positive sample of the head branch, while the non-head center point in the head box is used as the negative sample of the head branch.

[0119] Step 3: Use the training sample set as input to train the CSP network based on the new CSP network architecture to obtain the pedestrian detection network model.

[0120] Step 4: Perform pedestrian detection using the trained pedestrian detection network model. The input to the pedestrian detection network model is an image, and the output is a heatmap of the pedestrian's center point and a heatmap of the pedestrian's scale.

[0121] Please refer to the explanation of the method above for the specific implementation process of each step.

[0122] It should be understood that, in the embodiments of the present invention, the processor may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. The memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of the memory may also include non-volatile random access memory. For example, the memory may also store device type information.

[0123] It should be emphasized that the examples described in this invention are illustrative rather than limiting. Therefore, this invention is not limited to the examples described in the specific embodiments. Any other embodiments derived by those skilled in the art based on the technical solutions of this invention, without departing from the spirit and scope of this invention, whether modifications or substitutions, are also within the protection scope of this invention.

[0124] Example 4:

[0125] The present invention provides a readable storage medium storing a computer program that is invoked by a processor to execute the steps of a feature comparison pedestrian detection method based on head center point assistance.

[0126] Specific implementation:

[0127] Step 1: Construct a new CSP network architecture. The CSP network architecture is described in detail in Example 1.

[0128] Step 2: Construct a training sample set, in which the samples are collected pedestrian images and the pedestrians are labeled. The head center point is calculated based on the pedestrian labeling, and the head center point is used as the positive sample of the head branch, while the non-head center point in the head box is used as the negative sample of the head branch.

[0129] Step 3: Use the training sample set as input to train the CSP network based on the new CSP network architecture to obtain the pedestrian detection network model.

[0130] Step 4: Perform pedestrian detection using the trained pedestrian detection network model. The input to the pedestrian detection network model is an image, and the output is a heatmap of the pedestrian's center point and a heatmap of the pedestrian's scale.

[0131] Please refer to the explanation of the method above for the specific implementation process of each step.

[0132] The readable storage medium is a computer-readable storage medium, which can be an internal storage unit of the controller described in any of the foregoing embodiments, such as the controller's hard drive or memory. The readable storage medium can also be an external storage device of the controller, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the controller. Further, the readable storage medium can include both the controller's internal storage unit and external storage devices. The readable storage medium is used to store the computer program and other programs and data required by the controller. The readable storage medium can also be used to temporarily store data that has been output or will be output.

[0133] Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0134] It should be emphasized that the examples described in this invention are illustrative rather than limiting. Therefore, this invention is not limited to the examples described in the specific embodiments. Any other embodiments derived by those skilled in the art based on the technical solutions of this invention, without departing from the spirit and scope of this invention, whether modifications or substitutions, are also within the protection scope of this invention.

Claims

1. A pedestrian detection method based on head center point-assisted feature comparison, characterized in that: Includes the following steps: A new CSP network architecture is constructed, which adds a head branch, a feature comparison branch, and a detection confidence enhancer to the original CSP network architecture. The head branch and the feature comparison branch are both convolutional layers, and are set up in parallel with the pedestrian center point branch and the pedestrian scale branch in the original CSP network architecture. The pedestrian center point heatmap and the scale heatmap obtained based on the pedestrian center point branch and the pedestrian scale branch, as well as the head center point heatmap obtained based on the head branch, are used as inputs to the detection confidence enhancer to enhance the confidence of the pedestrian detection results. The enhancement formula for the detection confidence enhancer is expressed as follows: ； in, To enhance the detection confidence, The original detection confidence level, The set threshold for detecting the head center point. To detect the confidence weighting parameter, The confidence level of the head center point; If the detection confidence enhancer determines that the confidence level of the head center point in the detection frame is greater than or equal to the head center point detection threshold, then... This enhances the detection confidence of pedestrians, resulting in new pedestrian detection results. Construct a training sample set, in which the samples are collected pedestrian images and the pedestrians are labeled. Calculate the head bounding box and head center point based on the pedestrian labels, and use the head center point as the positive sample of the head branch, and the non-head center point in the head bounding box as the negative sample of the head branch. The pedestrian detection network model is obtained by training the CSP network based on the new CSP network architecture using the training sample set as input. The head branch and the feature comparison branch respectively construct the head branch loss and the feature comparison loss. During the training phase, the head branch loss and the feature comparison loss are used to supervise the network learning. The feature comparison branch calculates the feature cosine similarity between the predicted box and the real box corresponding to the pedestrian detection result, and applies the feature cosine similarity as the focus weight to the regression box loss. Pedestrian detection is performed using a trained pedestrian detection network model.

2. The method according to claim 1, characterized in that: The head branch loss is expressed as: ； ； In the formula, This is the probability estimated by the pedestrian detection network model, indicating whether a head center exists at position (i,j). Indicates the true label, Indicates a positive sample; γ and β are both focus hyperparameters, and β is a hyperparameter; W and H are the width and height of the input image; The custom symbols defined for the above formula, is the Gaussian mask for the non-head center point (i,j) in the head bounding box, and K represents the number of heads in the image.

3. The method according to claim 2, characterized in that: The process of calculating the head center point based on pedestrian annotation is as follows: Construct a head frame on the pedestrian frame based on the pedestrian standard; Calculate the center point of the head frame as the head center point; Next, calculate the Gaussian mask for the non-head center point within the header frame; wherein the Gaussian mask is calculated according to the following formula: ； ； Where K represents the number of heads in the image. Let (i,j) be the coordinates of the k-th head center point. Let the width and height of the head frame of the k-th head be denoted as . is the variance of the Gaussian mask corresponding to the k-th header, which is proportional to the height and width of the header box; e is the base.

4. The method according to claim 1, characterized in that: The feature contrast loss function takes the following form: ； ； Where sim is the feature cosine similarity between the predicted bounding box and the ground truth bounding box, and γ is the focus parameter. For feature contrast loss function, The focus weight parameters are for the feature contrast loss. The regression loss is for the predicted bounding box.

5. The method according to claim 1, characterized in that: The new CSP network architecture also includes a head-sensing NMS module following the detection confidence enhancer. The enhanced confidence detection results are input into the head perception NMS module. Then, based on the number of head center points detected within the pedestrian detection box that are greater than the head center point detection threshold, the NMS threshold of each pedestrian is dynamically adjusted. The pedestrian detection results are then updated using the non-maximum suppression NMS algorithm. The updated NMS threshold is expressed as follows: ； in, n is the pedestrian NMS threshold. head This represents the number of head center points detected within the current pedestrian detection box whose confidence level is greater than the head center point detection threshold. The threshold weighting parameter for NMS.

6. The method according to claim 1, characterized in that: The new architecture of the CSP network is as follows: the input image is processed by the backbone network ResNet to form a feature map. The feature map is processed by a 3×3×256 convolution and then connected to four branches. Each of the four branches is composed of a 1×1 convolution, namely the pedestrian center point branch, the pedestrian scale branch, the head branch, and the feature comparison branch. The pedestrian center point branch and pedestrian scale branch generate the detection results of pedestrian center point heat map and scale heat map; The head branch generates a heatmap of the head center point; Subsequently, the detection confidence enhancer and the head perception NMS module are connected. The pedestrian center point heatmap, the scale heatmap obtained based on the pedestrian center point branch and the pedestrian scale branch, and the head center point heatmap obtained based on the head branch are all inputs to the detection confidence enhancer. After the detection confidence enhancer, the head perception NMS module is connected, and the head perception NMS module outputs the final result.

7. A detection system based on the feature comparison pedestrian detection method assisted by head center point as described in any one of claims 1-6, characterized in that: include: The network construction module is used to construct a new CSP network architecture. The new CSP network architecture adds a head branch, a feature comparison branch, and a detection confidence enhancer to the original CSP network architecture. The head branch and the feature comparison branch are both convolutional layers and are set up in parallel with the pedestrian center point branch and the pedestrian scale branch in the original CSP network architecture. The pedestrian center point heatmap and the scale heatmap obtained based on the pedestrian center point branch and the pedestrian scale branch, as well as the head center point heatmap obtained based on the head branch, are used as inputs to the detection confidence enhancer to enhance the confidence of the pedestrian detection results. The sample construction module is used to construct a training sample set, in which the sample consists of collected pedestrian images and the pedestrians are labeled. The head bounding box and the head center point are calculated based on the pedestrian labels. The head center point is used as the positive sample of the head branch, and the non-head center point in the head bounding box is used as the negative sample of the head branch. The training module is used to train a pedestrian detection network model by inputting the training sample set into the CSP network based on the new CSP network architecture. The head branch and the feature comparison branch respectively construct the head branch loss and the feature comparison loss. During the training phase, the head branch loss and the feature comparison loss are used to supervise the network learning. The detection module is used to detect pedestrians using the trained pedestrian detection network model.

8. An electronic terminal, characterized in that: At least includes: One or more processors; A memory that stores one or more computer programs; The processor invokes the computer program to execute: The steps of the feature comparison pedestrian detection method based on head center point assistance as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: The computer program is stored and is invoked by the processor for execution. The steps of the feature comparison pedestrian detection method based on head center point assistance as described in any one of claims 1-6.

Images