Method and apparatus for detecting dense pedestrians

Abstract

This application discloses a method and apparatus for dense pedestrian detection, belonging to the field of pedestrian detection technology. The method includes: acquiring a dense pedestrian image to be detected; inputting the dense pedestrian image into a feature layer to obtain a dense pedestrian feature map output by the feature layer; inputting the dense pedestrian feature map into a detection head, performing bounding box regression processing through a first detection structure of the detection head, and predicting the detection target confidence level through a second detection structure of the detection head; correcting the output result of the second detection structure based on the output result of the first detection structure to obtain a bounding box regression feature map and a detection target confidence feature map output by the detection head; inputting the bounding box regression feature map and the detection target confidence feature map into an output layer to obtain the pedestrian detection result output by the output layer. This method can improve detection accuracy and detection rate, and prevent the loss of detection targets.

Description

Technical Field

[0001] This application belongs to the field of pedestrian detection technology, and in particular relates to a method and apparatus for dense pedestrian detection. Background Technology

[0002] Pedestrian detection is a technology that uses computer vision to determine whether there are pedestrians in an image or video sequence and to locate them accurately. It can be combined with pedestrian tracking, pedestrian re-identification and other technologies, and is widely used in fields such as intelligent video surveillance, artificial intelligence systems, vehicle driver assistance systems, intelligent robots, human behavior analysis and intelligent transportation.

[0003] However, in densely populated areas, conventional pedestrian detection algorithms suffer from low detection accuracy, easy loss of targets, and high false negative rates. Summary of the Invention

[0004] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a dense pedestrian detection method and apparatus, which can effectively improve detection accuracy, prevent the loss of detection targets, and increase the detection rate.

[0005] Firstly, this application provides a method for dense pedestrian detection, the method comprising:

[0006] Obtain dense pedestrian images to be detected;

[0007] The dense pedestrian image is input into the dense pedestrian detection model to obtain the pedestrian detection results output by the dense pedestrian detection model;

[0008] The dense pedestrian detection model includes a feature layer, a detection head, and an output layer. The step of inputting the dense pedestrian image into the dense pedestrian detection model to obtain the pedestrian detection result output by the dense pedestrian detection model further includes:

[0009] The dense pedestrian image is input into the feature layer to obtain the dense pedestrian feature map output by the feature layer;

[0010] The dense pedestrian feature map is input into the detection head, the detection box regression is performed through the first detection structure of the detection head, and the confidence of the detected target is predicted through the second detection structure of the detection head.

[0011] Based on the output of the first detection structure, the output of the second detection structure is corrected to obtain the detection box regression feature map and the detection target confidence feature map output by the detection head;

[0012] The regression feature map of the detection box and the confidence feature map of the detected target are input into the output layer to obtain the pedestrian detection result output by the output layer.

[0013] According to the dense pedestrian detection method of this application, the dense pedestrian image is processed by the dense pedestrian detection model, and the output result of the detection box regression processing is used to correct the output result of the predicted target confidence, which effectively improves the detection accuracy, prevents the target from being lost, and increases the detection rate.

[0014] According to one embodiment of this application, the first detection structure includes a first detection box convolutional layer and a second detection box convolutional layer, and the second detection structure includes a first confidence convolutional layer and a second confidence convolutional layer. The step of correcting the output of the second detection structure based on the output of the first detection structure to obtain the detection box regression feature map and the detection target confidence feature map output by the detection head includes:

[0015] Obtain the first detection box feature map output by the first detection box convolutional layer, and obtain the first confidence feature map output by the first confidence convolutional layer;

[0016] The first detection box feature map is input into the target convolutional layer of the detection head to obtain the second detection box feature map output by the target convolutional layer.

[0017] Based on the compression function, the feature map of the second detection box is compressed to obtain the feature map of the third detection box.

[0018] The third detection box feature map and the first confidence feature map are multiplied and then input into the second confidence convolutional layer to obtain the detection target confidence feature map output by the second confidence convolutional layer;

[0019] The first detection box feature map is input into the second detection box convolutional layer to obtain the detection box regression feature map output by the second detection box convolutional layer.

[0020] According to one embodiment of this application, the dense pedestrian detection model is trained through the following steps:

[0021] Obtain a dense pedestrian sample set and a sample label set corresponding to the dense pedestrian sample set. The dense pedestrian sample set includes multiple dense pedestrian sample images, and the sample label set includes sample labels that correspond one-to-one with the multiple dense pedestrian sample images. Each sample label includes multiple sample pedestrian detection boxes on the dense pedestrian sample image.

[0022] The multiple dense pedestrian sample images of the dense pedestrian sample set are input into the dense pedestrian detection model to be trained, and multiple predicted pedestrian detection boxes of the dense pedestrian sample images output by the dense pedestrian detection model are obtained.

[0023] Based on the sample pedestrian detection boxes and the predicted pedestrian detection boxes, the dense pedestrian sample set is divided into a positive sample set and a negative sample set;

[0024] Calculate the target loss function based on the set of positive samples and the set of negative samples;

[0025] The dense pedestrian detection model is updated based on the target loss function.

[0026] According to one embodiment of this application, after dividing the dense pedestrian sample set into a positive sample set and a negative sample set, and before calculating the target loss function based on the positive sample set and the negative sample set, the method further includes:

[0027] Based on the positive sample set, calculate the intersection-union ratio of each predicted pedestrian detection box and each sample pedestrian detection box in the dense pedestrian sample image;

[0028] The first sample pedestrian detection box with the largest intersection-over-union ratio (IoU) with the first predicted pedestrian detection box in the dense pedestrian sample image is assigned to the first predicted pedestrian detection box.

[0029] According to one embodiment of this application, calculating the target loss function based on the positive sample set and the negative sample set includes:

[0030] Based on the set of positive samples and the set of negative samples, determine the confidence loss function;

[0031] Based on the set of positive samples, determine the detection box loss function;

[0032] The target loss function is determined based on the confidence loss function and the detection box loss function.

[0033] According to one embodiment of this application, determining the confidence loss function based on the positive sample set and the negative sample set includes:

[0034] Based on the positive sample set, the intersection-union ratio of the predicted pedestrian detection box and the sample pedestrian detection box is calculated to determine the target coefficient;

[0035] Based on the target coefficient and the prediction confidence corresponding to the predicted pedestrian detection box, determine the confidence loss function corresponding to the positive sample set;

[0036] Based on the negative sample set, the confidence loss function corresponding to the negative sample set is determined according to the prediction confidence corresponding to the predicted pedestrian detection box.

[0037] The target coefficient is used to make the learning weight of the positive sample set greater than the learning weight of the negative sample set.

[0038] According to one embodiment of this application, the feature layer includes a feature extraction module and a feature fusion module. The step of inputting the dense pedestrian image into the feature layer to obtain the dense pedestrian feature map output by the feature layer includes:

[0039] The dense pedestrian image is input into the feature extraction module to obtain the first pedestrian image features output by the feature extraction module;

[0040] The first pedestrian image features are input into the feature fusion module to obtain the dense pedestrian feature map output by the feature fusion module.

[0041] According to one embodiment of this application, the feature extraction module includes three convolutional feature extraction networks, the feature fusion module includes three feature fusion layers corresponding to the three convolutional feature extraction networks, and the dense pedestrian detection model includes three detection heads corresponding to the three feature fusion layers.

[0042] According to one embodiment of this application, the pedestrian detection result includes multiple target pedestrian detection boxes of the dense pedestrian image. The step of inputting the regression feature map of the detection boxes and the confidence feature map of the detected targets into the output layer to obtain the pedestrian detection result output by the output layer includes:

[0043] Multiple target locations with confidence scores greater than the detection confidence threshold are determined from the target confidence feature map;

[0044] Based on the multiple target locations, multiple target detection boxes are determined in the detection box regression feature map;

[0045] By filtering out overlapping targets in the multiple target detection boxes, the multiple target pedestrian detection boxes are obtained.

[0046] Secondly, this application provides a dense pedestrian detection device, the device comprising:

[0047] The acquisition module is used to acquire images of dense pedestrians to be detected;

[0048] The processing module is used to input the dense pedestrian image into the dense pedestrian detection model and obtain the pedestrian detection result output by the dense pedestrian detection model;

[0049] The dense pedestrian detection model includes a feature layer, a detection head, and an output layer. The processing module is further used for:

[0050] The dense pedestrian image is input into the feature layer to obtain the dense pedestrian feature map output by the feature layer;

[0051] The dense pedestrian feature map is input into the detection head, the detection box regression is performed through the first detection structure of the detection head, and the confidence of the detected target is predicted through the second detection structure of the detection head.

[0052] Based on the output of the first detection structure, the output of the second detection structure is corrected to obtain the detection box regression feature map and the detection target confidence feature map output by the detection head;

[0053] The regression feature map of the detection box and the confidence feature map of the detected target are input into the output layer to obtain the pedestrian detection result output by the output layer.

[0054] According to the dense pedestrian detection device of this application, the dense pedestrian detection model processes the dense pedestrian image, and the output result of the detection box regression processing is used to correct the output result of the predicted target confidence, which effectively improves the detection accuracy, prevents the target from being lost, and increases the detection rate.

[0055] Thirdly, this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the dense pedestrian detection method as described in the first aspect above.

[0056] Fourthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the dense pedestrian detection method as described in the first aspect above.

[0057] Fifthly, this application provides a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the dense pedestrian detection method as described in the first aspect.

[0058] In a sixth aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the dense pedestrian detection method as described in the first aspect above.

[0059] The above-described one or more technical solutions in the embodiments of this application have at least one of the following technical effects:

[0060] In terms of model structure, the regression branch of the detection head is used to correct the value of the confidence prediction branch of the second detection structure, which can solve the problem of target loss caused by misalignment between detection classification and regression.

[0061] Furthermore, positive and negative sample sampling and label sample reassignment are performed during model training. By adjusting the label allocation, the detection rate of overlapping pedestrians is optimized, thereby achieving accurate pedestrian detection in dense crowds.

[0062] Furthermore, in loss calculation, the intersection-union ratio (IUU) of the predicted pedestrian detection boxes and the sample pedestrian detection boxes is used as the target coefficient to increase the learning weight of positive samples and improve the detection rate.

[0063] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0064] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0065] Figure 1 This is one of the flowcharts of the dense pedestrian detection method provided in the embodiments of this application;

[0066] Figure 2 This is the second flowchart of the dense pedestrian detection method provided in the embodiments of this application;

[0067] Figure 3 This is a schematic diagram of the inference process of the dense pedestrian detection model provided in the embodiments of this application;

[0068] Figure 4 This is a schematic diagram of the inference process of the detection head provided in the embodiments of this application;

[0069] Figure 5 This is one of the schematic diagrams of the training process of the dense pedestrian detection model provided in the embodiments of this application;

[0070] Figure 6 This is the second schematic diagram of the training process of the dense pedestrian detection model provided in the embodiments of this application;

[0071] Figure 7 This is a schematic diagram of the structure of the dense pedestrian detection device provided in the embodiments of this application;

[0072] Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0073] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0074] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0075] The dense pedestrian detection method, dense pedestrian detection device, electronic device, and readable storage medium provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0076] Among them, the dense pedestrian detection method can be applied to the terminal, specifically executed by the hardware or software in the terminal.

[0077] The terminal includes, but is not limited to, portable communication devices such as mobile phones or tablets with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that, in some embodiments, the terminal may not be a portable communication device, but rather a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0078] The following embodiments describe a terminal including a display and a touch-sensitive surface. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, mouse, and joystick.

[0079] The dense pedestrian detection method provided in this application embodiment can be executed by an electronic device or a functional module or entity in an electronic device that can implement the dense pedestrian detection method. The electronic devices mentioned in this application embodiment include, but are not limited to, mobile phones, tablets, computers, cameras, and wearable devices. The dense pedestrian detection method provided in this application embodiment will be described below using an electronic device as the execution subject.

[0080] like Figure 1 As shown, the dense pedestrian detection method includes steps 110 and 120.

[0081] Step 110: Obtain images of dense pedestrians to be detected.

[0082] Among them, dense pedestrian images can be images of a large crowd captured by image acquisition devices or video surveillance equipment.

[0083] For example, dense pedestrian images can be images of crowds during peak travel times in a subway scene.

[0084] For example, dense pedestrian images can be images of crowds during peak shopping hours in a shopping mall scene.

[0085] Step 120: Input the dense pedestrian image into the dense pedestrian detection model to obtain the pedestrian detection results output by the dense pedestrian detection model.

[0086] Among them, the dense pedestrian detection model can be an object detection model trained based on a dense pedestrian sample set and a sample label set corresponding to the dense pedestrian sample set.

[0087] In this embodiment, the dense pedestrian detection model includes a feature layer, a detection head, and an output layer. The feature layer is a structure for feature extraction, the detection head is a structure for predicting the location of target types, and the output layer is a structure for predicting output.

[0088] like Figure 2 As shown, step 120, inputting the dense pedestrian image into the dense pedestrian detection model to obtain the pedestrian detection results output by the dense pedestrian detection model, further includes:

[0089] Step 121: Input the dense pedestrian image into the feature layer to obtain the dense pedestrian feature map output by the feature layer.

[0090] In this step, the input to the feature layer is a dense pedestrian image, the feature layer extracts features from the dense pedestrian image, and the output of the feature layer is a dense pedestrian feature map.

[0091] Step 122: Input the dense pedestrian feature map into the detection head, perform bounding box regression processing through the first detection structure of the detection head, and predict the confidence of the detected target through the second detection structure of the detection head.

[0092] In this embodiment, the detection head includes a first detection structure and a second detection structure. The first detection structure is used to perform regression processing on the detection boxes of pedestrian detection targets on the dense pedestrian feature map, and adjust the shape and position of the detection boxes. The second detection structure is used to predict the confidence of pedestrian detection targets on the dense pedestrian feature map.

[0093] Step 123: Based on the output of the first detection structure, correct the output of the second detection structure to obtain the detection bounding box regression feature map and the detection target confidence feature map output by the detection head.

[0094] In the detection head, the output of the first detection structure is used to correct the output of the second detection structure in order to align the confidence level and detection box of the pedestrian detection target, effectively preventing the loss of the detection target and improving the accuracy of pedestrian detection.

[0095] The input to the detection head is a dense pedestrian feature map. The detection head performs bounding box regression processing through the first detection structure and predicts the confidence of the detected target through the second detection structure. The output of the first detection structure is then used to correct the output of the second detection structure, aligning the confidence of the pedestrian detection target with the bounding box. The output of the detection head is a bounding box regression feature map and a detection target confidence feature map.

[0096] Among them, the detection box regression feature map is used to represent the detection box information of the pedestrian detection target after regression processing, and the detection target confidence feature map is used to represent the confidence information of the pedestrian detection target.

[0097] Step 124: Input the detection box regression feature map and the detection target confidence feature map into the output layer to obtain the pedestrian detection results output by the output layer.

[0098] In this embodiment, the input to the output layer is the detection box regression feature map and the detection target confidence feature map, and the output of the output layer is the pedestrian detection result. The pedestrian detection result includes multiple target pedestrian detection boxes in the dense pedestrian image, and each target pedestrian detection box corresponds to a pedestrian detection target in the dense pedestrian image.

[0099] It should be noted that pedestrian detection targets in dense crowds have a high degree of overlap. Large overlap of detection targets will lead to a high degree of overlap of detection boxes. Detection methods in related technologies often focus on the center position of the detection target, which may result in the loss of detection boxes for detection targets with similar centers, or the detection box covering multiple detection targets. This leads to low detection accuracy, easy loss of targets, and a high rate of missed detections.

[0100] In this embodiment, within the detection head of the dense pedestrian detection model, the output of the first detection structure performing detection box regression processing is used to correct the output of the second detection structure predicting the confidence of the detected target. Aligning the confidence of the pedestrian detection target with the detection box can solve the problem of target loss caused by misalignment between detection classification and regression, improve detection accuracy, prevent target loss, and increase the detection rate of dense pedestrian images.

[0101] According to the dense pedestrian detection method provided in the embodiments of this application, the dense pedestrian detection model is used to process the dense pedestrian image, and the output result of the detection box regression processing is used to correct the output result of the predicted target confidence, which effectively improves the detection accuracy, prevents the target from being lost, and improves the detection rate.

[0102] In some embodiments, the first detection structure includes a first detection box convolutional layer and a second detection box convolutional layer, and the second detection structure includes a first confidence convolutional layer and a second confidence convolutional layer.

[0103] In this embodiment, step 123, correcting the output of the second detection structure based on the output of the first detection structure to obtain the detection bounding box regression feature map and the detection target confidence feature map output by the detection head, may include:

[0104] Obtain the first detection box feature map output by the first detection box convolutional layer, and obtain the first confidence feature map output by the first confidence convolutional layer;

[0105] The first detection bounding box feature map is input into the target convolutional layer of the detection head to obtain the second detection bounding box feature map output by the target convolutional layer.

[0106] Based on the compression function, the feature map of the second detection box is compressed to obtain the feature map of the third detection box.

[0107] The third detection box feature map and the first confidence feature map are multiplied and then input into the second confidence convolutional layer to obtain the detection target confidence feature map output by the second confidence convolutional layer.

[0108] The feature map of the first detection box is input into the convolutional layer of the second detection box to obtain the regression feature map of the detection box output by the convolutional layer of the second detection box.

[0109] It is understandable that the detection head is divided into two branches: a first detection structure and a second detection structure. The first detection structure includes a first detection box convolutional layer and a second detection box convolutional layer, and the second detection structure includes a first confidence convolutional layer and a second confidence convolutional layer. Between the first detection structure and the second detection structure, there is also a target convolutional layer for correction.

[0110] The first detection box convolutional layer performs detection box regression processing on the dense pedestrian feature map and outputs the first detection box feature map. The first detection box feature map is input into the target convolutional layer to obtain the second detection box feature map. The second detection box feature map is compressed to obtain the third detection box feature map. The third detection box feature map is multiplied with the first confidence feature map output by the first confidence convolutional layer. The result of the multiplication is input into the second confidence convolutional layer to correct the output structure of the second detection structure.

[0111] In this embodiment, the output of the first detection box convolutional layer can be input to both the target convolutional layer and the second detection box convolutional layer; the input of the second confidence convolutional layer includes the output of the first confidence convolutional layer and the output of the target convolutional layer.

[0112] The following is a specific example.

[0113] like Figure 4 As shown, the detection head is divided into a first detection structure for the regression of the detection box on the left and a second detection structure for the confidence of the detection target on the right.

[0114] In this embodiment, the dense pedestrian feature map input to the detection head is first processed by two convolutional layers CONV3×3, and then input to the first detection structure and the second detection structure respectively.

[0115] The first detection structure for the left-side bounding box regression consists of, from top to bottom, the first and second bounding box convolutional layers. The first CONV3×3 is the first bounding box convolutional layer, and the second CONV3×3 is the second bounding box convolutional layer.

[0116] The first detection box feature map output from the first detection box convolutional layer is output to the target convolutional layer CONV between the two branches. The second detection box feature map output from the target convolutional layer is compressed using the compression function (Sigmoid) to the range [0,1], thus obtaining the third detection box feature map.

[0117] The first detection box feature map output by the first detection box convolutional layer is also output to the second detection box convolutional layer below, where it is processed to output the detection box regression feature map.

[0118] The second detection structure on the right side, which detects the confidence level of the target, consists of a first confidence level convolutional layer and a second confidence level convolutional layer from top to bottom. The first CONV3×3 is the first confidence level convolutional layer, and the second CONV3×3 is the second confidence level convolutional layer.

[0119] The first confidence feature map output from the first confidence convolutional layer is fed into the second confidence convolutional layer. The input of the second confidence convolutional layer also includes the third detection box feature map, which is processed by the second confidence convolutional layer to output the target confidence feature map. The regression branch is used to correct the confidence prediction branch value, keeping the classification confidence and the regression detection box information aligned, effectively preventing the loss of the detection target.

[0120] In some embodiments, the dense pedestrian detection model is trained through the following steps:

[0121] Obtain a dense pedestrian sample set and a corresponding sample label set;

[0122] Multiple dense pedestrian sample images from a dense pedestrian sample set are input into a dense pedestrian detection model to be trained, and multiple predicted pedestrian detection boxes are obtained from the dense pedestrian sample images output by the dense pedestrian detection model.

[0123] Based on the sample pedestrian detection boxes and the predicted pedestrian detection boxes, the dense pedestrian sample set is divided into a positive sample set and a negative sample set;

[0124] Calculate the target loss function based on the positive sample set and the negative sample set;

[0125] Update the dense pedestrian detection model based on the target loss function.

[0126] Among them, the dense pedestrian sample set includes multiple dense pedestrian sample images, the sample label set includes sample labels corresponding one-to-one to the multiple dense pedestrian sample images, and each sample label includes multiple sample pedestrian detection frames on the dense pedestrian sample image.

[0127] In actual execution, a large number of dense pedestrian sample images can be collected for the dense crowd scene, and the human bodies in the dense pedestrian sample images are labeled with sample pedestrian detection frames.

[0128] In the training stage of the dense pedestrian detection model, input the dense pedestrian sample images into the to-be-trained dense pedestrian detection model, and process them through the feature layer, detection head and output layer to output multiple predicted pedestrian detection frames of the dense pedestrian sample images.

[0129] In this embodiment, positive and negative sample sampling is performed on the dense pedestrian sample set according to the predicted pedestrian detection frames and sample pedestrian detection frames output by the dense pedestrian detection model.

[0130] In actual execution, all the predicted pedestrian detection frames of the dense pedestrian sample set can be denoted as A, and traverse the sample pedestrian detection frame box of the sample label set label , select the top k with high confidence from all the predicted pedestrian detection frames as candidate pedestrian detection frames, and calculate the intersection over union (IOU) between the candidate pedestrian detection frames and the sample pedestrian detection frames.

[0131] According to the value calculated from the intersection over union between the candidate pedestrian detection frames and the sample pedestrian detection frames, fit the Gaussian mixture distribution through the expectation maximization (EM) algorithm as follows:

[0132]

[0133]

[0134] Among them, x represents the value calculated from the intersection over union between the candidate pedestrian detection frames and the sample pedestrian detection frames, p represents the probability density function, σ represents the standard deviation, and μ represents the mathematical expectation.

[0135] According to the fitting result, according to the preset threshold μ1, select p(A i ) > MAX(μ0, μ1) as the positive sample, that is, P i ; select MIN(μ0, μ1) < p(A i ) < MAX(μ0, μ1) as the ignored sample, denoted as Ii ; Calculate the set of all positive samples P = {P i ,...}, ignoring the sample set The negative sample set N = API is obtained.

[0136] In this embodiment, the target loss function is calculated by dividing the positive sample set and the negative sample set, the dense pedestrian detection model is iteratively optimized, the model parameters are updated, and a trained dense pedestrian detection model is obtained.

[0137] In some embodiments, after dividing the dense pedestrian sample set into a positive sample set and a negative sample set, and before calculating the target loss function based on the positive sample set and the negative sample set, the dense pedestrian detection method further includes:

[0138] Based on the positive sample set, calculate the intersection-union ratio of each predicted pedestrian detection box and each sample pedestrian detection box in the dense pedestrian sample image;

[0139] The first sample pedestrian detection box with the largest intersection-union ratio (IoU) with the first predicted pedestrian detection box in the dense pedestrian sample image is assigned to the first predicted pedestrian detection box.

[0140] In related technologies, in densely populated scenes, human bodies overlap. Detection targets with a high degree of overlap are very likely to correspond to the same predicted detection box, which can lead to inaccurate or missing predicted detection boxes.

[0141] In this embodiment, for the pre-divided positive and negative sample sets, the label samples of the positive sample set are redistributed, and the positive samples are assigned to different detection targets. By adjusting the label allocation, the detection rate of overlapping pedestrians is optimized, thereby achieving accurate pedestrian detection in dense crowds.

[0142] In actual implementation, a positive sample set P is obtained. The cross-union ratio (CUI) is calculated between the predicted pedestrian detection box of each target in the positive sample set P and each sample pedestrian detection box. For each predicted pedestrian detection box, the sample pedestrian detection box with the largest CUI is selected as its assigned target.

[0143] Wherein, the first predicted pedestrian detection box is any predicted pedestrian detection box in the positive sample set, and the first sample pedestrian detection box is the sample pedestrian detection box with the largest intersection-union ratio with the first predicted pedestrian detection box.

[0144] In this embodiment, to address the issues of inaccurate detection and low detection rate caused by high overlap in dense crowds, the model training process adjusts label allocation by sampling positive and negative samples and redistributing label samples, thereby optimizing the detection rate of overlapping pedestrians and achieving accurate pedestrian detection in dense crowds.

[0145] In some embodiments, calculating the target loss function based on the positive sample set and the negative sample set may include:

[0146] Determine the confidence loss function based on the positive and negative sample sets;

[0147] Determine the detection box loss function based on the set of positive samples;

[0148] The target loss function is determined based on the confidence loss function and the detection box loss function.

[0149] In this embodiment, the target loss function includes two parts: a confidence loss function and a detection box loss function. The confidence loss function is used to determine whether a pixel in the image contains the target and the approximate location of the target. The detection box loss function is used to measure the accuracy of the regressed detection box.

[0150] It should be noted that, for the confidence loss function, the pixel position is tested and calculated using the positive and negative sample sets; for the detection box loss function, the detection box of the pedestrian detection target is tested and calculated using the positive sample set.

[0151] In some embodiments, the confidence loss function is determined based on the positive sample set and the negative sample set, including:

[0152] Based on the positive sample set, the intersection-union ratio of the predicted pedestrian detection box and the sample pedestrian detection box is calculated to determine the target coefficient;

[0153] Based on the target coefficient and the prediction confidence corresponding to the predicted pedestrian detection box, determine the confidence loss function corresponding to the positive sample set;

[0154] Based on the negative sample set, the confidence loss function corresponding to the negative sample set is determined according to the prediction confidence corresponding to the predicted pedestrian detection box.

[0155] The target coefficient is used to make the learning weight of the positive sample set greater than the learning weight of the negative sample set.

[0156] It should be noted that each image includes a small number of positive samples and a large number of negative samples. By setting the target coefficient, positive samples are given a larger learning weight, ensuring that the learning of positive samples by the dense pedestrian detection model is fully effective.

[0157] The target coefficient is used to characterize the matching score between the predicted pedestrian detection box and the corresponding sample pedestrian detection box.

[0158] Understandably, for positive samples, the intersection-union ratio (IUU) between the predicted pedestrian detection box and the corresponding sample pedestrian detection box is greater than 0.

[0159] For a set of positive samples, calculate the intersection-union ratio (IUR) of the predicted pedestrian detection box and the sample pedestrian detection box for a certain positive sample. Use the IUR as the target coefficient for that positive sample. Then, determine the confidence loss function based on the target coefficient and the prediction confidence corresponding to the predicted pedestrian detection box output by the dense pedestrian detection model.

[0160] For example, in the positive sample set P, the predicted pedestrian detection box is box. pred The pedestrian detection bounding box for the sample is a box. label The target coefficient is q = iou(box) pred ,box label ).

[0161] In this embodiment, for the positive sample set, the corresponding confidence loss function can be as follows:

[0162] L conf =-q(qlog(p)+(1-q)log(1-p))

[0163] Among them, L conf Let q be the confidence loss function, q be the target coefficient, and p be the prediction confidence.

[0164] For the negative sample set, the confidence loss function is determined directly based on the prediction confidence of the predicted pedestrian detection box output by the dense pedestrian detection model.

[0165] In this embodiment, for the negative sample set, the corresponding confidence loss function can be as follows:

[0166] L conf =-αp γ log(1-p)

[0167] Among them, L conf is the confidence loss function, p is the prediction confidence, and α and γ are hyperparameters used to adjust the importance of negative samples in the calculation of the loss function.

[0168] In practice, the formulas for the target coefficient and the confidence loss function can be integrated as follows:

[0169]

[0170]

[0171] Where P is the set of positive samples, N is the set of negative samples, and L... conf Let q be the confidence loss function, p be the prediction confidence, and α and γ be hyperparameters. pred To predict pedestrian detection bounding boxes, label The sample pedestrian detection box.

[0172] In this embodiment, the objective loss function of the dense pedestrian detection model can be as follows:

[0173]

[0174] Among them, L loss Let L be the target loss function. conf Let L be the confidence loss function. reg Let n be the loss function for the detection box. P n is the number of positive samples. N Let P be the number of negative samples, P be the set of positive samples, and N be the set of negative samples.

[0175] In practice, the detection box loss function can be the L1 norm of the detection box, L... reg =|box pred -box label |, box pred To predict pedestrian detection bounding boxes, label The sample pedestrian detection box.

[0176] The following is a specific example.

[0177] like Figure 6 As shown, the input dataset and label set are used in the dense pedestrian detection model to be trained, namely the dense pedestrian sample set and the sample label set corresponding to the dense pedestrian sample set.

[0178] The output feature map is obtained through inference using a dense pedestrian detection model, which includes multiple predicted pedestrian detection boxes in dense pedestrian sample images.

[0179] Based on the predicted pedestrian detection boxes and sample pedestrian detection boxes, positive and negative samples are sampled from the dense pedestrian sample set, and the label samples are redistributed from the positive sample set.

[0180] Calculate the confidence loss function and the detection box loss function, iterate the data to optimize the model, and obtain a well-trained dense pedestrian detection model.

[0181] In some embodiments, the feature layer includes a feature extraction module and a feature fusion module. Step 121, inputting the dense pedestrian image into the feature layer to obtain the dense pedestrian feature map output by the feature layer, may include:

[0182] The dense pedestrian image is input into the feature extraction module to obtain the first pedestrian image features output by the feature extraction module;

[0183] The features of the first pedestrian image are input into the feature fusion module to obtain a dense pedestrian feature map output by the feature fusion module.

[0184] In this embodiment, the feature extraction module may include a feature extraction network, which performs convolutional feature extraction on dense pedestrian images to obtain first pedestrian image features at different sampling multiples. The feature fusion module fuses the first pedestrian image features to obtain a dense pedestrian feature map output by the feature layer.

[0185] In some embodiments, the feature extraction module includes three convolutional feature extraction networks, the feature fusion module includes three feature fusion layers corresponding to the three convolutional feature extraction networks, and the dense pedestrian detection model includes three detection heads corresponding to the three feature fusion layers.

[0186] like Figure 3 As shown, after the three convolutional feature extraction networks extract features, the obtained first pedestrian image features are input into the corresponding three feature fusion layers. The three feature fusion layers present a feature pyramid (FPN) structure. The features are fused through the three feature fusion layers, and the result of feature fusion is then input into the corresponding three-level detection head.

[0187] In practice, the structure of the dense pedestrian detection model can be designed based on a one-stage anchor-free convolutional detection model. Three convolutional feature extraction networks obtain feature maps downsampled by 8x, 16x, and 32x respectively. The feature maps are then fused by multi-scale feature maps through FPN and finally output to the three-level detection head.

[0188] In some embodiments, the pedestrian detection result includes multiple target pedestrian detection boxes in a dense pedestrian image. Step 124 involves inputting the detection box regression feature map and the detection target confidence feature map into the output layer to obtain the pedestrian detection result output by the output layer, including:

[0189] Multiple target locations with confidence scores greater than the detection confidence threshold are identified from the target confidence feature map;

[0190] Based on multiple target locations, determine multiple target detection boxes in the detection box regression feature map;

[0191] By filtering out overlapping targets from multiple target detection boxes, multiple target pedestrian detection boxes are obtained.

[0192] It is understandable that step 124 belongs to the post-processing stage of the dense pedestrian detection model. The confidence feature map of the detection target and the regression feature map of the detection box are obtained from the model inference results. The model post-processing is performed to obtain multiple target pedestrian detection boxes of the dense pedestrian image, so as to achieve accurate pedestrian detection in dense crowds.

[0193] In this embodiment, multiple target locations with confidence scores greater than the detection confidence threshold are determined from the target confidence feature map. Target detection boxes are obtained from the detection box regression feature map based on the target locations. Overlapping targets among multiple target detection boxes can be filtered using non-maximum suppression (NMS) to obtain the final output target pedestrian detection box.

[0194] Understandably, during the model training phase, the output layer of the dense pedestrian detection model also performs the corresponding model post-processing.

[0195] The following is a specific example.

[0196] like Figure 5 As shown, prepare a pedestrian detection dataset for dense crowds, i.e., a dense pedestrian sample set.

[0197] A reasoning model is constructed. The dense pedestrian detection model includes a feature layer, a detection head, and a processing layer. Based on the pedestrian detection dataset, model reasoning is performed through the feature layer and the detection head.

[0198] The confidence feature map of the detected target and the regression feature map of the detection box obtained from the model inference results are input into the output layer for model post-processing. The model is trained based on the model post-processing results to obtain a trained dense pedestrian detection model. The trained dense pedestrian detection model can directly detect dense pedestrian images to be detected.

[0199] This application addresses the problem of inaccurate detection and low detection rate caused by high overlap in dense crowds. During model training, positive and negative sample sampling and label sample reassignment are performed. By adjusting the label allocation, the detection rate of overlapping pedestrians is optimized, achieving accurate pedestrian detection in dense crowds. In terms of model structure, the regression branch (first detection structure) of the detection head is used to correct the confidence prediction branch (second detection structure) value, which can solve the problem of target loss caused by misalignment between detection classification and regression. In loss calculation, the intersection-union ratio (IUU) of the predicted pedestrian detection box and the sample pedestrian detection box is calculated as the target coefficient, increasing the learning weight of positive samples and improving the detection rate.

[0200] The dense pedestrian detection method provided in this application can be executed by a dense pedestrian detection device. This application uses the example of a dense pedestrian detection device executing the dense pedestrian detection method to illustrate the dense pedestrian detection device provided in this application.

[0201] This application also provides a dense pedestrian detection device.

[0202] like Figure 7 As shown, the dense pedestrian detection device includes:

[0203] The acquisition module 710 is used to acquire dense pedestrian images to be detected;

[0204] The processing module 720 is used to input dense pedestrian images into a dense pedestrian detection model and obtain pedestrian detection results output by the dense pedestrian detection model;

[0205] The dense pedestrian detection model includes a feature layer, a detection head, and an output layer. The processing module is specifically used for:

[0206] The dense pedestrian image is input into the feature layer to obtain the dense pedestrian feature map output by the feature layer;

[0207] The dense pedestrian feature map is input into the detection head, the detection box regression is performed through the first detection structure of the detection head, and the confidence of the detected target is predicted through the second detection structure of the detection head.

[0208] Based on the output of the first detection structure, the output of the second detection structure is corrected to obtain the detection bounding box regression feature map and the detection target confidence feature map output by the detection head;

[0209] The regression feature map of the detection box and the confidence feature map of the detected target are input into the output layer to obtain the pedestrian detection results output by the output layer.

[0210] According to the dense pedestrian detection device provided in the embodiments of this application, the dense pedestrian detection model processes the dense pedestrian image, and the output result of the detection box regression processing is used to correct the output result of the predicted target confidence, which effectively improves the detection accuracy, prevents the target from being lost, and improves the detection rate.

[0211] In some embodiments, the first detection structure includes a first detection box convolutional layer and a second detection box convolutional layer, and the second detection structure includes a first confidence convolutional layer and a second confidence convolutional layer. The processing module 720 is used to obtain the first detection box feature map output by the first detection box convolutional layer and obtain the first confidence feature map output by the first confidence convolutional layer.

[0212] The first detection bounding box feature map is input into the target convolutional layer of the detection head to obtain the second detection bounding box feature map output by the target convolutional layer.

[0213] Based on the compression function, the feature map of the second detection box is compressed to obtain the feature map of the third detection box.

[0214] The third detection box feature map and the first confidence feature map are multiplied and then input into the second confidence convolutional layer to obtain the detection target confidence feature map output by the second confidence convolutional layer.

[0215] The feature map of the first detection box is input into the convolutional layer of the second detection box to obtain the regression feature map of the detection box output by the convolutional layer of the second detection box.

[0216] In some embodiments, the processing module 720 is configured to train a dense pedestrian detection model through the following steps:

[0217] Obtain a dense pedestrian sample set and a sample label set corresponding to the dense pedestrian sample set. The dense pedestrian sample set includes multiple dense pedestrian sample images, and the sample label set includes sample labels that correspond one-to-one with the multiple dense pedestrian sample images. Each sample label includes multiple sample pedestrian detection boxes on the dense pedestrian sample image.

[0218] Multiple dense pedestrian sample images from a dense pedestrian sample set are input into a dense pedestrian detection model to be trained, and multiple predicted pedestrian detection boxes are obtained from the dense pedestrian sample images output by the dense pedestrian detection model.

[0219] Based on the sample pedestrian detection boxes and the predicted pedestrian detection boxes, the dense pedestrian sample set is divided into a positive sample set and a negative sample set;

[0220] Calculate the target loss function based on the positive and negative sample sets;

[0221] The dense pedestrian detection model is updated based on the objective loss function.

[0222] In some embodiments, the processing module 720 is further configured to calculate the intersection-union ratio of each predicted pedestrian detection box and each sample pedestrian detection box in the dense pedestrian sample image based on the positive sample set;

[0223] The first sample pedestrian detection box with the largest intersection-union ratio (IoU) with the first predicted pedestrian detection box in the dense pedestrian sample image is assigned to the first predicted pedestrian detection box.

[0224] In some embodiments, the processing module 720 is used to determine a confidence loss function based on the positive sample set and the negative sample set;

[0225] Determine the detection box loss function based on the set of positive samples;

[0226] The target loss function is determined based on the confidence loss function and the detection box loss function.

[0227] In some embodiments, the processing module 720 is used to calculate the intersection-union ratio of the predicted pedestrian detection box and the sample pedestrian detection box based on the positive sample set, and determine the target coefficient;

[0228] Based on the target coefficient and the prediction confidence corresponding to the predicted pedestrian detection box, determine the confidence loss function corresponding to the positive sample set;

[0229] Based on the negative sample set, the confidence loss function corresponding to the negative sample set is determined according to the prediction confidence corresponding to the predicted pedestrian detection box.

[0230] The target coefficient is used to make the learning weight of the positive sample set greater than the learning weight of the negative sample set.

[0231] In some embodiments, the feature layer includes a feature extraction module and a feature fusion module. The processing module 720 is used to input dense pedestrian images into the feature extraction module to obtain the first pedestrian image features output by the feature extraction module.

[0232] The features of the first pedestrian image are input into the feature fusion module to obtain a dense pedestrian feature map output by the feature fusion module.

[0233] In some embodiments, the feature extraction module includes three convolutional feature extraction networks, the feature fusion module includes three feature fusion layers corresponding to the three convolutional feature extraction networks, and the dense pedestrian detection model includes three detection heads corresponding to the three feature fusion layers.

[0234] In some embodiments, the pedestrian detection result includes multiple target pedestrian detection boxes in a dense pedestrian image, and the processing module 720 is used to determine multiple target locations with confidence scores greater than a detection confidence threshold from the detection target confidence feature map;

[0235] Based on multiple target locations, determine multiple target detection boxes in the detection box regression feature map;

[0236] By filtering out overlapping targets from multiple target detection boxes, multiple target pedestrian detection boxes are obtained.

[0237] The dense pedestrian detection device in this application embodiment can be an electronic device or a component of an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0238] The dense pedestrian detection device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0239] The dense pedestrian detection device provided in this application embodiment can achieve... Figures 1 to 6 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0240] In some embodiments, such as Figure 8 As shown, this application embodiment also provides an electronic device 800, including a processor 801, a memory 802, and a computer program stored in the memory 802 and executable on the processor 801. When the program is executed by the processor 801, it implements the various processes of the above-described dense pedestrian detection method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0241] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0242] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described dense pedestrian detection method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0243] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0244] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described dense pedestrian detection method.

[0245] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0246] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described dense pedestrian detection method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0247] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0248] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0249] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0250] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0251] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0252] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A method of dense pedestrian detection, characterized by, include: Obtain dense pedestrian images to be detected; The dense pedestrian image is input into the dense pedestrian detection model to obtain the pedestrian detection results output by the dense pedestrian detection model; The dense pedestrian detection model includes a feature layer, a detection head, and an output layer. The step of inputting the dense pedestrian image into the dense pedestrian detection model to obtain the pedestrian detection result output by the dense pedestrian detection model further includes: The dense pedestrian image is input into the feature layer to obtain the dense pedestrian feature map output by the feature layer; The dense pedestrian feature map is input into the detection head, the detection box regression is performed through the first detection structure of the detection head, and the confidence of the detected target is predicted through the second detection structure of the detection head. Based on the output of the first detection structure, the output of the second detection structure is corrected to obtain the detection box regression feature map and the detection target confidence feature map output by the detection head; The regression feature map of the detection box and the confidence feature map of the detected target are input into the output layer to obtain the pedestrian detection result output by the output layer.

2. The dense pedestrian detection method of claim 1, wherein, The first detection structure includes a first detection box convolutional layer and a second detection box convolutional layer. The second detection structure includes a first confidence convolutional layer and a second confidence convolutional layer. The step of correcting the output of the second detection structure based on the output of the first detection structure to obtain the detection box regression feature map and the detection target confidence feature map output by the detection head includes: Obtain the first detection box feature map output by the first detection box convolutional layer, and obtain the first confidence feature map output by the first confidence convolutional layer; The first detection box feature map is input into the target convolutional layer of the detection head to obtain the second detection box feature map output by the target convolutional layer. Based on the compression function, the feature map of the second detection box is compressed to obtain the feature map of the third detection box. The third detection box feature map and the first confidence feature map are multiplied and then input into the second confidence convolutional layer to obtain the detection target confidence feature map output by the second confidence convolutional layer; The first detection box feature map is input into the second detection box convolutional layer to obtain the detection box regression feature map output by the second detection box convolutional layer.

3. The dense pedestrian detection method according to claim 1, characterized in that, The dense pedestrian detection model is trained through the following steps: Obtain a dense pedestrian sample set and a sample label set corresponding to the dense pedestrian sample set. The dense pedestrian sample set includes multiple dense pedestrian sample images, and the sample label set includes sample labels that correspond one-to-one with the multiple dense pedestrian sample images. Each sample label includes multiple sample pedestrian detection boxes on the dense pedestrian sample image. The multiple dense pedestrian sample images of the dense pedestrian sample set are input into the dense pedestrian detection model to be trained, and multiple predicted pedestrian detection boxes of the dense pedestrian sample images output by the dense pedestrian detection model are obtained. Based on the sample pedestrian detection boxes and the predicted pedestrian detection boxes, the dense pedestrian sample set is divided into a positive sample set and a negative sample set; Calculate the target loss function based on the set of positive samples and the set of negative samples; The dense pedestrian detection model is updated based on the target loss function.

4. The dense pedestrian detection method of claim 3, wherein, After dividing the dense pedestrian sample set into a positive sample set and a negative sample set, and before calculating the target loss function based on the positive sample set and the negative sample set, the method further includes: Based on the positive sample set, calculate the intersection-union ratio of each predicted pedestrian detection box and each sample pedestrian detection box in the dense pedestrian sample image; The first sample pedestrian detection box with the largest intersection-over-union ratio (IoU) with the first predicted pedestrian detection box in the dense pedestrian sample image is assigned to the first predicted pedestrian detection box.

5. The dense pedestrian detection method of claim 4, wherein, The step of calculating the target loss function based on the positive sample set and the negative sample set includes: Based on the set of positive samples and the set of negative samples, determine the confidence loss function; Based on the set of positive samples, determine the detection box loss function; The target loss function is determined based on the confidence loss function and the detection box loss function.

6. The dense pedestrian detection method of claim 5, wherein, The step of determining the confidence loss function based on the positive sample set and the negative sample set includes: Based on the positive sample set, the intersection-union ratio of the predicted pedestrian detection box and the sample pedestrian detection box is calculated to determine the target coefficient; Based on the target coefficient and the prediction confidence corresponding to the predicted pedestrian detection box, determine the confidence loss function corresponding to the positive sample set; Based on the negative sample set, the confidence loss function corresponding to the negative sample set is determined according to the prediction confidence corresponding to the predicted pedestrian detection box. The target coefficient is used to make the learning weight of the positive sample set greater than the learning weight of the negative sample set.

7. The dense pedestrian detection method according to any one of claims 1-6, wherein, The feature layer includes a feature extraction module and a feature fusion module. The step of inputting the dense pedestrian image into the feature layer to obtain the dense pedestrian feature map output by the feature layer includes: The dense pedestrian image is input into the feature extraction module to obtain the first pedestrian image features output by the feature extraction module; The first pedestrian image features are input into the feature fusion module to obtain the dense pedestrian feature map output by the feature fusion module.

8. The dense pedestrian detection method of claim 7, wherein, The feature extraction module includes three convolutional feature extraction networks, the feature fusion module includes three feature fusion layers corresponding to the three convolutional feature extraction networks, and the dense pedestrian detection model includes three detection heads corresponding to the three feature fusion layers.

9. The dense pedestrian detection method according to any one of claims 1-6, wherein, The pedestrian detection result includes multiple target pedestrian detection boxes in the dense pedestrian image. The step of inputting the regression feature map of the detection box and the confidence feature map of the detected target into the output layer to obtain the pedestrian detection result output by the output layer includes: Multiple target locations with confidence scores greater than the detection confidence threshold are determined from the target confidence feature map; Based on the multiple target locations, multiple target detection boxes are determined in the detection box regression feature map; By filtering out overlapping targets in the multiple target detection boxes, the multiple target pedestrian detection boxes are obtained.

10. A dense pedestrian detection device, characterized in that, include: The acquisition module is used to acquire images of dense pedestrians to be detected; The processing module is used to input the dense pedestrian image into the dense pedestrian detection model and obtain the pedestrian detection result output by the dense pedestrian detection model; The dense pedestrian detection model includes a feature layer, a detection head, and an output layer. The processing module is further used for: The dense pedestrian image is input into the feature layer to obtain the dense pedestrian feature map output by the feature layer; The dense pedestrian feature map is input into the detection head, the detection box regression is performed through the first detection structure of the detection head, and the confidence of the detected target is predicted through the second detection structure of the detection head. Based on the output of the first detection structure, the output of the second detection structure is corrected to obtain the detection box regression feature map and the detection target confidence feature map output by the detection head; The regression feature map of the detection box and the confidence feature map of the detected target are input into the output layer to obtain the pedestrian detection result output by the output layer.

Images