Dense crowd target detection method and device and medium
Through the multi-scale channel segmentation module and generalized enhanced feature pyramid network structure, combined with improved loss function, the problems of slow detection speed and missed detection in dense crowd detection are solved, achieving higher accuracy and robustness , suitable for target detection in dense crowd scenarios.
Patent Information
- Application Number
- CN202411408021.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-10
- Publication Date
- 2025-01-10
AI Technical Summary
Traditional two-stage object detection algorithms have problems such as slow detection speed, large calculation amount, missed detection and missed detection in dense crowd detection, making it difficult to achieve comprehensive and accurate detection in dense crowd scenarios.
Using multi-scale channel segmentation module and generalized enhanced feature pyramid network structure, the human body features of the size and size are retained through the fusion of feature maps of four different scales to prevent missed detection and occlusion information. Combined with standard detection heads and small object detection heads, use improved The loss function is trained.
It improves the accuracy and robustness of target detection for intensive populations, reduces missed target detection, and is suitable for intensive scenarios of different sizes and complexities, and is suitable for edge device deployment.
Smart Images
Figure CN119296037A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of target detection, and in particular to a method, device and medium for detecting targets in dense crowds. Background Art
[0002] Crowd detection is of great significance to the safety and flow control of public places, and is achieved through target detection methods. Traditional two-stage target detection algorithms, such as Faster-rcnn and DETR, classify areas where targets may exist before performing detection. Although the accuracy is high, the training process is cumbersome, the amount of calculation and parameters is huge, and the detection speed is slow. At the same time, there are some limitations when dealing with dense crowd detection.
[0003] With the rapid development of deep learning, single-stage target detection methods have been applied in crowd detection. Compared with the traditional two-stage method, the single-stage target detection algorithm has the advantages of lightweight and fast detection speed. It is more suitable for deployment on edge devices and can better balance accuracy and computing speed. However, in dense crowd scenes, the different distances of pedestrians lead to large differences in feature scales, and the high overlap of the crowd leads to blurred boundaries, which often cause problems such as missed detection and false detection in the detection results. Therefore, how to comprehensively and accurately detect the target human body in dense crowd scenes is a technical problem that needs to be solved. Summary of the invention
[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a method, device and medium for dense crowd target detection. Through the multi-scale channel segmentation module and the generalized enhanced feature pyramid network structure, the large and small scale human features are effectively retained, and the incomplete human information that is occluded is prevented from being missed, so as to make the detection accurate.
[0005] The purpose of the present invention can be achieved by the following technical solutions:
[0006] According to one aspect of the present invention, a method for detecting a dense crowd target is provided. The method is based on YOLOv8-MGH and comprises the following steps:
[0007] S1. Preprocess the original dense crowd image dataset to obtain a training set and a test set;
[0008] S2, extracting features from the training set, and obtaining feature maps of four different scales through a multi-scale channel segmentation module;
[0009] S3, through the generalized enhanced feature pyramid network structure, the feature maps of different scales are fused to obtain multi-scale features, deep features, semantic information and spatial information, and the feature maps of different scales are input into the detection module;
[0010] S4. Predict the human body position based on the learned features and information and calculate the loss function.
[0011] The target detection head used in the method includes a standard detection head and a small target detection head, and the small target detection head is used to detect background human bodies and occluded human bodies.
[0012] Furthermore, the preprocessing of S1 is to adjust the resolution of the original dense crowd image to obtain a dense crowd data set pair, the data set pair including the processed dense crowd image and the corresponding position label. The dense crowd data set pair is divided into a training set and a test set according to a ratio of 9:1.
[0013] Furthermore, the multi-scale channel segmentation module of S2 includes four pooling cores for obtaining feature maps of four different scales and retaining large-scale and small-scale
[0014] The feature maps of the four different scales are spliced according to the depth dimension.
[0015] Furthermore, the generalized enhanced feature pyramid network structure of S3 fuses the high-scale and low-scale feature maps obtained by the feature extraction with the current-scale feature map, retains deep and shallow feature information, obtains semantic information and spatial information, and enables shallow feature information to directly reach the detection head.
[0016] Furthermore, the loss function of S4 includes classification loss, and the classification loss uses improved cross entropy loss to measure the performance of the model by calculating the cross entropy between the probability distribution predicted by the model and the probability distribution of the true label, and the expression is:
[0017]
[0018] Among them, q is the IoU of the predicted box and the true box, that is, the intersection of the predicted box and the true box divided by the union of the two boxes; p is the score, if the boxes intersect, that is, q>0, it is a positive sample, if the boxes do not intersect, that is, q=0, it is a negative sample; α and γ are adjustment factors.
[0019] The loss function also includes regression loss, which includes CIoU loss and DistributionFocal loss; the expression of CIoU loss is:
[0020]
[0021] Where IoU is the IoU value between the predicted boundary and the true boundary box; ρ is the Euclidean distance between the center points of the predicted boundary box and the true boundary box; c is the sum of the diagonal lengths of the predicted boundary and the true boundary box; v is the penalty term in the logarithmic space of the width and height of the predicted boundary and the true boundary box; α and γ are adjustment factors; b is the bounding box predicted by the model; b gt is the bounding box of the real annotation, and is the correct answer read from the dataset;
[0022] The expression of Distribution Focal loss is:
[0023]
[0024] Among them, s i is the s-shaped output of the network; γ i and i+1 is the interval order and γ is the label.
[0025] According to a second aspect of the present invention, there is provided an electronic device, comprising a memory and a processor, wherein a computer program is stored in the memory, and the method described above is implemented when the processor executes the program.
[0026] According to a third aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein the program implements the method described above when executed by a processor.
[0027] Compared with the prior art, the present invention has the following beneficial effects:
[0028] (1) Improving detection accuracy: The present invention can effectively retain more incomplete human body information that is occluded by each other through a multi-scale channel segmentation module and a generalized enhanced feature pyramid network structure. In addition, through four detection modules of different scales, the sensitivity of the feature network to small targets is enhanced, solving the problem of missed detection of small human targets in the back row, thereby improving the accuracy of target detection in dense crowds.
[0029] (2) Enhanced robustness of the detection system: In the case of overlapping human targets and small human targets in the background, the recognition detection rate is maintained at a high level, showing good robustness.
[0030] (3) Wide range of applications: It is suitable for dense scenarios of different scales and complexities and can be used on edge devices, which is of great significance for monitoring safety hazards and controlling the flow of people. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 Flowchart of the method for detecting targets in dense crowds;
[0032] Figure 2This is a network framework diagram of the dense crowd target detection method;
[0033] Figure 3 It is the structural diagram of the multi-scale channel segmentation module;
[0034] Figure 4 This is the structural diagram of the generalized enhanced feature pyramid network;
[0035] Figure 5 The figure is a comparison chart of the detection results of dense crowd targets by the method of the present invention and the existing method. DETAILED DESCRIPTION
[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0037] like Figure 1 As shown in FIG. 1 , a dense crowd target detection method is based on YOLOv8-MGH, which includes the following steps:
[0038] S1. Preprocess the original dense crowd image dataset to obtain a training set and a test set;
[0039] S2, extracting features from the training set, and obtaining feature maps of four different scales through a multi-scale channel segmentation module;
[0040] S3, through the generalized enhanced feature pyramid network structure, the feature maps of different scales are fused to obtain multi-scale features, deep features, semantic information and spatial information, and the feature maps of different scales are input into the detection module;
[0041] S4. Predict the human body position based on the learned features and information and calculate the loss function.
[0042] The network diagram of this method is as follows Figure 2 The target detection heads used include a standard detection head and a small target detection head, wherein the small target detection head is used to detect background human bodies and occluded human bodies.
[0043] The preprocessing in S1 is to adjust the resolution of the original dense crowd image and unify the resolution to 640×640, and obtain 19370 dense crowd dataset pairs, including the processed dense crowd image and the corresponding position label. The dense crowd dataset pairs are divided into training sets and test sets according to the ratio of 9:1, that is, 17433 dense crowd dataset pairs are randomly selected for training, and the remaining 1937 dense crowd images are used for testing.
[0044] S2 is the feature extraction stage, and the specific steps are:
[0045] S201, inputting the dense crowd image in the training set into the feature extraction network for training;
[0046] S202, performing a step-by-step feature extraction process on the input dense crowd image through a composite convolution module and a C2F module;
[0047] S203, through the multi-scale channel segmentation module, the input feature map is divided into four parts in the channel dimension, and then respectively subjected to average pooling, 1×1 convolution and upsampling operations, and finally integrated into a feature map.
[0048] like Figure 3 The figure shows a multi-scale channel segmentation module, which includes four pooling kernels for obtaining feature maps of four different scales and retaining feature information of large and small scales. The feature maps of the four different scales are spliced according to the depth dimension. In the four branches, multi-scale features are extracted by setting pooling kernels of different sizes to enhance the model's perception of human features of different scales. The feature map is normalized to between 0 and 1 through average pooling and activation function; it is multiplied with the original input feature map on the channel, so that the model can dynamically filter channels with more feature information and suppress those channels with fewer features. Residual connections are also added to obtain the final output of the module.
[0049] S204: Obtain feature maps of four different scales.
[0050] S3 is the feature fusion stage, and the specific steps are as follows:
[0051] S301, obtaining feature maps of four scales obtained by the feature extraction network, and inputting them into the feature fusion network respectively;
[0052] S302, performing feature fusion through a generalized enhanced feature pyramid structure, first performing bottom-up fusion, fusing a deeper feature map of the feature fusion network with a current level and a shallower level feature map of the feature extraction network;
[0053] S303, performing top-down fusion again, adjusting the channels of the obtained feature map, and then fusing it with the feature maps of the current level and deeper levels;
[0054] S304, obtaining feature maps of four different scales that integrate multi-scale features, deep features, spatial and semantic information for final detection.
[0055] like Figure 4The generalized enhanced feature pyramid network structure shown here fuses the high-scale and low-scale feature maps extracted from the feature with the current scale feature map, retains deep and shallow feature information, obtains semantic information and spatial information, and allows shallow feature information to directly reach the detection head.
[0056] S4 is the detection stage, and the specific steps include:
[0057] S401, input the feature maps of the four scales generated in S3 into the detection module for detection;
[0058] S402: Give the predicted position and category based on the learned features, and calculate the loss function.
[0059] The loss function includes classification loss, which uses improved cross entropy loss to measure the performance of the model by calculating the cross entropy between the probability distribution predicted by the model and the probability distribution of the true label. The expression is:
[0060]
[0061] Among them, q is the IoU of the predicted box and the true box, that is, the intersection of the predicted box and the true box divided by the union of the two boxes; p is the score, if the boxes intersect, that is, q>0, it is a positive sample, if the boxes do not intersect, that is, q=0, it is a negative sample; α and γ are adjustment factors.
[0062] The loss function also includes regression loss, which includes CIoU loss and Distribution Focal loss; the expression of CIoU loss is:
[0063]
[0064] Where IoU is the IoU value between the predicted boundary and the true boundary box; ρ is the Euclidean distance between the center points of the predicted boundary box and the true boundary box; c is the sum of the diagonal lengths of the predicted boundary and the true boundary box; v is the penalty term in the logarithmic space of the width and height of the predicted boundary and the true boundary box; α and γ are adjustment factors; b is the bounding box predicted by the model; b gt is the bounding box of the real annotation, and is the correct answer read from the dataset;
[0065] The expression of Distribution Focal loss is:
[0066]
[0067] Among them, s i is the s-shaped output of the network; γ i and i+1 is the interval order and γ is the label.
[0068] The method uses the SGD optimization algorithm, the learning rate is 0.01, the learning rate adjustment strategy uses cosine learning rate descent, the batch size is 8, and a total of 150 iterations of training are performed.
[0069] This example selects a portion of images from the public dense crowd image datasets CrowdHuman and Widerperson for performance verification, and performs human instance detection on dense popular images. It is compared with existing target detection methods: Faster-rcnn, SSD, YOLOv5, YOLOv7, and YOLOv8. The detection effect is shown in the figure below. Figure 5 As shown. Figure 5 The comparison results shown show that this embodiment can accurately detect human targets when there are human bodies of different scales in the front and back of the image, or when the human bodies are blocked by each other. It also has a good detection effect in terms of the number of detected human bodies and faces.
[0070] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the described module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.
[0071] The electronic device of the present invention includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) or loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for device operation can also be stored. The CPU, ROM and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.
[0072] A plurality of components in the device are connected to the I / O interface, including: an input unit, such as a keyboard, a mouse, etc.; an output unit, such as various types of displays, speakers, etc.; a storage unit, such as a disk, an optical disk, etc.; and a communication unit, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit allows the device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks. The processing unit performs the various methods and processes described above, such as the method of the present invention. For example, in some embodiments, the method of the present invention may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via a ROM and / or a communication unit. When the computer program is loaded into the RAM and executed by the CPU, one or more steps of the method of the present invention described above may be performed. Alternatively, in other embodiments, the CPU may be configured to perform the method of the present invention in any other appropriate manner (e.g., by means of firmware).
[0073] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
[0074] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.
[0075] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0076] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A method for detecting dense crowd targets, the method is based on YOLOv8-MGH, characterized in that: The following steps are involved: S1. Preprocess the original dense crowd image dataset to obtain a training set and a test set; S2, extracting features from the training set, and obtaining feature maps of four different scales through a multi-scale channel segmentation module; S3, through the generalized enhanced feature pyramid network structure, the feature maps of different scales are fused to obtain multi-scale features, deep features, semantic information and spatial information, and the feature maps of different scales are input into the detection module; S4, the detection module predicts the human body position based on the learned features and information and calculates the loss function.
2. A method for detecting dense crowd targets according to claim 1, characterized in that: The target detection heads used include a standard detection head and a small target detection head. The small target detection head is used to detect background human bodies and occluded human bodies.
3. A method for detecting dense crowd targets according to claim 1, characterized in that: The preprocessing of S1 is to adjust the resolution of the original dense crowd image to obtain a dense crowd data set pair, and the data set pair includes the processed dense crowd image and the corresponding position label.
4. The method for detecting dense crowd targets according to claim 1, characterized in that: The multi-scale channel segmentation module of S2 includes four pooling cores for obtaining feature maps of four different scales and retaining large-scale and small-scale feature information.
5. A method for detecting dense crowd targets according to claim 4, characterized in that: The four feature maps of different scales are spliced according to the depth dimension.
6. A method for detecting dense crowd targets according to claim 1, characterized in that: The generalized enhanced feature pyramid network structure of S3 fuses the high-scale and low-scale feature maps obtained by feature extraction with the current scale feature map, retains deep and shallow feature information, obtains semantic information and spatial information, and allows shallow feature information to directly reach the detection head.
7. A method for detecting dense crowd targets according to claim 1, characterized in that: The loss function of S4 includes classification loss, which uses improved cross entropy loss to measure the performance of the model by calculating the cross entropy between the probability distribution predicted by the model and the probability distribution of the true label. The expression is: Among them, q is the IoU of the predicted box and the true box, that is, the intersection of the predicted box and the true box divided by the union of the two boxes; p is the score, if the boxes intersect, that is, q>0, it is a positive sample, if the boxes do not intersect, that is, q=0, it is a negative sample; α and γ are adjustment factors.
8. A method for detecting dense crowd targets according to claim 7, characterized in that: The loss function also includes regression loss, which includes CIoU loss and Distribution Focal loss; the expression of CIoU loss is: Where IoU is the IoU value between the predicted boundary and the true boundary box; ρ is the Euclidean distance between the center points of the predicted boundary box and the true boundary box; c is the sum of the diagonal lengths of the predicted boundary and the true boundary box; v is the penalty term in the logarithmic space of the width and height of the predicted boundary and the true boundary box; α and γ are adjustment factors; b is the bounding box predicted by the model; b gt is the bounding box of the real annotation, and is the correct answer read from the dataset; The expression of Distribution Focal loss is: Among them, s i is the s-shaped output of the network; γ i and i+1 is the interval order and γ is the label.
9. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.