A crowd abnormal behavior detection method and system based on kernel density estimation

Abstract

The present application relates to the field of computer vision and intelligent monitoring technology, and provides a crowd abnormal behavior detection method and system based on kernel density estimation. The method comprises the following steps: detecting all pedestrians in each frame of the acquired video stream to obtain a pedestrian position data list; performing kernel density estimation on the pedestrian position data list obtained for each frame to obtain a density value of each frame; constructing a density sequence based on the crowd density values of the last N frames; performing smoothing processing on the density sequence by using a Hanning window filter; setting a sliding window size and a step length, sliding the sliding window on the smoothed density sequence, calculating the density value of each window to obtain a relative change amount between two adjacent windows; and if the relative change amount between at least two adjacent windows is greater than a set threshold, a crowd abnormal behavior occurs. The present application can significantly improve the real-time performance, accuracy and robustness of crowd abnormal behavior detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision and intelligent monitoring technology, and particularly relates to a crowd abnormal behavior detection method and system based on kernel density estimation. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute the prior art.

[0003] With the acceleration of urbanization and the increase of population density, the flow and density of crowds in public places are constantly increasing. Large-scale gatherings, sports events, concerts, transportation hubs, subway stations, shopping malls and other places with large crowds are prone to abnormal behaviors such as gathering, rioting or dispersing due to unexpected events or other external factors. Timely and effective identification and handling of these abnormal behaviors are crucial to maintaining public order and ensuring personal safety.

[0004] Existing monitoring systems mostly rely on video streams and manual monitoring. Due to the complex dynamic changes of crowds and the limitations of manual observation, these systems are difficult to achieve real-time and accurate identification of abnormal crowd behavior. In recent years, with the rapid development of deep learning and computer vision technology, target detection technology has made significant progress, making it possible to analyze crowd behavior through automated means. However, existing crowd behavior detection systems mostly focus on crowd counting or simple congestion detection in static scenes, and have not fully considered the dynamic change characteristics of crowd behavior. Especially in handling abnormal behaviors, how to calculate the density change of the crowd and combine the information of space and time dimensions to accurately judge the gathering or dispersing behavior of the crowd is still a technical problem to be solved. SUMMARY

[0005] To solve the technical problems in the background art, the present application provides a crowd abnormal behavior detection method and system based on kernel density estimation. The present application automatically detects abnormal gathering or dispersing behavior of crowds and triggers an alarm in time through target detection and kernel density estimation, and sliding window analysis of density sequence, effectively improving the real-time and accuracy of dynamic management of crowds in public places.

[0006] To achieve the above purpose, the present application adopts the following technical solutions:

[0007] The first aspect of the present application provides a crowd abnormal behavior detection method based on kernel density estimation.

[0008] A crowd abnormal behavior detection method based on kernel density estimation, comprising:

[0009] detecting all pedestrians in each frame of the obtained video stream to obtain a list of pedestrian position data;

[0010] performing kernel density estimation on the list of pedestrian position data of each frame to obtain a density value of each frame;

[0011] constructing a density sequence based on the crowd density values of the last N frames;

[0012] performing smoothing processing on the density sequence by using a Hanning window filter;

[0013] setting a sliding window size and a step length, sliding the sliding window on the smoothed density sequence, calculating a density value of each window to obtain a relative change amount between adjacent two windows, and if the relative change amount between at least two adjacent windows is greater than a set threshold, a crowd abnormal behavior occurs.

[0014] Further, all pedestrians in each frame of the acquired video stream are detected to obtain a list of pedestrian position data; the method comprises: scaling each frame of the acquired video stream to a fixed size and performing normalization processing, inputting the processed image into an improved YOLOv8 model to obtain a detection box of each pedestrian; determining position information of each pedestrian according to a center point coordinate of the detection box of each pedestrian; recording the position information of all pedestrians in each frame to obtain the list of pedestrian position data; wherein the improved YOLOv8 model is a multi-axis attention module introduced in a neck network of the original YOLOv8 model, which is used to perform horizontal and vertical strip pooling operations on a feature map output by a C2f module, and then fuse the feature map output by the C2f module.

[0015] Further, the kernel density estimation is performed on the list of pedestrian position data of each frame to obtain a density value of each frame; the method comprises: generating a density function by using kernel density estimation on a center point position of each pedestrian in the list of pedestrian position data of each frame; performing smoothing processing on the positions of all pedestrians in the space based on the density function to obtain the density value of each frame.

[0016] Further, the Hanning window filter is used to perform smoothing processing on the density sequence; the method comprises: performing data expansion on the density sequence; performing weighted smoothing processing on the expanded density sequence by using a Hanning window with a set length; removing the first and last data in the smoothed density sequence, and only keeping the data of the length of the original density sequence before data expansion.

[0017] Further, the size and step of the sliding window are set, the sliding window is slid on the density sequence after smoothing processing, the density value of each window is calculated, and the relative change amount between adjacent two windows is obtained; the method comprises: setting the size and step of the sliding window, sliding the sliding window on the density sequence after smoothing processing, calculating the average density of the frames in each window to obtain the density value of each window; the relative change amount between adjacent two windows is obtained according to the difference between the density values of adjacent two windows.

[0018] Further, if the relative change amount between at least two adjacent windows is greater than a set threshold value, crowd abnormal behavior occurs; the method comprises: introducing a voting method, that is, the relative change amounts between multiple windows in a time period are detected, and if the relative change amount between at least two windows is greater than a set threshold value, it is considered that crowd abnormal behavior occurs.

[0019] Further, after the crowd abnormal behavior occurs, a unique event identifier is assigned to each abnormal event, and a frame counter is started; if the frame number of the abnormal event exceeds a set number threshold value, an alarm is triggered, and an abnormal event survival identifier is set to 1; a alarm control thread is started, and the states of all abnormal events are checked every preset time, if the event identifier of each abnormal event is not empty and the abnormal event survival identifier is 0, it is judged whether the abnormal event has triggered an alarm; if the alarm has not been triggered, it is considered that the abnormal event is a false alarm, and all information of the abnormal event is reset; if the alarm has been triggered, the video is stopped from being saved, the alarm video is pushed, and all information of the abnormal event is reset.

[0020] The second aspect of the application provides a crowd abnormal behavior detection system based on kernel density estimation.

[0021] A crowd abnormal behavior detection system based on kernel density estimation, comprising:

[0022] A pedestrian position data acquisition module configured to detect all pedestrians in each frame of the acquired video stream to obtain a list of pedestrian position data;

[0023] A crowd density value calculation module configured to perform kernel density estimation on the list of pedestrian position data obtained for each frame to obtain a density value of each frame;

[0024] A density sequence maintenance module configured to construct a density sequence based on the crowd density values of the last N frames;

[0025] A smoothing processing module configured to perform smoothing processing on the density sequence by using a Hanning window filter;

[0026] The sliding window density comparison module is configured to: set the sliding window size and step size, slide the sliding window on the smoothed density sequence, calculate the density value of each window to obtain the relative change between two adjacent windows; if the relative change between at least two consecutive adjacent windows is greater than a set threshold, abnormal crowd behavior occurs.

[0027] A third aspect of the present invention provides a computer-readable storage medium.

[0028] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for detecting abnormal crowd behavior based on kernel density estimation as described in the first aspect above.

[0029] A fourth aspect of the present invention provides a computer device.

[0030] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the steps of the method for detecting abnormal crowd behavior based on kernel density estimation as described in the first aspect above.

[0031] Compared with the prior art, the beneficial effects of the present invention are:

[0032] This invention proposes a method and system for detecting abnormal crowd behavior based on kernel density estimation. Utilizing an improved YOLOv8 model, the system can more accurately detect pedestrian positions in videos, ensuring high real-time efficiency and accuracy. Simultaneously, kernel density estimation (KDE) is employed to calculate the crowd density value for each frame, accurately reflecting crowd distribution and avoiding errors inherent in traditional methods. A Hanning window filter is used to smooth the density sequence, reducing noise interference and making density changes more stable, thus improving the accuracy of abnormal behavior detection.

[0033] This invention employs a sliding window mechanism to further enhance the system's robustness. By comparing density changes across different time periods, it can capture highly correlated behavioral changes. Combined with a comprehensive judgment of density changes across multiple windows, it ensures the accuracy of abnormal behavior detection. Upon detecting abnormal behavior and consistently exceeding a certain threshold, an alarm mechanism is triggered, recording and pushing the abnormal video. The alarm process is managed by an independent thread, ensuring that the video is automatically saved after the event ends.

[0034] This invention can significantly improve the real-time performance, accuracy, and robustness of abnormal crowd behavior detection. It has the advantages of low noise interference, strong real-time performance, and wide adaptability, effectively improving the safety monitoring and emergency response capabilities of public places. Attached Figure Description

[0035] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0036] Figure 1 This is a flowchart illustrating the abnormal behavior detection method for crowds based on kernel density estimation as shown in this invention;

[0037] Figure 2 This is a model structure diagram of the MAA module shown in this invention;

[0038] Figure 3 This is a structural diagram of the improved YOLOv8 model shown in this invention;

[0039] Figure 4 This is a flowchart illustrating the alarm triggering process shown in this invention;

[0040] Figure 5 This is a flowchart illustrating the process of ending an alarm, as shown in this invention;

[0041] Figure 6 This is a framework diagram of the abnormal crowd behavior detection system based on kernel density estimation shown in this invention. Detailed Implementation

[0042] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0043] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0044] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0045] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and systems according to various embodiments of this disclosure. It should be noted that each block in a flowchart or block diagram may represent a module, segment, or portion of code, which may include one or more executable instructions for implementing the logical functions specified in the various embodiments. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutively represented blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, may be implemented using a dedicated hardware-based system that performs the specified functions or operations, or using a combination of dedicated hardware and computer instructions.

[0046] Example 1

[0047] like Figure 1 As shown, this embodiment provides a method for detecting abnormal crowd behavior based on kernel density estimation. This embodiment uses the application of this method to a server as an example for illustration. It is understood that this method can also be applied to terminals, and can also be applied to systems including terminals, servers, and other components, and can be implemented through interaction between the terminal and the server. The server can be an independent physical server, a server cluster composed of multiple physical servers, or a distributed system. It can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communication, middleware services, domain name services, CDN security services, and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein. In this embodiment, the method includes the following steps:

[0048] Detect all pedestrians in each frame of the acquired video stream to obtain a list of pedestrian location data;

[0049] Kernel density estimation is performed on the list of pedestrian location data obtained for each frame to obtain the density value of each frame;

[0050] Construct a density sequence based on the crowd density values ​​of the most recent N frames;

[0051] The density sequence is smoothed using a Hanning window filter;

[0052] Set the sliding window size and step size, slide the sliding window on the smoothed density sequence, calculate the density value of each window to obtain the relative change between two adjacent windows; if the relative change between at least two consecutive adjacent windows is greater than the set threshold, abnormal crowd behavior occurs.

[0053] This invention proposes a method for detecting abnormal crowd behavior based on kernel density estimation. By detecting pedestrian positions in real time within a video, the system ensures high real-time efficiency and accuracy. Simultaneously, kernel density estimation is used to calculate the crowd density value for each frame, accurately reflecting crowd distribution and avoiding errors inherent in traditional methods. A Hanning window filter is applied to smooth the density sequence, reducing noise interference and making the density sequence changes more stable, thereby improving the accuracy of abnormal behavior detection.

[0054] Specifically, the abnormal crowd behavior detection method based on kernel density estimation described in this embodiment includes:

[0055] Step (1): Pedestrian location information collection.

[0056] To capture rich pedestrian-related features and improve the accuracy of pedestrian recognition, this invention uses an improved YOLOv8 model for detection, introducing a multi-axis attention (MAA) module into its neck network, such as... Figure 2 As shown, existing attention modules suffer from insufficient generalization ability due to the use of batch normalization, making it difficult to effectively capture long-range dependencies and potentially reducing the channel dimension of feature maps. In contrast, the MAA module uses strip pooling to capture long-range dependencies in both horizontal and vertical directions. This allows it to preserve the features of the target region and ignore irrelevant information without increasing complexity or reducing channel dimension, thus significantly improving the performance and generalization ability of deep neural network models.

[0057] Specifically, this involves integrating the MAA module into the neck network portion of the YOLOv8 model, such as... Figure 3 As shown. The output of the C2f module. As input, and using strip pooling on its output, the operation is performed along the horizontal direction (H,1) and the vertical direction (1,W) in each channel of the feature map, respectively, to generate a feature representation in the c-th channel with height h and width w.

[0058]

[0059]

[0060] After obtaining feature maps in the horizontal and vertical directions and Then, one-dimensional convolution Conv1d and group normalization GraoupNorm are used to enhance and process spatial information, and the Sigmoid function is chosen as the activation function.

[0061] z H =σ(GraoupNorm(Conv1d(y H )))

[0062] z W =σ(GroupNorm(Conv1d(y W )))

[0063] in,

[0064] Finally, the output x of C2f will be... c Multiplying the attention maps in both directions yields the output of the MAA module.

[0065] Y = x c ×z H ×z W

[0066] This invention effectively improves the model's accuracy in recognizing pedestrians in complex scenarios by adding a MAA module, providing a reliable data source for collecting pedestrian location information.

[0067] This invention uses the YOLOv8 model to collect pedestrian location information, and mainly includes the following steps:

[0068] Step (1-1): Scale the video frames acquired from the camera to the fixed size (640x640) required by the model and normalize them to ensure the image fits the network input. The preprocessed image is then fed into a YOLOv8 deep convolutional neural network, where features are extracted layer by layer. The improved YOLOv8 model can effectively detect pedestrian targets of different sizes, thus ensuring the recognition effect of both long-distance and short-distance targets.

[0069] Step (1-2): Output the coordinates of the pedestrian detection box (x) min ,y min ,x max ,y max The position coordinates of each pedestrian are calculated using the formula for the center point of the detection box.

[0070] In each frame of the video stream, the coordinates of the center points of all detected pedestrians will be recorded, forming a pedestrian location data list X = [x1, x2, ..., x...]. nThis list contains the locations of all pedestrians in the frame and is updated in real time for subsequent crowd density calculations. This process captures changes in pedestrian positions in real time from the camera feed, providing fundamental data for subsequent density analysis.

[0071] Step (2): Calculate the population density value.

[0072] Kernel density estimation (KDE) is a nonparametric statistical method used to estimate the distribution of data in a multidimensional space. Based on the obtained pedestrian location data list X = [x1, x2, ..., x...] for each frame... n The basic form of kernel density estimation is:

[0073]

[0074] in, Here, is the estimated density value at position x, n is the number of data points (i.e., the number of people detected in the frame), h is the bandwidth parameter, controlling the smoothness of the kernel function, d is the dimension of the data (2 in this method), representing the coordinates in the plane, and K(·) is the kernel function, measuring the relationship between the current data point x and each data point x. i The similarity between them.

[0075] The kernel function K is usually a symmetric function. Common kernel functions include Gaussian kernels, uniform kernels, and triangular kernels. This invention chooses the Gaussian kernel, whose function is as follows:

[0076]

[0077] in, It is the normalized distance. The reason for choosing a Gaussian kernel is that it can assign a smooth weight to each data point, so that data points farther away from the current point x contribute less to the density at that location, while data points closer to the current point x contribute more to the density.

[0078] Bandwidth h is a key parameter in kernel density estimation, determining the "width" of the kernel function. If the bandwidth h is too large, the estimated density function will be too smooth, leading to a loss of detail; if h is too small, the density function may be too sharp and susceptible to noise interference. In this invention, the bandwidth h is set to 1.0, and its adjustment can optimize the detection performance.

[0079] For each frame, the calculated pedestrian location data list is used to calculate crowd density using kernel density estimation (KDE). Specifically, KDE generates a density function based on the center point of each pedestrian and smooths the positions of all pedestrians in space, ultimately calculating the average density value for a frame. Higher crowd density indicates a more concentrated distribution of pedestrians in that frame; conversely, lower density indicates a more dispersed distribution. The results of KDE can effectively reflect the degree of crowd aggregation in the current frame, providing a quantitative density basis for subsequent abnormal behavior detection.

[0080] Step (3): Density sequence maintenance.

[0081] In order to continuously track and analyze changes in population density, this invention sets up a density sequence D of size N = [d1, d2, ..., dn]. N This queue is used to store the crowd density values ​​of the most recent N frames. Whenever a new frame is detected, the system updates the density sequence, enqueuing the density value of the latest frame and dequeuing the density value of the oldest frame, keeping the queue length constant at N.

[0082] This density sequence is used to store crowd density information over a period of time, enabling the system to identify abnormal crowd behavior based on continuous trends in the density sequence. This design allows the system to utilize historical information over a period, rather than relying solely on the data from the current frame, making the detection process more robust and reducing false positives caused by single-frame noise or short-term fluctuations.

[0083] Step (4): Density data smoothing.

[0084] In this invention, crowd density is calculated through kernel density estimation. However, due to noise in video frames, detection errors, and other factors, the density values ​​between frames may fluctuate significantly. Directly using these density values ​​to determine abnormal behavior may lead to false alarms. Therefore, this invention employs a Hanning window filter to smooth the density values, reducing noise interference and making density changes more consistent with actual crowd behavior trends.

[0085] Specifically, this invention uses a Hanning window filter to smooth the density values ​​in a density sequence. The Hanning window is a weighted average filtering method that effectively reduces high-frequency noise in a data sequence, making the density sequence changes smoother. Its mathematical expression is:

[0086]

[0087] Here, w(m) is the value of the window function, M is the length of the window, and m is the index within the window. The Hanning window is characterized by gradually decreasing to zero at the boundaries and approaching 1 in the middle, thus giving greater weight to values ​​at the center of the sequence and less weight to values ​​at the edges. This allows it to smooth the sequence and reduce edge effects and high-frequency noise.

[0088] In this invention, the density sequence D = [d1, d2, ..., d N This sequence contains crowd density values ​​from the most recent N frames. The Hanning window is used to smooth this density sequence, and the specific process is as follows:

[0089] Step (4-1): Density data expansion: Add a portion of data to the beginning and end of the density sequence (mirror expansion) to prevent abnormal fluctuations or jumps in the data at the beginning and end of the sequence, which would cause the data values ​​to deviate from the normal range and affect the accuracy of the model.

[0090] Step (4-2): Hanning window weighting: Use a Hanning window of length window_len to perform weighted smoothing on the expanded density sequence. The weighting process is completed using convolution operations.

[0091] Step (4-3): Remove padding data: In the smoothed sequence, the first and last... These values ​​are added due to expansion, so they need to be removed from the final result, keeping only the data of the original sequence length.

[0092] Through this process, the smoothed density sequence Q = [q1, q2, ..., q] is obtained. N This allows for a better capture of the true trends in population density, rather than being affected by momentary abnormal fluctuations.

[0093] Step (5): Sliding window density comparison

[0094] Since the changes in crowd density between frames are subtle, a sliding window approach is used for density comparison to more accurately analyze these changes. The specific process is as follows:

[0095] Step (5-1): Calculate the population density using the kernel density estimation method, and smooth the density sequence using the Hanning window filter to obtain the pre-processed density sequence Q = [q1, q2, ..., q N ].

[0096] Step (5-2): Set a sliding window of size w and a step size of s, and calculate the density value within each window. For each window, calculate the average density of all frames within that window as the density value of that window:

[0097]

[0098] Among them, Window i The window density value represents the overall population density over a period of time.

[0099] Step (5-3): For two consecutive sliding windows, calculate the relative change in window density. If the relative change is greater than a set threshold θ, i.e.:

[0100] Window i+1 -Window i |>θ

[0101] This indicates that significant changes in population behavior occurred during this period. A positive density change rate indicates that the average density of the current window is higher than that of the previous window, meaning the population has become denser, possibly indicating population clustering; a negative density change rate indicates that the average density of the current window is lower than that of the previous window, meaning the population has become sparser, possibly indicating population dispersion.

[0102] To avoid misjudgments caused by noise in a single window, a voting method is introduced. This involves detecting density changes across multiple windows within a given time period (a queue of frame density changes of length N). If the density changes of multiple windows exceed a threshold θ within this time period, abnormal behavior is considered to have occurred during that period. The voting method's judgment formula is as follows:

[0103]

[0104] in, It is an indicator function that takes the value 1 if the relative change between the front and back windows exceeds the threshold θ, and takes the value 0 otherwise. k is the voting threshold, which means that when the number of times the relative change between the front and back windows exceeds the threshold θ reaches k, it is considered that abnormal behavior has occurred.

[0105] The sliding window method allows for segmented processing of the density sequence and comparison of the average density changes between adjacent sliding windows. This approach effectively avoids the impact of single-frame fluctuations on overall judgment and better captures the trend changes in overall crowd behavior. The window size and stride settings will be optimized based on the actual scenario to ensure both timely response to density changes and maintain detection stability.

[0106] Step (6): Push alarm for abnormal events.

[0107] Once the system detects a density change exceeding a threshold and determines that abnormal behavior has occurred (such as rapid crowd gathering or dispersal), the system will automatically trigger an alarm mechanism. Figure 4 and Figure 5 As shown, the system's abnormal alarm process includes the following steps:

[0108] Step (6-1): Event Detection and Initialization: When the system detects abnormal behavior, it assigns a unique event identifier (eventID) and starts the frame counter (frame). Whenever the number of frames of an abnormal event exceeds the set threshold (∈), the system triggers an alarm, sets the alarm start counter (start) to 1, and sets the frame counter (frame) to 0, repeating this alarm cycle until the alarm ends. Simultaneously, the system begins using the video writer (video_writer) to save the frame data of the abnormal event in real time and sets the event live flag (live) to 1, indicating that an abnormal event is occurring.

[0109] Step (6-2): Video Saving and Alarm Control: During the event, the system continuously updates frame data and saves video clips in real time. If a new abnormal event occurs, the system updates the event identifier and continues to save the new video clips.

[0110] Step (6-3): Alarm End Monitoring Thread: The system starts an alarm control thread in parallel, checking the status of all events every 2 seconds. If an event's eventID is not empty and its live flag is 0, the system will determine whether the event has triggered an alarm.

[0111] If the alarm is not triggered (start=0), the event is considered a false alarm, and the system will reset all information about the event.

[0112] If an alarm has been triggered (start>0), the system will stop saving the video, push the alarm video, and reset all information about the event.

[0113] Step (6-4): Event termination processing: Through the above process, after confirming that the event has ended, the system restores the status of all events to the initial state in preparation for detecting the next abnormal event.

[0114] The real-time anomaly detection and alarm mechanism designed in this invention can effectively improve the safety management capabilities of public places. Through early warning, safety hazards caused by crowd gatherings or emergencies can be addressed promptly, reducing potential risks.

[0115] Example 2

[0116] This embodiment provides a system for detecting abnormal crowd behavior based on kernel density estimation.

[0117] like Figure 6 As shown, a crowd abnormal behavior detection system based on kernel density estimation includes:

[0118] The pedestrian location data acquisition module is configured to: detect all pedestrians in each frame of the acquired video stream and obtain a list of pedestrian location data;

[0119] The crowd density calculation module is configured to perform kernel density estimation on the list of pedestrian location data obtained for each frame to obtain the density value of each frame.

[0120] The density sequence maintenance module is configured to construct a density sequence based on the crowd density values ​​of the most recent N frames.

[0121] A smoothing module is configured to smooth the density sequence using a Hanning window filter;

[0122] The sliding window density comparison module is configured to: set the sliding window size and step size, slide the sliding window on the smoothed density sequence, calculate the density value of each window to obtain the relative change between two adjacent windows; if the relative change between at least two consecutive adjacent windows is greater than a set threshold, abnormal crowd behavior occurs.

[0123] In some embodiments, the pedestrian location data acquisition module is specifically configured to: scale each acquired video stream frame to a fixed size and normalize it; input the processed image into an improved YOLOv8 model to obtain the detection box for each pedestrian; determine the location information of each pedestrian based on the center point coordinates of the detection box for each pedestrian; record the location information of all pedestrians in each frame to obtain a pedestrian location data list; wherein, the improved YOLOv8 model is a multi-axis attention module introduced into the neck network of the original YOLOv8 model, which is used to perform strip pooling operations in both horizontal and vertical directions on the feature map output by the C2f module, and then fuse it with the feature map output by the C2f module.

[0124] In some embodiments, the crowd density value calculation module is specifically configured to: use kernel density estimation to generate a density function for the center point position of each pedestrian in the pedestrian position data list obtained in each frame; and based on the density function, smooth the position of all pedestrians in space to obtain the density value of each frame.

[0125] In some embodiments, the smoothing module is specifically configured to: perform data expansion on the density sequence; use a Hanning window of a set length to perform weighted smoothing on the expanded density sequence; remove the first and last data in the smoothed density sequence, and retain only the data of the original density sequence length before data expansion;

[0126] In some embodiments, the sliding window density comparison module is specifically configured to: set the sliding window size and step size, slide the sliding window on the smoothed density sequence, calculate the average density of the frames within each window, and obtain the density value of each window; and obtain the relative change between two adjacent windows based on the difference between the density values ​​of two adjacent windows.

[0127] The voting method is introduced, which involves detecting the relative changes between multiple windows within a certain time period. If the relative changes between at least two windows are greater than a set threshold, then abnormal behavior of the crowd is considered to have occurred.

[0128] In some embodiments, the system further includes: an abnormal event alarm module, configured to: assign a unique event identifier to each abnormal event after abnormal crowd behavior occurs, and start a frame counter; if the number of frames of the abnormal event exceeds a set threshold, trigger an alarm and set the abnormal event survival identifier to 1; start an alarm control thread to check the status of all abnormal events at preset intervals; if the event identifier of each abnormal event is not empty and the abnormal event survival identifier is 0, determine whether the abnormal event has triggered an alarm; if the alarm has not been triggered, consider the abnormal event to be a false alarm and reset all information of the abnormal event; if the alarm has been triggered, stop saving video, push alarm video, and reset all information of the abnormal event.

[0129] This invention automatically detects abnormal crowd gathering or dispersal behavior and triggers alarms in a timely manner by using target detection and kernel density estimation, combined with sliding window analysis of density sequences, effectively improving the real-time performance and accuracy of dynamic crowd management in public places.

[0130] Example 3

[0131] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the crowd abnormal behavior detection method based on kernel density estimation as described in Embodiment 1 above.

[0132] Example 4

[0133] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the method for detecting abnormal crowd behavior based on kernel density estimation as described in Embodiment 1 above.

[0134] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0135] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0136] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0137] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0138] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0139] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A crowd abnormal behavior detection method based on kernel density estimation, characterized in that, The method comprises the following steps: Detect all pedestrians in each frame of the acquired video stream to obtain a pedestrian position data list; the method comprises the following steps: Scale each frame of the acquired video stream to a fixed size and perform normalization processing, and input the processed image into an improved YOLOv8 model to obtain a detection box of each pedestrian; Determine the position information of each pedestrian according to the center point coordinates of the detection box of each pedestrian; Record the position information of all pedestrians in each frame to obtain a pedestrian position data list; The improved YOLOv8 model is a multi-axis attention module introduced in the neck network of the original YOLOv8 model, which is used to perform horizontal and vertical strip pooling operations on the feature maps output by the C2f module, and then fuse the feature maps output by the C2f module; Perform kernel density estimation on the pedestrian position data list obtained for each frame to obtain a density value of each frame; Based on the recent N frame crowd density value, construct a density sequence; Smooth the density sequence by using a Hanning window filter; Set the size and step length of the sliding window, slide the sliding window on the smoothed density sequence, calculate the density value of each window, and obtain the relative change amount between adjacent two windows; if the relative change amount between at least two adjacent windows is greater than a set threshold, a crowd abnormal behavior occurs.

2. The crowd abnormal behavior detection method based on kernel density estimation according to claim 1, characterized in that, The method for performing kernel density estimation on the pedestrian position data list obtained for each frame to obtain a density value of each frame comprises the following steps: using kernel density estimation, generating a density function for the center point position of each pedestrian in the pedestrian position data list obtained for each frame; based on the density function, performing smoothing processing on the positions of all pedestrians in the space to obtain a density value of each frame. 3.The crowd abnormal behavior detection method based on kernel density estimation of claim 1, wherein, The method for smoothing the density sequence by using a Hanning window filter comprises the following steps: Perform data expansion on the density sequence; Perform weighted smoothing processing on the expanded density sequence by using a Hanning window with a set length; Remove the first and last data in the smoothed density sequence, and only keep the data length of the original density sequence before data expansion.

4. The crowd abnormal behavior detection method based on kernel density estimation according to claim 1, characterized in that, The method for setting the size and step length of the sliding window, sliding the sliding window on the smoothed density sequence, calculating the density value of each window, and obtaining the relative change amount between adjacent two windows comprises the following steps: Set the size and step length of the sliding window, slide the sliding window on the smoothed density sequence, calculate the average density of the frames in each window, and obtain the density value of each window; According to the difference between the density values of adjacent two windows, obtain the relative change amount between adjacent two windows.

5. The crowd abnormal behavior detection method based on kernel density estimation according to claim 1 or 4, characterized in that, The method for determining whether a crowd abnormal behavior occurs when the relative change amount between at least two adjacent windows is greater than a set threshold comprises the following steps: introducing a voting method, that is, detecting the relative change amount between multiple windows within a time period, and if the relative change amount between at least two windows is greater than a set threshold, it is considered that a crowd abnormal behavior occurs.

6. The crowd abnormal behavior detection method based on kernel density estimation according to claim 1, characterized in that, After a crowd abnormal behavior occurs, a unique event identifier is assigned to each abnormal event, and a frame counter is started; If the number of frames of the abnormal event exceeds a set number threshold, an alarm is triggered, and the abnormal event survival identifier is set to 1; An alarm control thread is started, and every preset time, the status of all abnormal events is checked. If the alarm is not triggered, it is considered that the abnormal event is a false alarm, and all information of the abnormal event is reset; if the alarm has been triggered, the video is stopped saving, the alarm video is pushed, and all information of the abnormal event is reset.

7. A crowd abnormal behavior detection system based on kernel density estimation, characterized in that, Comprise: The pedestrian position data acquisition module is configured to detect all pedestrians in each frame of the acquired video stream to obtain a pedestrian position data list; the method comprises: scaling each frame of the acquired video stream to a fixed size and performing normalization processing, inputting the processed image into an improved YOLOv8 model to obtain a detection box of each pedestrian; determining the position information of each pedestrian according to the center point coordinates of the detection box of each pedestrian; recording the position information of all pedestrians in each frame to obtain a pedestrian position data list; The improved YOLOv8 model is a multi-axis attention module introduced in the neck network of the original YOLOv8 model, which is used to perform horizontal and vertical strip pooling operations on the feature maps output by the C2f module, and then fuse the feature maps output by the C2f module; The crowd density value calculation module is configured to perform kernel density estimation on the pedestrian position data list obtained for each frame to obtain a density value for each frame. a density sequence maintenance module configured to construct a density sequence based on the crowd density values of the most recent N frames; The smoothing processing module is configured to perform smoothing processing on the density sequence using a Hanning window filter. The sliding window density comparison module is configured to set a sliding window size and a step, slide the sliding window on the smoothed density sequence, calculate the density value of each window, and obtain the relative change amount between adjacent two windows; if the relative change amount between at least two adjacent windows is greater than a set threshold, a crowd abnormal behavior occurs.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to realize the steps in the crowd abnormal behavior detection method based on kernel density estimation according to any one of claims 1-6.

9. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to realize the steps in the crowd abnormal behavior detection method based on kernel density estimation according to any one of claims 1-6.

Images