Crowd abnormal behavior detection method and system based on kernel density estimation

Abstract

The invention relates to the technical field of computer vision and intelligent monitoring, and provides a crowd abnormal behavior detection method and system based on kernel density estimation. The method comprises the following steps: detecting all pedestrians of each frame in an obtained video stream to obtain a pedestrian position data list; performing kernel density estimation on the pedestrian position data list obtained in each frame to obtain a density value of each frame; constructing a density sequence based on the crowd density values of the latest N frames; smoothing the density sequence by using a Hanning window filter; setting the size and step length of a sliding window, sliding the sliding window on the smoothed density sequence, and calculating the density value of each window to obtain the relative variation between two adjacent windows; and if the relative variation between at least two continuous adjacent windows is greater than a set threshold value, determining that the abnormal behavior of the crowd occurs. According to the method, the real-time performance, precision and robustness of crowd abnormal behavior detection can be remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and intelligent monitoring, and in particular to a method and system for detecting abnormal behavior of a crowd based on kernel density estimation. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the acceleration of urbanization and the increase in population density, the mobility and density of people in public places are constantly increasing. Large gatherings, sports events, concerts, transportation hubs, subway stations, shopping malls and other crowded places are often prone to abnormal behaviors such as gathering, rioting, or dispersing due to emergencies or other external factors. Timely and effective identification and handling of these abnormal behaviors are crucial to maintaining public order and protecting personal safety.

[0004] Most existing monitoring systems rely on video streaming and manual monitoring. Due to the complex dynamic changes of crowds and the limitations of manual observation, it is difficult for these systems to accurately identify abnormal crowd behavior in real time. In recent years, with the rapid development of deep learning and computer vision technology, target detection technology has made significant progress, and it has become possible to analyze crowd behavior through automated means. However, existing crowd behavior detection systems mostly focus on counting people in static scenes or simple congestion detection, and have not fully considered the dynamic characteristics of crowd behavior. Especially when dealing with abnormal behavior, how to accurately judge the gathering or dispersion of crowds by calculating the density changes of the crowd and combining the information of spatial and temporal dimensions is still a technical problem that needs to be solved urgently. Summary of the invention

[0005] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a method and system for detecting abnormal crowd behavior based on kernel density estimation. The present invention automatically detects abnormal gathering or dispersion behavior of the crowd and triggers an alarm in time through target detection and kernel density estimation, combined with sliding window analysis of density sequences, thereby effectively improving the real-time and accuracy of dynamic management of crowds in public places.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A first aspect of the present invention provides a method for detecting abnormal crowd behavior based on kernel density estimation.

[0008] A method for detecting abnormal crowd behavior based on kernel density estimation, comprising:

[0009] Detect all pedestrians in each frame of the acquired video stream and obtain a list of pedestrian position data;

[0010] Perform kernel density estimation on the pedestrian position data list obtained in each frame to obtain the density value of each frame;

[0011] Based on the crowd density values ​​of the most recent N frames, a density sequence is constructed;

[0012] Using a Hanning window filter to smooth the density sequence;

[0013] Set the sliding window size and step size, slide the sliding window on the smoothed density sequence, calculate the density value of each window to obtain the relative change between two adjacent windows; if the relative change between at least two consecutive adjacent windows is greater than the set threshold, abnormal crowd behavior occurs.

[0014] Furthermore, the detection obtains all pedestrians in each frame of the video stream to obtain a pedestrian position data list; the method includes: scaling each frame of the obtained video stream to a fixed size, normalizing it, and inputting the processed image into an improved YOLOv8 model to obtain a detection frame for each pedestrian; determining the position information of each pedestrian according to the center point coordinates of the detection frame of each pedestrian; recording the position information of all pedestrians in each frame to obtain a pedestrian position data list; wherein the improved YOLOv8 model introduces a multi-axis attention module into the neck network of the original YOLOv8 model, which is used to perform strip pooling operations on the feature map output by the C2f module in both horizontal and vertical directions, and then fuse it with the feature map output by the C2f module.

[0015] Furthermore, kernel density estimation is performed on the pedestrian position data list obtained for each frame to obtain a density value for each frame; the method includes: using kernel density estimation to generate a density function for the center point position of each pedestrian in the pedestrian position data list obtained for each frame; based on the density function, smoothing the positions of all pedestrians in space to obtain a density value for each frame.

[0016] Furthermore, the density sequence is smoothed using a Hanning window filter; the method includes: data expansion of the density sequence; using a Hanning window of a set length to perform weighted smoothing on the expanded density sequence; removing the front and back data in the density sequence after smoothing, and only retaining data of the original density sequence length before data expansion.

[0017] Furthermore, the sliding window size and step size are set, the sliding window is slid on the density sequence after smoothing, and the density value of each window is calculated to obtain the relative change between two adjacent windows; the method includes: setting the sliding window size and step size, sliding the sliding window on the density sequence after smoothing, calculating the average density of the frames in each window, and obtaining the density value of each window; and obtaining the relative change between two adjacent windows according to the difference between the density values ​​of two adjacent windows.

[0018] Furthermore, if the relative change between at least two consecutive adjacent windows is greater than a set threshold, abnormal crowd behavior occurs; the method includes: introducing a voting method, that is, within a time period, detecting the relative change between multiple windows, if the relative change between at least two windows is greater than the set threshold, it is considered that abnormal crowd behavior occurs.

[0019] Furthermore, after abnormal crowd behavior occurs, a unique event identifier is assigned to each abnormal event, and a frame counter is started; if the number of frames of the abnormal event exceeds the set number threshold, an alarm is triggered, and the abnormal event survival identifier is set to 1; an alarm control thread is started to check the status of all abnormal events at preset intervals, and if the event identifier of each abnormal event is not empty and the abnormal event survival identifier is 0, it is determined whether the abnormal event has triggered an alarm; if the alarm is not triggered, the abnormal event is considered to be a false alarm, and all information about the abnormal event will be reset; if the alarm has been triggered, stop saving the video, push the alarm video, and reset all information about the abnormal event.

[0020] A second aspect of the present invention provides a system for detecting abnormal crowd behavior based on kernel density estimation.

[0021] A crowd abnormal behavior detection system based on kernel density estimation, comprising:

[0022] A pedestrian position data acquisition module is configured to: detect all pedestrians in each frame of the acquired video stream to obtain a pedestrian position data list;

[0023] The crowd density value calculation module is configured to: perform kernel density estimation on the pedestrian position data list obtained in each frame to obtain the density value of each frame;

[0024] A density sequence maintenance module is configured to: construct a density sequence based on crowd density values ​​of the most recent N frames;

[0025] A smoothing processing module, which is configured to: use a Hanning window filter to smooth the density sequence;

[0026] The sliding window density comparison module is configured as follows: setting the sliding window size and step size, sliding the sliding window on the smoothed density sequence, calculating the density value of each window to obtain the relative change between two adjacent windows; if the relative change between at least two consecutive adjacent windows is greater than the set threshold, abnormal crowd behavior occurs.

[0027] A third aspect of the present invention provides a computer-readable storage medium.

[0028] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the method for detecting abnormal crowd behavior based on kernel density estimation as described in the first aspect above.

[0029] A fourth aspect of the present invention provides a computer device.

[0030] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the method for detecting abnormal crowd behavior based on kernel density estimation as described in the first aspect above are implemented.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] The present invention proposes a method and system for detecting abnormal crowd behavior based on kernel density estimation. The improved YOLOv8 model can more accurately detect the position of pedestrians in the video, ensuring that the system has high efficiency, real-time performance and accuracy. At the same time, the kernel density estimation (KDE) is used to calculate the crowd density value of each frame, accurately reflecting the crowd distribution and avoiding errors in traditional methods. The density sequence is smoothed by the Hanning window filter to reduce noise interference, make the density change more stable, and improve the accuracy of abnormal behavior judgment.

[0033] The present invention uses a sliding window mechanism to further enhance the robustness of the system. By comparing the density changes in different time periods, it can capture the behavior changes with strong correlation, and combine the comprehensive judgment of multi-window density changes to ensure the accuracy of abnormal behavior detection. After the abnormal behavior is detected and continues to exceed a certain threshold, the alarm mechanism is triggered to record and push the abnormal video. The alarm process is managed by an independent thread to ensure that the video is automatically saved after the event ends.

[0034] The present invention can significantly improve the real-time, accuracy and robustness of abnormal crowd behavior detection, has the advantages of low noise interference, strong real-time performance and wide adaptability, and effectively improves the safety monitoring and emergency response capabilities of public places. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0036] Figure 1 is a flow chart of a method for detecting abnormal crowd behavior based on kernel density estimation shown in the present invention;

[0037] Figure 2 It is a model structure diagram of the MAA module shown in the present invention;

[0038] Figure 3 It is a structural diagram of the improved YOLOv8 model shown in the present invention;

[0039] Figure 4 It is a flowchart of triggering an alarm shown in the present invention;

[0040] Figure 5 It is a flowchart of ending alarm shown in the present invention;

[0041] Figure 6 It is a framework diagram of a crowd abnormal behavior detection system based on kernel density estimation shown in the present invention. DETAILED DESCRIPTION

[0042] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0043] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0044] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0045] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the methods and systems according to various embodiments of the present disclosure. It should be noted that each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of a code may include one or more executable instructions for implementing the logical functions specified in each embodiment. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the flowchart and / or block diagram, and the combination of boxes in the flowchart and / or block diagram can be implemented using a dedicated hardware-based system that performs a specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0046] Embodiment 1

[0047] like Figure 1 As shown, this embodiment provides a method for detecting abnormal behavior of a crowd based on kernel density estimation. This embodiment uses the method applied to a server as an example for illustration. It can be understood that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communications, middleware services, domain name services, security services CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in this application. In this embodiment, the method includes the following steps:

[0048] Detect all pedestrians in each frame of the acquired video stream and obtain a list of pedestrian position data;

[0049] Perform kernel density estimation on the pedestrian position data list obtained in each frame to obtain the density value of each frame;

[0050] Based on the crowd density values ​​of the most recent N frames, a density sequence is constructed;

[0051] Using a Hanning window filter to smooth the density sequence;

[0052] Set the sliding window size and step size, slide the sliding window on the smoothed density sequence, calculate the density value of each window to obtain the relative change between two adjacent windows; if the relative change between at least two consecutive adjacent windows is greater than the set threshold, abnormal crowd behavior occurs.

[0053] The present invention proposes a method for detecting abnormal crowd behavior based on kernel density estimation. By real-time detection of pedestrian positions in the video, the system is ensured to have high efficiency, real-time performance and accuracy. At the same time, the density value of each frame is calculated by kernel density estimation to accurately reflect the distribution of the crowd and avoid errors in traditional methods. The density sequence is smoothed by the Hanning window filter to reduce noise interference, making the density sequence change more stable and improving the accuracy of abnormal behavior judgment.

[0054] Specifically, the method for detecting abnormal crowd behavior based on kernel density estimation described in this embodiment includes:

[0055] Step (1): Collect pedestrian location information.

[0056] In order to capture rich features related to pedestrians and improve the accuracy of pedestrian recognition, the present invention uses an improved YOLOv8 model for detection and introduces a multi-axis attention module (Multi-Axis Attention, MAA) into its neck network. Figure 2 As shown in the figure. The existing attention module has insufficient generalization ability due to the use of batch normalization, making it difficult to effectively capture long-distance dependencies and possibly reducing the channel dimension of the feature map. In contrast, the MAA module obtains long-distance dependencies in both horizontal and vertical directions through strip pooling technology, which can retain the features of the target area and ignore irrelevant information without increasing complexity or reducing channel dimensions, thereby significantly improving the performance and generalization ability of the deep neural network model.

[0057] Specifically, the MAA module is integrated into the neck network part of the YOLOv8 model, such as Figure 3 As shown. The output of the C2f module As input, a strip pooling operation is applied to its output, which is performed on each channel of the feature map along the horizontal direction (H, 1) and the vertical direction (1, W), generating a feature representation on the cth channel at height h and width w.

[0058]

[0059]

[0060] After obtaining the feature maps in the horizontal and vertical directions and Finally, one-dimensional convolution Conv1d and group normalization GraoupNorm are used to enhance and process spatial information, and the Sigmoid function is selected as the activation function.

[0061] z H =σ(GraoupNorm(Conv1d(y H )))

[0062] z W =σ(GroupNorm(Conv1d(y W )))

[0063] in,

[0064] Finally, the output x of C2f c Multiply it with the attention map in both directions to get the output of the MAA module

[0065] Y = x c × H × W

[0066] By adding the MAA module, the present invention effectively improves the model's recognition accuracy of pedestrians in complex scenes and provides a reliable data source for collecting pedestrian location information.

[0067] The present invention uses the YOLOv8 model to collect pedestrian location information, which mainly includes the following steps:

[0068] Step (1-1): Scale the video frames obtained from the camera to the fixed size (640x640) required by the model and normalize them to ensure that the image fits the network input. The preprocessed image enters the YOLOv8 deep convolutional neural network to extract features layer by layer. The improved YOLOv8 model can effectively detect pedestrian targets of different sizes, thereby ensuring the recognition effect of long-distance and close-range targets.

[0069] Step (1-2): Output the coordinates of the pedestrian detection frame (x min ,y min ,x max ,y max ), calculate the position coordinates of each pedestrian through the center point formula of the detection frame

[0070] In each frame of the video stream, the coordinates of the center points of all detected pedestrians will be recorded to form a pedestrian position data list X = [x1, x2, ..., x n]. This list contains the positions of all pedestrians in the frame, and is updated in real time for each frame for subsequent crowd density calculation. This process can capture the position changes of pedestrians in the camera image in real time, providing basic data for subsequent density analysis.

[0071] Step (2): Calculation of crowd density value.

[0072] Kernel Density Estimation (KDE) is a nonparametric statistical method used to estimate the distribution of data in multidimensional space. According to the pedestrian position data list X = [x1, x2, ..., x n ], the basic form of kernel density estimation is:

[0073]

[0074] in, is the estimated density value at position x, n is the number of data points (i.e., the number of pedestrians detected in the frame), h is the bandwidth parameter, which controls the smoothness of the kernel function, d is the dimension of the data, which is 2 in this method, representing the coordinates in the plane, and K(·) is the kernel function, which measures the difference between the current data point x and each data point x. i The similarities between.

[0075] The kernel function K is usually a symmetric function. Common kernel functions include Gaussian kernel, uniform kernel, triangular kernel, etc. The present invention selects Gaussian kernel here. The form of Gaussian kernel function is:

[0076]

[0077] in, is the normalized distance. The reason why the Gaussian kernel is chosen is that it can assign a smooth weight to each data point, so that data points farther away from the current point x contribute less to the density of the position, while data points closer to the current point x contribute more to the density.

[0078] Bandwidth h is a key parameter in kernel density estimation, which determines the "width" of the kernel function. If the bandwidth h is too large, the estimated density function will be too smooth, resulting in loss of details; if h is too small, the density function may be too sharp and interfered by noise. In the present invention, the bandwidth h is set to 1.0, and the detection effect can be optimized by adjusting it.

[0079] For the pedestrian position data list calculated in each frame, the kernel density estimation method is used to calculate the crowd density. Specifically, KDE generates a density function based on the center point position of each pedestrian, and smoothes the positions of all pedestrians in space, and finally calculates the average density value of a frame. The higher the crowd density, the more concentrated the pedestrians are in the frame; conversely, the lower the density, the more dispersed the pedestrians are. The results of KDE can well reflect the degree of crowd aggregation in the current frame, providing a quantitative density basis for subsequent abnormal behavior detection.

[0080] Step (3): Density sequence maintenance.

[0081] In order to continuously track and analyze the changes in crowd density, the present invention sets a density sequence D = [d1, d2, ..., d N ], which is used to store the crowd density values ​​of the most recent N frames. Whenever a new frame is detected, the system updates the density sequence, queues the density value of the latest frame, and dequeues the density value of the oldest frame, keeping the queue length constant at N.

[0082] This density sequence is used to store crowd density information over a period of time, so that the system can judge abnormal crowd behavior through the continuous change trend in the density sequence. Through this design, the system can use historical information over a period of time instead of relying solely on the data of the current frame, making the detection process more robust and reducing misjudgments caused by single-frame noise or short-term fluctuations.

[0083] Step (4): Smoothing of density data.

[0084] In the present invention, the crowd density is calculated by kernel density estimation, but due to the noise of the video frame, detection error and other reasons, the density value between frames may fluctuate greatly. If these density values ​​are directly used to judge abnormal behavior, false alarms may result. Therefore, the present invention uses a Hanning Window Filter to smooth the density value, reduce the interference of noise, and make the density change more consistent with the actual crowd behavior trend.

[0085] Specifically, the present invention uses a Hanning window filter to smooth the density values ​​in the density sequence. The Hanning window is a weighted average filtering method that can effectively reduce the high-frequency noise in the data sequence and make the density sequence change more smoothly. Its mathematical expression is:

[0086]

[0087] Among them, w(m) is the value of the window function, M is the length of the window, and m is the index within the window. The characteristic of the Hanning window is that it gradually decreases to zero at the boundary and approaches 1 in the middle part, so it gives more weight to the values ​​in the center of the sequence and less weight to the values ​​at the edge of the sequence. This makes it possible to smooth the sequence and reduce edge effects and high-frequency noise.

[0088] In the present invention, the density sequence D = [d1, d2, ..., d N ] contains the crowd density values ​​of the most recent N frames. The density sequence is smoothed using the Hanning window. The specific process is as follows:

[0089] Step (4-1): Density data expansion: Add a part of data at the beginning and end of the density sequence (mirror extension) to prevent abnormal fluctuations or jumps in the data at the beginning and end of the sequence, so that the data value deviates from the normal range and affects the accuracy of the model.

[0090] Step (4-2): Hanning window weighting: Use a Hanning window with a length of window_len to perform weighted smoothing on the expanded density sequence. The weighting process is completed using a convolution operation.

[0091] Step (4-3): Remove padding data: In the smoothed sequence, the first and last The values ​​are added due to expansion, so these data need to be removed from the final result, leaving only the data of the original sequence length.

[0092] Through this process, the smoothed density sequence Q = [q1, q2, ..., q N ], so that the real changing trend of crowd density can be better captured instead of being disturbed by instantaneous abnormal fluctuations.

[0093] Step (5): Sliding window density comparison

[0094] Since the crowd density changes between frames are subtle, in order to more accurately analyze the changes in crowd density, a sliding window method is used to compare the density. The specific process is as follows:

[0095] Step (5-1): Calculate the crowd density by kernel density estimation method, and use Hanning window filter to smooth the density sequence to obtain the preprocessed density sequence Q = [q1, q2, ..., q N ].

[0096] Step (5-2): Set a sliding window of size w and set the step size to s, and calculate the density value in each window. For each window, calculate the average density of all frames in the window as the density value of the window:

[0097]

[0098] Among them, Window i Indicates the window density value, which represents the overall density of the crowd over a period of time.

[0099] Step (5-3): For two consecutive sliding windows, calculate the relative change of window density. If the relative change is greater than the set threshold θ, that is:

[0100] |Window i+1 -Window i |>θ

[0101] It is considered that a significant change in crowd behavior has occurred during this period. If the density change rate calculation result is a positive value, it means that the average density of the current window is higher than that of the previous window, that is, the crowd has become denser, which may represent a gathering of people; if the density change rate calculation result is a negative value, it means that the average density of the current window is lower than that of the previous window, that is, the crowd has become sparse, which may represent a scattered crowd.

[0102] In order to avoid misjudgment caused by noise in a single window, a voting method is introduced, that is, in a time period (in a frame density change queue with a queue length of N), the density changes between multiple windows are detected. If the density changes of multiple windows exceed the threshold θ in this time period, it is considered that abnormal behavior has occurred in this time period. The voting method judgment formula is:

[0103]

[0104] in, is an indicator function. If the relative change of the front and rear windows exceeds the threshold θ, it takes 1, otherwise it takes 0. k is the voting threshold, which means that when the number of times the relative change of the front and rear windows exceeds the threshold θ reaches k, it is considered that abnormal behavior has occurred.

[0105] The sliding window method can process the density sequence in segments and compare the changes in the average density values ​​of two adjacent sliding windows. This method can effectively avoid the impact of single-frame fluctuations on overall judgment and better capture the trend changes of overall crowd behavior. The window size and step size settings will be optimized according to the actual scenario to ensure that it can respond to density changes in a timely manner while maintaining the stability of detection.

[0106] Step (6): Push notification of abnormal event alarm.

[0107] Once the system detects that the density change exceeds the threshold and determines that abnormal behavior has occurred (such as rapid gathering or dispersion of people), the system will automatically trigger the alarm mechanism. Figure 4 and Figure 5 As shown in the figure, the abnormal alarm process of the system includes the following steps:

[0108] Step (6-1): Event detection and initialization: When the system detects abnormal behavior, it assigns a unique event ID eventID and starts the frame counter frame. Whenever the number of frames of an abnormal event exceeds the set threshold ∈, the system triggers an alarm and sets the alarm start counter start to 1 and frame to 0. This cycle repeats until the alarm ends. At the same time, the system starts using the video writer video_writer to save the frame data of the abnormal event in real time, and sets the event live flag live to 1, indicating that an abnormal event is occurring.

[0109] Step (6-2): Video storage and alarm control: During the event, the system continuously updates the frame data and saves the video clips in real time. If a new abnormal event occurs, the system will update the event identifier and continue to save new video clips.

[0110] Step (6-3): Alarm end monitoring thread: The system starts an alarm control thread in parallel, checking the status of all events every 2 seconds. If the eventID of an event is not empty and the live flag is 0, the system will determine whether the event has triggered an alarm:

[0111] If the alarm is not triggered (start=0), the event is considered to be a false alarm and the system will reset all information of the event.

[0112] If the alarm is triggered (start>0), the system stops saving the video, pushes the alarm video, and resets all information about the event.

[0113] Step (6-4): Event end processing: Through the above process, after confirming that the event is over, the system restores the status of all events to the initial state to prepare for detecting the next abnormal event.

[0114] The real-time anomaly detection and alarm mechanism designed by the present invention can effectively improve the safety management capabilities of public places. Through early warning, the potential safety hazards caused by crowd gathering or emergencies can be dealt with in a timely manner, reducing possible risks.

[0115] Embodiment 2

[0116] This embodiment provides a system for detecting abnormal crowd behavior based on kernel density estimation.

[0117] like Figure 6 As shown, a crowd abnormal behavior detection system based on kernel density estimation includes:

[0118] A pedestrian position data acquisition module is configured to: detect all pedestrians in each frame of the acquired video stream to obtain a pedestrian position data list;

[0119] The crowd density value calculation module is configured to: perform kernel density estimation on the pedestrian position data list obtained in each frame to obtain the density value of each frame;

[0120] A density sequence maintenance module is configured to: construct a density sequence based on crowd density values ​​of the most recent N frames;

[0121] A smoothing processing module, which is configured to: use a Hanning window filter to smooth the density sequence;

[0122] The sliding window density comparison module is configured as follows: setting the sliding window size and step size, sliding the sliding window on the smoothed density sequence, calculating the density value of each window to obtain the relative change between two adjacent windows; if the relative change between at least two consecutive adjacent windows is greater than the set threshold, abnormal crowd behavior occurs.

[0123] In some embodiments, the pedestrian position data acquisition module is specifically configured to: scale each acquired frame of the video stream to a fixed size, normalize it, and input the processed image into the improved YOLOv8 model to obtain the detection frame of each pedestrian; determine the position information of each pedestrian according to the center point coordinates of the detection frame of each pedestrian; record the position information of all pedestrians in each frame to obtain a pedestrian position data list; wherein the improved YOLOv8 model introduces a multi-axis attention module into the neck network of the original YOLOv8 model, which is used to perform strip pooling operations on the feature map output by the C2f module in both horizontal and vertical directions, and then fuse it with the feature map output by the C2f module.

[0124] In some embodiments, the crowd density value calculation module is specifically configured as follows: using kernel density estimation to generate a density function for the center point position of each pedestrian in the pedestrian position data list obtained in each frame; based on the density function, smoothing the positions of all pedestrians in space to obtain the density value of each frame.

[0125] In some embodiments, the smoothing processing module is specifically configured to: perform data expansion on the density sequence; use a Hanning window of a set length to perform weighted smoothing on the expanded density sequence; remove the front and back data in the density sequence after the smoothing process, and only retain the data of the original density sequence length before the data expansion;

[0126] In some embodiments, the sliding window density comparison module is specifically configured to: set the sliding window size and step size, slide the sliding window on the smoothed density sequence, calculate the average density of the frames in each window, and obtain the density value of each window; obtain the relative change between the two adjacent windows according to the difference between the density values ​​of the two adjacent windows;

[0127] The voting method is introduced, that is, the relative changes between multiple windows are detected within a time period. If the relative changes between at least two windows are greater than the set threshold, it is considered that abnormal crowd behavior has occurred.

[0128] In some embodiments, the system also includes: an abnormal event alarm module, which is configured to: after abnormal crowd behavior occurs, assign a unique event identifier to each abnormal event and start a frame counter; if the number of frames of the abnormal event exceeds a set threshold, trigger an alarm and set the abnormal event survival identifier to 1; start an alarm control thread to check the status of all abnormal events at preset time intervals, if the event identifier of each abnormal event is not empty and the abnormal event survival identifier is 0, determine whether the abnormal event has triggered an alarm; if the alarm is not triggered, it is considered that the abnormal event is a false alarm and all information about the abnormal event will be reset; if the alarm has been triggered, stop saving the video, push the alarm video, and reset all information about the abnormal event.

[0129] The present invention uses target detection and kernel density estimation, combined with sliding window analysis of density sequences, to automatically detect abnormal gathering or dispersion of crowds and trigger alarms in a timely manner, effectively improving the real-time and accuracy of dynamic management of crowds in public places.

[0130] Embodiment 3

[0131] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the method for detecting abnormal crowd behavior based on kernel density estimation as described in the first embodiment above are implemented.

[0132] Embodiment 4

[0133] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the method for detecting abnormal crowd behavior based on kernel density estimation as described in the first embodiment are implemented.

[0134] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0135] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0136] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0137] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0138] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0139] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for detecting abnormal crowd behavior based on kernel density estimation, characterized in that: include: Detect all pedestrians in each frame of the acquired video stream and obtain a list of pedestrian position data; Perform kernel density estimation on the pedestrian position data list obtained in each frame to obtain the density value of each frame; Based on the crowd density values ​​of the most recent N frames, a density sequence is constructed; Using a Hanning window filter to smooth the density sequence; Set the sliding window size and step size, slide the sliding window on the smoothed density sequence, calculate the density value of each window to obtain the relative change between two adjacent windows; if the relative change between at least two consecutive adjacent windows is greater than the set threshold, abnormal crowd behavior occurs.

2. The method for detecting abnormal crowd behavior based on kernel density estimation according to claim 1, characterized in that: The method of detecting all pedestrians in each frame of the acquired video stream to obtain a pedestrian position data list comprises: Each frame of the acquired video stream is scaled to a fixed size and normalized. The processed image is input into the improved YOLOv8 model to obtain the detection frame of each pedestrian. Determine the location information of each pedestrian based on the coordinates of the center point of the detection frame of each pedestrian; Record the location information of all pedestrians in each frame and obtain a list of pedestrian location data; Among them, the improved YOLOv8 model introduces a multi-axis attention module in the neck network of the original YOLOv8 model, which is used to perform strip pooling operations in both horizontal and vertical directions on the feature map output by the C2f module, and then fuse it with the feature map output by the C2f module.

3. The method for detecting abnormal crowd behavior based on kernel density estimation according to claim 1, characterized in that: The method comprises: using kernel density estimation to generate a density function for the center point position of each pedestrian in the pedestrian position data list obtained for each frame; and based on the density function, smoothing the positions of all pedestrians in space to obtain the density value of each frame.

4. The method for detecting abnormal crowd behavior based on kernel density estimation according to claim 1, characterized in that: The method of using a Hanning window filter to smooth the density sequence comprises: Performing data expansion on the density sequence; The expanded density sequence is weighted and smoothed using a Hanning window of set length; Remove the first and last data in the density sequence after smoothing, and only keep the data of the original density sequence length before data expansion.

5. The method for detecting abnormal crowd behavior based on kernel density estimation according to claim 1, characterized in that: The method of setting the sliding window size and step length, sliding the sliding window on the smoothed density sequence, and calculating the density value of each window to obtain the relative change between two adjacent windows includes: Set the sliding window size and step size, slide the sliding window on the smoothed density sequence, calculate the average density of the frames in each window, and obtain the density value of each window; According to the difference between the density values ​​of two adjacent windows, the relative change between the two adjacent windows is obtained.

6. The method for detecting abnormal crowd behavior based on kernel density estimation according to claim 1 or 5, characterized in that: If the relative change between at least two consecutive adjacent windows is greater than a set threshold, abnormal crowd behavior occurs; the method includes: introducing a voting method, that is, within a time period, detecting the relative change between multiple windows, if the relative change between at least two windows is greater than the set threshold, it is considered that abnormal crowd behavior occurs.

7. The method for detecting abnormal crowd behavior based on kernel density estimation according to claim 1, characterized in that: After abnormal crowd behavior occurs, a unique event identifier is assigned to each abnormal event and a frame counter is started; If the number of frames of an abnormal event exceeds the set threshold, an alarm is triggered and the abnormal event survival flag is set to 1; Start an alarm control thread, check the status of all abnormal events at preset intervals, and if the event flag of each abnormal event is not empty and the abnormal event survival flag is 0, determine whether the abnormal event has triggered an alarm; If the alarm is not triggered, the abnormal event is considered to be a false alarm and all information about the abnormal event will be reset; if the alarm is triggered, stop saving the video, push the alarm video, and reset all information about the abnormal event.

8. A crowd abnormal behavior detection system based on kernel density estimation, characterized in that: include: A pedestrian position data acquisition module is configured to: detect all pedestrians in each frame of the acquired video stream to obtain a pedestrian position data list; The crowd density value calculation module is configured to: perform kernel density estimation on the pedestrian position data list obtained in each frame to obtain the density value of each frame; A density sequence maintenance module is configured to: construct a density sequence based on crowd density values ​​of the most recent N frames; A smoothing processing module, which is configured to: use a Hanning window filter to smooth the density sequence; The sliding window density comparison module is configured as follows: setting the sliding window size and step size, sliding the sliding window on the smoothed density sequence, calculating the density value of each window to obtain the relative change between two adjacent windows; if the relative change between at least two consecutive adjacent windows is greater than the set threshold, abnormal crowd behavior occurs.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the method for detecting abnormal crowd behavior based on kernel density estimation as described in any one of claims 1 to 7 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the method for detecting abnormal crowd behavior based on kernel density estimation as described in any one of claims 1 to 7 are implemented.

Images