A campus safety analysis method based on multi-point personnel clustering

Abstract

The application provides a campus safety analysis method based on multi-point personnel clustering, and relates to the technical field of image processing, comprising: acquiring a monitoring video stream of a public area of a campus, and performing image data screening on the monitoring video stream to obtain corresponding video monitoring screening images; based on an image data enhancement technology, performing image data enhancement on the video monitoring screening images to obtain corresponding video monitoring enhancement images; performing feature extraction on the video monitoring enhancement images to obtain corresponding multi-point personnel clustering features; and analyzing whether there is a campus safety risk according to the multi-point personnel clustering features.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically a campus security analysis method based on multi-point personnel clustering. Background Technology

[0002] Campus safety is a core issue of great concern in the education sector. Campus public areas have frequent personnel flow and complex scenarios, and crowds often gather. Unreasonable crowds may often pose hidden safety hazards. Timely and accurate identification of crowd status and assessment of safety risks are of great significance to campus security work.

[0003] Currently, campus security monitoring relies heavily on manual duty or simple image recognition technology. Manual duty is inefficient, making it difficult to achieve real-time monitoring with full coverage and all-time hours. It is also susceptible to human negligence, which makes it impossible to effectively provide early warning and scientific assessment of campus security risks and meet the needs of refined and intelligent campus security prevention and control.

[0004] Therefore, a campus security analysis method based on multi-point personnel clustering is proposed. Summary of the Invention

[0005] To address the aforementioned technical problems, the present invention aims to provide a campus security analysis method based on multi-point personnel clustering.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a campus security analysis method based on multi-point personnel clustering, the method comprising: Acquire surveillance video streams from public areas on campus and perform image data filtering on the surveillance video streams, including key frame calibration of the surveillance video streams and calculation of pixel difference change rate, to obtain the first frame and auxiliary frames, and then obtain the corresponding video surveillance filtered images. Based on image data enhancement technology, the video surveillance filtered image is enhanced to obtain the global brightness grayscale image and single-channel grayscale image of the corresponding frame and extract local texture hash features, thereby obtaining the corresponding enhanced video surveillance image. Based on the YOLOv5n lightweight deep learning algorithm and Euclidean distance, feature extraction is performed on the video surveillance enhancement image and density clustering is used to obtain the corresponding multi-point personnel cluster features; wherein, the multi-point personnel cluster features include cluster line-of-sight features, cluster number, and cluster dispersion features; Based on the characteristics of the multi-point personnel clusters, analyze whether there are any campus safety risks.

[0007] Furthermore, the video surveillance filtered image refers to a set of several frames of video surveillance filtered images after image data filtering, including the video surveillance filtered images corresponding to the first frame and auxiliary frames. The corresponding several frames are based on the calibration key frame as the first frame and the remaining frames as auxiliary frames.

[0008] Furthermore, the process of filtering image data in the surveillance video stream includes: The monitoring video stream is calibrated with key frames. Based on the actual traffic demand in different public areas of the campus, key frame calibration rules are set. The pixel difference between adjacent monitoring video stream frames is calculated based on the inter-frame difference method. When the pixel difference is greater than the preset pixel difference threshold, the frame is determined to be a candidate key frame. Based on the pixel difference change rate between adjacent candidate keyframes, the candidate keyframes are initially screened. If the pixel difference change rate between adjacent candidate keyframes is greater than the preset pixel difference change rate threshold, the latter candidate keyframe of the two adjacent candidate keyframes is marked as the formal keyframe and used as the first frame of the video surveillance image screening group. According to a preset time interval, several frames are extracted from the video stream after the first frame as candidate auxiliary frames; image sharpness detection is performed on each candidate auxiliary frame, and the sharpness index of the candidate auxiliary frame is calculated based on the gray-scale variance method. If the sharpness index of the candidate auxiliary frame meets the preset value, it is marked as an official auxiliary frame and used as an auxiliary frame for screening video surveillance image groups. Based on the first frame and the supplementary frames, the corresponding video surveillance filter images are obtained.

[0009] Furthermore, the process of performing image data enhancement on the selected video surveillance images includes: The global brightness grayscale image and single-channel grayscale image of each frame of video surveillance filtering image are obtained. Local texture hash features are extracted from a preset set of non-continuous reference frames to construct a texture hash library. The global brightness grayscale image is divided into grid blocks, and block-level hash fingerprints are generated for each block. Fuzzy matching is performed according to the texture hash library to classify the region types, including stable background regions and changing foreground regions. Construct a multi-scale Laplacian pyramid for a single-channel grayscale image to generate the corresponding gradient response map; The dynamic fusion weights are calculated based on the region type, gradient strength, and signal-to-noise ratio; the gradient response maps are then weighted and fused according to the dynamic fusion weights, and the three channels are merged to obtain the corresponding video surveillance enhancement image.

[0010] Furthermore, the process of feature extraction from the enhanced video surveillance image includes: Based on the YOLOv5n lightweight deep learning algorithm, people are detected in each frame of the video surveillance enhancement image, and the set of effective people individuals in the corresponding video surveillance enhancement image is output. Density clustering is performed based on the Euclidean distance of the set of effective people individuals to obtain a set of people clusters, wherein each people cluster has no less than one person individual. The number of clusters is then counted. Iterate through all non-continuous sampling frames within the time window, and calculate the gaze concentration and spatial dispersion of each personnel cluster frame by frame; set a concentration threshold and a dispersion threshold; based on the gaze concentration, spatial dispersion, concentration threshold, and dispersion threshold, obtain the corresponding cluster gaze characteristics and cluster dispersion characteristics; wherein the cluster gaze characteristics include the average duration of cluster gaze concentration and the overall direction of cluster gaze concentration; the cluster dispersion characteristics include the average duration of cluster dispersion and the cluster dispersion rate.

[0011] Furthermore, the process of acquiring clustered line-of-sight features includes: Based on the gaze concentration degree and the concentration degree threshold, each frame's personnel cluster is marked as either clustered or non-clustered; the cumulative duration of the clustered state is calculated, and the average duration of gaze concentration within the cluster is obtained based on the total duration of the personnel cluster; the mean vector of the gaze vectors of all individuals within the corresponding personnel cluster is extracted; the mean vector is mapped to a comprehensive pointing space with 8 horizontal directions and 3 vertical levels, thereby outputting the comprehensive pointing of the clustered gaze concentration.

[0012] Furthermore, the process of obtaining cluster dispersion characteristics includes: Based on the spatial dispersion and dispersion threshold, each frame's personnel cluster is marked as either dispersed or non-dispersed; based on the cumulative duration of the dispersed state and the total duration of the personnel cluster, the average duration of cluster dispersion is obtained; adjacent frames are selected, the change in the dispersion of personnel clusters between the two frames is calculated, and the cluster dispersion rate is obtained based on the time interval between the two frames.

[0013] Furthermore, based on the aforementioned multi-point personnel cluster characteristics, the process of analyzing whether there are campus safety risks includes: A pre-built rule base for determining campus safety risks has been established, including: If the number of clusters is greater than or equal to a preset cluster number threshold, it is marked as a general clustering risk; otherwise, there is no clustering risk. If the general risk of gathering is met, and the average duration of the visual focus of the corresponding personnel cluster is greater than or equal to the preset visual focus duration threshold, or the visual focus of the corresponding personnel cluster is pointing in the same direction, then it is marked as a high risk of gathering, and an instruction is issued to the relevant security department. If the general clustering risk is met, and the average clustering and dispersal time of the corresponding personnel cluster is less than or equal to the preset clustering and dispersal time threshold or the clustering and dispersal rate is greater than or equal to the preset clustering and dispersal rate threshold, then the general clustering risk is corrected to no clustering risk. If none of the above risk conditions are met, the condition is determined to be safe.

[0014] Compared with existing technologies, the beneficial effects of this invention are as follows: By acquiring surveillance video streams from public areas of the campus and filtering the image data of the surveillance video streams, corresponding filtered video surveillance images are obtained; based on image data enhancement technology, the filtered video surveillance images are enhanced to obtain corresponding enhanced video surveillance images; feature extraction is performed on the enhanced video surveillance images to obtain corresponding multi-point personnel cluster features; and based on the multi-point personnel cluster features, the existence of campus security risks is analyzed. By extracting multi-point personnel cluster features, comprehensive information such as the number of clusters, line of sight, and dispersion is captured, breaking the limitations of single feature analysis; combined with a preset risk classification and judgment rule library, the security risks corresponding to different cluster states can be accurately classified and flexibly corrected, enabling timely identification and reasonable handling of security risks, effectively reducing campus security hazards, providing scientific and efficient technical support for campus security work, and ensuring the safety and order stability of personnel in public areas of the campus. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0016] Figure 1 This is a schematic diagram illustrating the steps of a campus security analysis method based on multi-point personnel clustering.

[0017] Figure 2 Flowchart for selecting keyframes in a video.

[0018] Figure 3 This is a flowchart for image enhancement and people cluster feature extraction.

[0019] Figure 4 Flowchart for assessing safety risks associated with clustered people. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be described in detail below. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other implementation methods obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0022] like Figure 1 As shown, a campus security analysis method based on multi-point personnel clustering is characterized by the following: Step S1: Obtain the surveillance video stream of the public area of ​​the campus, and perform image data filtering on the surveillance video stream, including key frame calibration of the surveillance video stream and calculation of pixel difference change rate, to obtain the first frame and auxiliary frames, and then obtain the corresponding video surveillance filtered image. Step S2: Based on image data enhancement technology, perform image data enhancement on the selected video surveillance images, obtain the global brightness grayscale image and single-channel grayscale image of the corresponding frame, and extract local texture hash features to obtain the corresponding enhanced video surveillance image; Step S3: Based on the YOLOv5n lightweight deep learning algorithm and Euclidean distance, feature extraction is performed on the video surveillance enhancement image and density clustering is used to obtain the corresponding multi-point personnel cluster features; wherein, the multi-point personnel cluster features include cluster line-of-sight features, cluster number, and cluster dispersion features; Step S4: Based on the characteristics of the multi-point personnel clusters, analyze whether there are any campus safety risks.

[0023] Optionally, in this embodiment of the application, the campus public areas include densely populated areas such as the teaching building lobby, the canteen entrance and exit, the playground stands, the library entrance, the main road intersection, the dormitory area, and the gymnasium square; each area is equipped with high-definition network cameras with a uniform resolution of 1920×1080 and 30fps, and the video stream is pushed to the campus security backend in real time. The backend receives and caches complete video frames every second to form a continuous monitoring video stream.

[0024] In one exemplary embodiment, the cafeteria entrance camera, numbered C-003, monitors a video stream with a resolution of 1920×1080 and a frame rate of 30fps. Each frame is approximately 1.2MB in size. The background caches the video frame queue in 1-second increments for a continuous caching time of 30 minutes, providing a complete data source for subsequent image filtering.

[0025] It should be noted that the video surveillance filtered image refers to a set of several frames of video surveillance filtered images after image data is filtered, including the video surveillance filtered images corresponding to the first frame and the auxiliary frames. Among them, the corresponding several frames are based on the calibration key frame as the first frame and the remaining frames as auxiliary frames.

[0026] like Figure 2 As shown in the embodiments of this application, the specific process of filtering image data in the surveillance video stream includes: The monitoring video stream is calibrated with key frames. Based on the actual traffic demand in different public areas of the campus, key frame calibration rules are set. The pixel difference between adjacent monitoring video stream frames is calculated based on the inter-frame difference method. When the pixel difference is greater than the preset pixel difference threshold, the frame is determined to be a candidate key frame. It should be noted that the keyframe calibration rule is that the higher the traffic volume in an area, the higher the keyframe calibration density. For example, during peak hours (7:00-8:30, 11:30-13:00, 17:00-18:30) in the cafeteria entrance and teaching building lobby, if the traffic volume is ≥15 people / minute, the keyframe calibration interval is set to 2 frames (i.e., 0.08 seconds). For another example, during off-peak hours in the library, if the traffic volume is ≤5 people / minute, the keyframe calibration interval is set to 10 frames (0.4 seconds).

[0027] The inter-frame difference method uses grayscale difference, and the pixel difference is the sum of the grayscale differences of corresponding pixels in two frames. The preset pixel difference threshold is uniformly set to 12000 (i.e., 8-bit grayscale image, 1080P resolution).

[0028] In one exemplary embodiment, during peak hours at the cafeteria entrance and exit, if the grayscale difference pixel difference between two consecutive frames (e.g., frame numbers F120 and F121) is 18500, which is greater than the threshold of 12000, then frame F121 is determined to be a candidate keyframe. During off-peak hours at the library, if the grayscale difference pixel difference between two consecutive frames (e.g., frame numbers F450 and F451) is 3200, which is less than the threshold of 12000, then it is not determined to be a candidate keyframe, and the process continues to traverse subsequent frames.

[0029] Optionally, in this embodiment of the application, candidate keyframes are initially screened based on the rate of change of pixel difference between adjacent candidate keyframes. If the rate of change of pixel difference between adjacent candidate keyframes is greater than a preset threshold for the rate of change of pixel difference, then the latter candidate keyframe of the two adjacent candidate keyframes is marked as a formal keyframe and used as the first frame of the video surveillance image screening group.

[0030] It should be noted that the pixel difference change rate = (pixel difference of the next candidate keyframe - pixel difference of the previous candidate keyframe) / frame interval time, and the preset pixel difference change rate threshold is set to 5000 / second; furthermore, the pixel difference change rate threshold is adapted to the actual campus scene, which filters out pseudo keyframes caused by slight shaking and changes in light and shadow, while retaining the real keyframes corresponding to dynamic events such as people gathering, running, and chasing.

[0031] In an exemplary embodiment, two consecutive candidate keyframes are generated at the cafeteria entrance and exit: frame F121 with a pixel difference of 18500 and frame F125 with a pixel difference of 42800, with a frame interval of 0.16 seconds. The corresponding pixel difference change rate is calculated as (42800-18500) / 0.16 = 151875 / second, which is much greater than the threshold of 5000 / second. Therefore, frame F125 is designated as the official keyframe (i.e., the first frame). Two consecutive candidate keyframes are generated at the playground entrance: frame F200 with a pixel difference of 13200 and frame F204 with a pixel difference of 1580. 0. The frame interval is 0.16 seconds. The corresponding pixel difference change rate is calculated as (15800-13200) / 0.16=16250 / second, which is also greater than the threshold. Frame F204 is designated as a formal keyframe. There are two consecutive candidate keyframes at the library entrance: frame F500 with a pixel difference of 12800 and frame F505 with a pixel difference of 14100. The frame interval is 0.2 seconds. The corresponding pixel difference change rate is calculated as (14100-14000) / 0.2=500 / second, which is less than the threshold. They are not designated as formal keyframes for the time being.

[0032] Optionally, in this embodiment of the application, several frames are extracted from the video stream after the first frame as candidate auxiliary frames according to a preset time interval; image sharpness detection is performed on each candidate auxiliary frame, and the sharpness index of the candidate auxiliary frame is calculated based on the gray-scale variance method. If the sharpness index of the candidate auxiliary frame meets the preset value, it is marked as a formal auxiliary frame and used as an auxiliary frame for screening video surveillance image groups. It should be noted that the preset time interval is uniformly set to 0.5 seconds, and 5 candidate auxiliary frames are extracted from the first frame. The sharpness index of the candidate auxiliary frames is calculated based on the gray-scale variance method, which is the image gray-scale value variance. The preset sharpness threshold is set to 80. The image gray-scale value variance is determined to be sharp if it is greater than or equal to 80, and less than 80 if it is blurry.

[0033] In an exemplary embodiment, for example, after the first frame F125 at the cafeteria entrance / exit, candidate auxiliary frames F137, F149, F161, F173, and F185 are extracted. Among them, the image with a grayscale variance of 125 corresponding to F137 is determined to be clear and is marked as a formal auxiliary frame; the image with a grayscale variance of 98 corresponding to F149 is determined to be clear and is marked as a formal auxiliary frame; the image with a grayscale variance of 42 corresponding to F161 is determined to be blurry and is discarded; the image with a grayscale variance of 112 corresponding to F173 is determined to be clear and is marked as a formal auxiliary frame; the image with a grayscale variance of 75 corresponding to F185 is determined to be blurry and is discarded. Finally, the first frame corresponds to 3 formal auxiliary frames, forming a video surveillance screening image group (i.e., 1 first frame and 3 auxiliary frames).

[0034] like Figure 3 As shown in this embodiment, the specific process of image data enhancement for the video surveillance filtered images includes: Optionally, in this embodiment of the application, the global brightness grayscale image and the three single-channel grayscale images of R, G, and B of each frame of video surveillance screening image are obtained, and local texture hash features are extracted from a preset set of non-continuous reference frames to construct a texture hash library; It should be noted that the global brightness grayscale image is generated using weighted grayscale. The three single-channel grayscale images are the red channel grayscale image, the green channel grayscale image, and the blue channel grayscale image, respectively. The grayscale value of each channel is directly taken from the corresponding channel pixel value (0-255). The non-continuous reference frame set consists of key frames from each camera's history over the past 7 days, with one frame selected every hour. The texture hash feature uses 64-bit perceptual hashing, which is generated by dividing the image into blocks (8×8), performing discrete cosine transform, and extracting high-frequency components. The texture hash library is stored by camera number and date and is updated periodically.

[0035] In an exemplary embodiment, the cafeteria entrance and exit image F125 is filtered as follows: the average brightness of the global brightness grayscale image is 142 (0-255), corresponding to a monitoring scene with mixed natural light and indoor lighting during the day; the average brightness of the red channel grayscale image is 135, which is warm in tone and suitable for detecting skin color areas on faces; the average brightness of the green channel grayscale image is 158, which has the highest detail contrast and is suitable for extracting personnel outlines and green plant backgrounds; the average brightness of the blue channel grayscale image is 122, which is cool in tone and suitable for distinguishing sky and glass reflection areas; 1200 local texture hashes are extracted from the reference frame set (7 days of historical frames from camera C-003) to construct a texture hash library specific to this camera, with a hash matching accuracy ≥95%.

[0036] Optionally, in this embodiment of the application, the global brightness grayscale image is divided into grid blocks, block-level hash fingerprints are generated block by block, and fuzzy matching is performed according to the texture hash library to divide the region types, including stable background regions and changing foreground regions. It should be noted that the grid block size is set to 32×32 pixels; the block-level hash fingerprint is a 32-bit hash value, which is fuzzy matched with the texture hash library, and is judged as a match based on a Hamming distance ≤3; the matching rate of stable background domains is ≥85% (such as walls, ground, fixed facilities), and the matching rate of changing foreground domains is <85% (such as moving people, luggage, vehicles).

[0037] In an exemplary embodiment, the cafeteria entrance and exit image is divided into F125 grid blocks: stable background domain: walls, dining tables, door frames, a total of 1428 blocks, with a matching rate of 91%; variable foreground domain: queuing students, walking people, school bags, a total of 612 blocks, with a matching rate of 42%; the segmentation results accurately distinguish between static background and dynamic personnel areas, avoiding background interference with subsequent feature extraction.

[0038] Optionally, in this embodiment of the application, a multi-scale Laplacian pyramid is constructed for the three single-channel grayscale images of R, G, and B respectively to generate the corresponding gradient response map; It should be noted that each single-channel grayscale image independently constructs a 3-layer Laplacian pyramid (i.e., original image, 1 / 2 scale, and 1 / 4 scale), and the gradient of each layer is calculated using a 5×5 Laplacian operator; the three sets of gradient response maps retain the edge, contour, and texture information of each channel respectively, the red channel gradient response map enhances the facial contour, the green channel enhances the limb contour, and the blue channel weakens the reflection noise; in backlit and shadow scenes on campus, the signal-to-noise ratio of the fused gradient response map is improved by ≥35%.

[0039] In an exemplary embodiment, the image of a backlit scene at the library entrance is filtered as follows: the original image has large areas of shadow and insufficient exposure of faces; the red channel gradient response map clearly restores the facial contours of 12 students gathered together; the green channel gradient response map enhances the edges of limbs and clothing, and the contour contrast is increased from 28 to 76; the blue channel gradient response map filters glass reflection and overexposed sky noise, and the noise is reduced by 48%; the three sets of gradient response maps complement each other and provide multi-dimensional edge information for subsequent fusion.

[0040] The dynamic fusion weights are calculated based on the region type, gradient strength, and signal-to-noise ratio; the gradient response maps are then weighted and fused according to the dynamic fusion weights, and the three channels are merged to obtain the corresponding video surveillance enhancement image.

[0041] It should be noted that the single-channel dynamic fusion weight formula is: Dynamic fusion weight = (gradient strength × 0.6 + signal-to-noise ratio × 0.4) × region coefficient; the stable background domain coefficient is 0.3 (weakening the background), and the changing foreground domain coefficient is 1.0 (enhancing people); after the three sets of gradient response maps are weighted and fused, they are then fused with the global brightness grayscale map at a ratio of 0.7:0.3, and finally the RGB three-channel output enhanced image is merged, the resolution is maintained at 1920×1080, and the brightness, contrast and sharpness are adaptively optimized.

[0042] In one exemplary embodiment, a low-light scene of a playground grandstand at dusk: The original screening image had a brightness of 68, a contrast of 32, and a sharpness of 45; the enhanced image had a brightness of 138, a contrast of 82, and a sharpness of 91. The facial and limb contours of the 20 students gathered together were clearly distinguishable, with no obvious noise, meeting the accuracy requirements for subsequent personnel detection.

[0043] In this embodiment of the application, the specific process of feature extraction for the enhanced video surveillance image includes: Optionally, in this embodiment of the application, based on the YOLOv5n lightweight deep learning algorithm, people detection is performed on each frame of the video surveillance enhancement image, and a set of effective people individuals corresponding to the video surveillance enhancement image is output; density clustering is performed based on the Euclidean distance of the set of effective people individuals to obtain a set of people clusters, wherein each people cluster has no less than 1 person individual; and the number of clusters is counted. It should be noted that the YOLOv5n model has been fine-tuned using a dedicated campus scene dataset (50,000 campus personnel images), achieving a detection accuracy of ≥96% and an inference speed of ≥30 frames / second, making it suitable for real-time analysis; the Euclidean distance threshold is set to 1.2 meters (approximately 45 pixels in image pixels), and the distance between any two people within the same cluster is ≤1.2 meters; the number of clusters is the total number of independent clusters within a single frame.

[0044] In an exemplary embodiment, the enhanced image of the teaching building lobby during break time (10:00-10:20) shows that YOLOv5n detected 28 valid individuals; after density clustering, four clusters were obtained: Cluster A: 8 people (discussing homework, spacing 0.6-1.1 meters); Cluster B: 7 people (waiting for the elevator, spacing 0.5-0.9 meters); Cluster C: 6 people (chatting in the corridor, spacing 0.7-1.2 meters); Cluster D: 7 people (playing around, spacing 0.8-1.1 meters); The total number of clusters is 4, covering all areas where people gather in the lobby.

[0045] Optionally, in this embodiment, all non-continuous sampling frames within the time window are traversed, and the gaze concentration and spatial dispersion of each personnel cluster are calculated frame by frame; a concentration threshold and a dispersion threshold are set; based on the gaze concentration, spatial dispersion, concentration threshold, and dispersion threshold, the corresponding cluster gaze characteristics and cluster dispersion characteristics are obtained respectively; wherein the cluster gaze characteristics include the average duration of cluster gaze concentration and the comprehensive direction of cluster gaze concentration; the cluster dispersion characteristics include the average duration of cluster dispersion and the cluster dispersion rate.

[0046] It should be noted that the time window is set to 10 seconds, the non-continuous sampling frame interval is 2 seconds, and there are a total of 5 frames; the line-of-sight clustering degree is the percentage of people in the cluster whose line of sight is pointing in the same direction, and the clustering degree threshold is set to 70%; the spatial dispersion degree is the maximum distance between people in the cluster / the minimum distance between people in the cluster, and the dispersion degree threshold is set to 2.5; after calculating frame by frame, the cluster line-of-sight characteristics and dispersion characteristics are summarized.

[0047] In an exemplary embodiment, a scene of a dispute among people in a corner of the playground (14:30-14:40) is analyzed. Within a 10-second time window, there are 5 frames: a cluster of 6 people, with frame-by-frame gaze concentration of 85%, 92%, 88%, 90%, and 87%, all ≥70% threshold; and frame-by-frame spatial dispersion of 1.2, 1.5, 1.8, 2.1, and 2.3, all <2.5 threshold. Cluster gaze characteristics and dispersion characteristics are extracted for subsequent risk analysis.

[0048] It should be further explained that, in the specific implementation process, the process of obtaining clustered line-of-sight features includes: Optionally, in this embodiment, based on the gaze concentration degree and the concentration degree threshold, each frame's personnel cluster is marked as either clustered or non-clustered; the cumulative duration of the clustered state is counted, and the average duration of the clustered gaze concentration is obtained based on the total existence duration of the personnel cluster; the mean vector of the gaze vectors of all individuals within the corresponding personnel cluster is extracted; the mean vector is mapped to a comprehensive pointing space with 8 horizontal directions and 3 vertical levels, thereby outputting the comprehensive pointing of the clustered gaze concentration.

[0049] It should be noted that there are 8 horizontal directions: east, southeast, south, southwest, west, northwest, north, and northeast; and 3 vertical levels: up (looking up), middle (looking straight ahead), and down (looking down). The gaze vector is calculated from the coordinates of key head points (eyes and nose tip), and the mean vector is the average of the gaze vectors of all individuals within the cluster. The average duration of gaze aggregation in the cluster = cumulative duration of aggregation / total duration of existence, in seconds.

[0050] In an exemplary embodiment, the 12 people gathered at the library entrance constitute 6 groups of people, with 5 of these groups existing for a total of 10 seconds: all 5 frames are in a clustered state, with a cumulative duration of 10 seconds, and the average duration of the gaze convergence of the groups is 10 seconds; the average direction of the individual gaze vectors is: east-southeast in the horizontal 8 directions and eye-level in the vertical 3 levels; after mapping, the overall direction of the gaze convergence of the groups is east-southeast and eye-level, meaning that the 5 groups of people are focusing on the same group of people.

[0051] It should be further explained that, in the specific implementation process, the process of obtaining cluster dispersion characteristics includes: Optionally, in this embodiment of the application, the personnel clusters in each frame are marked as dispersed or non-dispersed based on the spatial dispersion and dispersion threshold; the average dispersion duration of the clusters is obtained based on the cumulative duration of the dispersed state and the total existence duration of the personnel clusters; adjacent frames are selected, the change in the dispersion of the personnel clusters between the two frames is calculated, and the cluster dispersion rate is obtained based on the time interval between the two frames.

[0052] It should be noted that the dispersion state is marked as follows: dispersion ≥ 2.5 is a dispersion state, and < 2.5 is a non-dispersion state; the average duration of cluster dispersion = cumulative duration of dispersion state / total duration of existence; the cluster dispersion rate = (dispersion of the next frame - dispersion of the previous frame) / frame interval time.

[0053] In an exemplary embodiment, a group of 9 people gathers at a main road intersection (briefly gathering during a break between classes and then dispersing), with a total duration of 10 seconds: the dispersion in the first 3 frames is 1.5, 1.8, and 2.2 (non-dispersed state), and the dispersion in the last 2 frames is 2.8 and 3.2 (dispersed state); the cumulative duration of the dispersed state is 4 seconds, and the average duration of the cluster dispersal is 4 seconds; the change in dispersion between adjacent frames is 2.8-2.2=0.6 and 3.2-2.8=0.4, with a frame interval of 2 seconds; the cluster dispersal rate is (0.6+0.4) / 2=0.5 / second, indicating that the cluster disperses quickly and there is no risk of continued gathering.

[0054] like Figure 4 As shown, in the specific implementation process, the process of analyzing whether there is a campus safety risk based on the multi-point personnel cluster characteristics includes: A pre-built rule base for determining campus safety risks has been established, including: If the number of clusters is greater than or equal to a preset cluster number threshold, it is marked as a general clustering risk; otherwise, there is no clustering risk. If the general risk of gathering is met, and the average duration of the visual focus of the corresponding personnel cluster is greater than or equal to the preset visual focus duration threshold, or the visual focus of the corresponding personnel cluster is pointing in the same direction, then it is marked as a high risk of gathering, and an instruction is issued to the relevant security department. If the general clustering risk is met, and the average clustering and dispersal time of the corresponding personnel cluster is less than or equal to the preset clustering and dispersal time threshold or the clustering and dispersal rate is greater than or equal to the preset clustering and dispersal rate threshold, then the general clustering risk is corrected to no clustering risk. If none of the above risk conditions are met, the condition is determined to be safe.

[0055] It should be noted that the preset thresholds are adapted to the campus scene: cluster number threshold: 3; line-of-sight gathering time threshold: 6 seconds; cluster dispersion time threshold: 3 seconds; cluster dispersion rate threshold: 0.4 / second.

[0056] In one exemplary embodiment, risk assessment is verified in different scenarios: Scenario 1: Gathering in the lobby of the teaching building during breaks (safety): Number of clusters = 4 (≥3, general gathering risk); Average duration of eye contact between clusters: Cluster A = 4 seconds, Cluster B = 5 seconds, Cluster C = 3 seconds, Cluster D = 4 seconds (all <6 seconds); Average duration of dispersal between clusters: all ≥5 seconds; Conclusion: There is only general clustering risk, no high risk, and no alarm is required.

[0057] Scenario 2: Six people arguing in a corner of the playground (high risk): Number of clusters = 8 (<3, no general cluster risk); Average duration of eye contact among clusters = 9 seconds (≥6 seconds); Overall direction of eye contact among clusters = same direction (focusing on the two parties in the argument); Conclusion: Mark high cluster risk, immediately issue instructions to the campus security department, and security personnel should arrive at the scene within 3 minutes to handle the situation.

[0058] Scenario 3: Brief gathering of 9 people at a main road intersection (no risk): Number of clusters = 1 (≥3 is not valid, general gathering risk is not valid); Average duration of cluster dispersal = 2 seconds (≤3 seconds); Cluster dispersal rate = 0.5 / second (≥0.4 / second); Conclusion: The risk of clustering has been revised to no level; the situation is deemed safe and no intervention is required.

[0059] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0060] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0061] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more electronic devices to execute all or part of the steps of the methods described in the various embodiments of this application.

[0062] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0063] In the several embodiments provided in this application, it should be understood that the disclosed application can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of units or modules may be electrical or other forms.

[0064] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0065] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0066] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A campus security analysis method based on multi-point personnel clustering, characterized in that, The method includes: Acquire surveillance video streams from public areas on campus and perform image data filtering on the surveillance video streams, including key frame calibration of the surveillance video streams and calculation of pixel difference change rate, to obtain the first frame and auxiliary frames, and then obtain the corresponding video surveillance filtered images. Based on image data enhancement technology, the video surveillance filtered image is enhanced to obtain the global brightness grayscale image and single-channel grayscale image of the corresponding frame and extract local texture hash features, thereby obtaining the corresponding enhanced video surveillance image. Based on the YOLOv5n lightweight deep learning algorithm and Euclidean distance, feature extraction is performed on the video surveillance enhancement image and density clustering is used to obtain the corresponding multi-point personnel cluster features; wherein, the multi-point personnel cluster features include cluster line-of-sight features, cluster number, and cluster dispersion features; Based on the characteristics of the multi-point personnel clusters, analyze whether there are any campus safety risks.

2. The campus security analysis method based on multi-point personnel clustering according to claim 1, characterized in that, The video surveillance filtered image refers to a set of several frames of video surveillance filtered images after image data is filtered, including the video surveillance filtered images corresponding to the first frame and the auxiliary frames. The corresponding several frames are based on the calibration key frame as the first frame and the remaining frames as auxiliary frames.

3. The campus security analysis method based on multi-point personnel clustering according to claim 2, characterized in that, The process of filtering image data in the surveillance video stream includes: The monitoring video stream is calibrated with key frames. Based on the actual traffic demand in different public areas of the campus, key frame calibration rules are set. The pixel difference between adjacent monitoring video stream frames is calculated based on the inter-frame difference method. When the pixel difference is greater than the preset pixel difference threshold, the frame is determined to be a candidate key frame. Based on the pixel difference change rate between adjacent candidate keyframes, the candidate keyframes are initially screened. If the pixel difference change rate between adjacent candidate keyframes is greater than the preset pixel difference change rate threshold, the latter candidate keyframe of the two adjacent candidate keyframes is marked as the formal keyframe and used as the first frame of the video surveillance image screening group. According to a preset time interval, several frames are extracted from the video stream after the first frame as candidate auxiliary frames; image sharpness detection is performed on each candidate auxiliary frame, and the sharpness index of the candidate auxiliary frame is calculated based on the gray-scale variance method. If the sharpness index of the candidate auxiliary frame meets the preset value, it is marked as an official auxiliary frame and used as an auxiliary frame for screening video surveillance image groups. Based on the first frame and the supplementary frames, the corresponding video surveillance filter images are obtained.

4. The campus security analysis method based on multi-point personnel clustering according to claim 3, characterized in that, The process of performing image data enhancement on the selected video surveillance images includes: The global brightness grayscale image and single-channel grayscale image of each frame of video surveillance filtering image are obtained. Local texture hash features are extracted from a preset set of non-continuous reference frames to construct a texture hash library. The global brightness grayscale image is divided into grid blocks, and block-level hash fingerprints are generated for each block. Fuzzy matching is performed according to the texture hash library to classify the region types, including stable background regions and changing foreground regions. Construct a multi-scale Laplacian pyramid for a single-channel grayscale image to generate the corresponding gradient response map; The dynamic fusion weights are calculated based on the region type, gradient strength, and signal-to-noise ratio; the gradient response maps are then weighted and fused according to the dynamic fusion weights, and the three channels are merged to obtain the corresponding video surveillance enhancement image.

5. The campus security analysis method based on multi-point personnel clustering according to claim 4, characterized in that, The process of feature extraction from the enhanced video surveillance image includes: Based on the YOLOv5n lightweight deep learning algorithm, people are detected in each frame of the video surveillance enhancement image, and the set of effective people individuals in the corresponding video surveillance enhancement image is output. Density clustering is performed based on the Euclidean distance of the set of effective people individuals to obtain a set of people clusters, wherein each people cluster has no less than one person individual. The number of clusters is then counted. Iterate through all non-continuous sampling frames within the time window, and calculate the gaze concentration and spatial dispersion of each personnel cluster frame by frame; set a concentration threshold and a dispersion threshold; based on the gaze concentration, spatial dispersion, concentration threshold, and dispersion threshold, obtain the corresponding cluster gaze characteristics and cluster dispersion characteristics; wherein the cluster gaze characteristics include the average duration of cluster gaze concentration and the overall direction of cluster gaze concentration; the cluster dispersion characteristics include the average duration of cluster dispersion and the cluster dispersion rate.

6. The campus security analysis method based on multi-point personnel clustering according to claim 5, characterized in that, The process of obtaining clustered line-of-sight features includes: Based on the gaze concentration degree and the concentration degree threshold, each frame's personnel cluster is marked as either clustered or non-clustered; the cumulative duration of the clustered state is calculated, and the average duration of gaze concentration within the cluster is obtained based on the total duration of the personnel cluster; the mean vector of the gaze vectors of all individuals within the corresponding personnel cluster is extracted; the mean vector is mapped to a comprehensive pointing space with 8 horizontal directions and 3 vertical levels, thereby outputting the comprehensive pointing of the clustered gaze concentration.

7. The campus security analysis method based on multi-point personnel clustering according to claim 6, characterized in that, The process of obtaining cluster dispersion characteristics includes: Based on the spatial dispersion and dispersion threshold, each frame's personnel cluster is marked as either dispersed or non-dispersed; based on the cumulative duration of the dispersed state and the total duration of the personnel cluster, the average duration of cluster dispersion is obtained; adjacent frames are selected, the change in the dispersion of personnel clusters between the two frames is calculated, and the cluster dispersion rate is obtained based on the time interval between the two frames.

8. The campus security analysis method based on multi-point personnel clustering according to claim 7, characterized in that, The process of analyzing whether there are campus safety risks based on the aforementioned multi-point personnel cluster characteristics includes: A pre-built rule base for determining campus safety risks has been established, including: If the number of clusters is greater than or equal to a preset cluster number threshold, it is marked as a general clustering risk; otherwise, there is no clustering risk. If the general risk of gathering is met, and the average duration of the visual focus of the corresponding personnel cluster is greater than or equal to the preset visual focus duration threshold, or the visual focus of the corresponding personnel cluster is pointing in the same direction, then it is marked as a high risk of gathering, and an instruction is issued to the relevant security department. If the general clustering risk is met, and the average clustering and dispersal time of the corresponding personnel cluster is less than or equal to the preset clustering and dispersal time threshold or the clustering and dispersal rate is greater than or equal to the preset clustering and dispersal rate threshold, then the general clustering risk is corrected to no clustering risk. If none of the above risk conditions are met, the condition is determined to be safe.

Images