Intelligent campus security abnormal behavior detection method and system based on image recognition

Abstract

The application relates to the field of computer vision and security technology, and discloses a kind of intelligent campus security abnormal behavior detection method and system based on image recognition, method includes: obtaining the continuous image sequence of monitoring area, and the multiple targets in image sequence are identified and motion tracking, to extract the motion trajectory of each target;According to the motion trajectory, the motion vector corresponding to each target is extracted;At least one target area is demarcated in the monitoring area, vector synthesis operation is carried out on all motion vectors in the target area, and the quantization parameter is obtained;The quantization parameter is compared with the preset order threshold value;When it is judged that the behavior mode of target group does not have organization, the real-time state parameter in the target area is obtained, the real-time state parameter is verified with the corresponding abnormal judgment benchmark, and an abnormal early warning signal is output according to the verification result.The application significantly improves the accuracy and signal-to-noise ratio of the early warning signal, so that the security resources can more concentratedly respond to the real and effective alarm.

Description

Technical Field

[0001] This application relates to the fields of computer vision and security technology, and more specifically, to a method and system for detecting abnormal behavior in smart campus security based on image recognition. Background Technology

[0002] In the security system of smart campuses, video surveillance has become an essential infrastructure for ensuring safety. Existing abnormal behavior detection technologies typically identify potential gathering risks by analyzing the density, number, or duration of crowds in the surveillance footage.

[0003] However, this method faces serious challenges in a school environment. Schools experience numerous normal, organized group activities, such as crowds moving in school corridors during breaks, orderly queues before meals in the cafeteria, and class assemblies on the playground. These activities visually appear as high-density gatherings of people, making them highly susceptible to misjudgment as abnormal events by traditional detection methods, resulting in a large number of invalid warnings. This high false alarm rate not only increases the burden on security personnel but, more seriously, dilutes their sensitivity to genuine alarms, potentially causing real abnormal gatherings to be buried under a sea of ​​false alarms and fail to be addressed in a timely manner. Fundamentally, the problem lies in the fact that current technology only focuses on the apparent physical characteristics of crowds, lacking the ability to deeply analyze and judge the inherent organization of group behavior, and thus failing to effectively distinguish between purposeful, orderly normal gatherings and disorganized, potentially dangerous abnormal gatherings.

[0004] To address the aforementioned issues, existing technologies urgently need improvement. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this application discloses a smart campus security abnormal behavior detection method and system based on image recognition. It aims to solve the problem that existing technologies only focus on the apparent physical characteristics of crowds, lack the ability to deeply analyze and judge the internal organization of group behavior, and cannot effectively distinguish between purposeful and orderly normal gatherings and disorganized and potentially dangerous abnormal gatherings.

[0006] The technical solution of this application is as follows: In a first aspect, this application discloses a method for detecting abnormal behavior in smart campus security based on image recognition, comprising the following steps: Acquire a continuous image sequence of the monitoring area, and identify and track multiple targets in the image sequence to extract the motion trajectory of each target; Extract the motion vectors corresponding to each target based on the motion trajectory; The layout information of the monitoring area is obtained, and at least one target area is delineated within the monitoring area based on the layout information. Vector synthesis is performed on all motion vectors within the target area to obtain quantitative parameters that characterize the consistency of the movement direction of the target group. The quantitative parameters are compared with the preset orderliness threshold to identify whether the behavior patterns of the target group are organized. When the behavior pattern of the target group is determined to be disorganized, real-time status parameters within the target area are obtained, the real-time status parameters are verified against the corresponding anomaly judgment benchmark, and an anomaly warning signal is output based on the verification results.

[0007] By introducing a step to judge the organization of the target group's behavior, the system first distinguishes between organized normal activities and unorganized potential abnormal clusters. Then, it conducts in-depth state parameter verification on the latter, thereby effectively avoiding misjudging normal high-density campus activities (such as queuing and passing through between classes) as abnormal, significantly reducing the system's false alarm rate, and improving the accuracy and reliability of the early warning.

[0008] Furthermore, the steps of performing vector synthesis on all motion vectors within the target area to obtain quantitative parameters characterizing the consistency of the target group's movement direction include: Convert the motion vectors of each target into unit direction vectors; The composite vector is obtained by combining all unit direction vectors within the target area; Obtain the total number of targets within the target area; The ratio of the magnitude of the synthesized vector to the total number of targets within the target area is calculated and used as a quantization parameter.

[0009] Furthermore, the real-time status parameters include at least the population distribution density, the average movement speed of the target group, and the duration of the current state; the anomaly judgment criteria include preset density threshold, preset speed threshold, and preset duration threshold. The steps of verifying real-time status parameters against the corresponding anomaly judgment benchmarks and outputting anomaly warning signals based on the verification results include: Compare the population distribution density, the average movement rate of the target group, and the duration of the current state with the corresponding thresholds in the anomaly judgment criteria. When the population density exceeds a preset density threshold, the average movement rate of the target group is lower than a preset speed threshold, and the duration of the current state reaches a preset duration threshold, the verification is deemed successful, and an abnormal warning signal is triggered.

[0010] Furthermore, before the step of verifying the real-time status parameters against the corresponding anomaly determination benchmark, the following steps are also included: Obtain the timestamp information for the current time period, and retrieve the corresponding historical normal feature database based on the timestamp information; Extract the expected density distribution and expected rate interval that match the current time period from the historical normal feature database; The anomaly detection criteria are modified based on the expected density distribution and the expected rate range.

[0011] Furthermore, the steps of identifying and tracking the motion of multiple targets in the image sequence to extract the motion trajectory of each target include: Extract local features from each image frame in the image sequence, and generate corresponding feature descriptors based on the local features; A pre-defined classifier is used to classify and identify feature descriptors in order to determine human-shaped targets in image frames; Based on the visual characteristics and spatial location of the human-shaped target, a prediction and correction algorithm is used to continuously track the movement trajectory of the human-shaped target.

[0012] Furthermore, the steps of comparing the quantified parameters with a preset orderliness threshold to identify whether the target group's behavioral patterns are organized include: If the quantification parameter is higher than the orderliness threshold, the behavior pattern of the target group is determined to be organized activity and has organizational characteristics. If the quantification parameter is lower than or equal to the orderliness threshold, the behavior pattern of the target group is determined to be a potential unorganized aggregation, lacking organization.

[0013] Furthermore, before obtaining the quantitative parameters used to characterize the consistency of the target group's movement direction, the process also includes: Determine whether the average movement speed of the target group is lower than the preset critical speed value; If the average movement speed is lower than the critical speed value, then extract the body orientation vector of each target within the target area; The steps of comparing quantitative parameters with preset orderliness thresholds to identify whether the behavioral patterns of a target group are organized include: The quantitative parameters are compared with the preset orderliness threshold, and combined with the body orientation vector of each target, the behavior pattern of the target group is identified as organized.

[0014] Furthermore, the steps of comparing the quantified parameters with a preset orderliness threshold and combining this with the body orientation vectors of each target to identify whether the behavioral patterns of the target group are organized include: Calculate the intersection distribution characteristics of the extension lines of the body orientation vector within the target area; Assess the visual focus convergence of the target group based on the characteristics of the intersection distribution; If the visual focus convergence conforms to the preset geometric distribution pattern and the quantification parameter is higher than the orderliness threshold, the target group is determined to be in an organized and clustered state of common concern, and thus possesses organization. Otherwise, the target group is deemed to lack organization.

[0015] Furthermore, if the quantization parameter is higher than the orderliness threshold, the method also includes: Real-time monitoring of the rate of change in the average movement speed of the target group; When the rate of change exceeds the preset jump threshold within a preset time threshold, the risk assessment procedure is initiated. The risk assessment process includes: Retrieve historical normal information for the current time period; Monitor the space occupancy rate of people within the target area and the average spatial distance between individuals; If the space occupancy rate rises above the preset warning threshold and the average space distance shrinks below the preset safe distance, it is determined that there is an abnormal flow in the target area. The average movement rate is compared with the historical peak rate in the historical normal information. If the average movement rate is higher than the historical peak rate, it is determined that an abnormal synchronous escape has occurred in the target area, triggering an emergency warning.

[0016] Secondly, this application also discloses a smart campus security abnormal behavior detection system based on image recognition, including: The image acquisition and tracking module is used to acquire a continuous image sequence of the monitoring area, and to identify and track the motion of multiple targets in the image sequence in order to extract the motion trajectory of each target; The vector extraction module is used to extract the motion vectors corresponding to each target based on the motion trajectory. The feature statistics module is used to acquire the layout information of the monitoring area, and delineate at least one target area within the monitoring area based on the layout information. It performs vector synthesis operation on all motion vectors within the target area to obtain quantitative parameters that characterize the consistency of the movement direction of the target group. The pattern recognition module is used to compare the quantification parameters with the preset orderliness threshold to identify whether the behavior patterns of the target group are organized. The early warning output module is used to obtain real-time status parameters in the target area when the behavior pattern of the target group is determined to be disorganized, verify the real-time status parameters with the corresponding anomaly judgment benchmark, and output an anomaly early warning signal based on the verification result.

[0017] In summary, this application provides a method and system for detecting abnormal behavior in smart campus security based on image recognition. The method analyzes the group movement vectors of people within a monitored area to calculate quantitative parameters characterizing the consistency of the group's movement direction, and uses these parameters to determine whether the group's behavior is organized. Subsequently, only for groups determined to be disorganized, further anomaly verification is performed using real-time status parameters such as personnel distribution density, average movement speed, and duration. This application fundamentally solves the problem of high false alarm rates in existing technologies due to their inability to distinguish between normal, orderly gatherings and disorderly loitering, significantly improving the accuracy and signal-to-noise ratio of early warning signals, reducing the workload of security personnel, and enabling security resources to respond more effectively to genuine alarms. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the image recognition-based smart campus security abnormal behavior detection method provided in an embodiment of this application.

[0019] Figure 2 This is a schematic diagram of the structure of the image recognition-based smart campus security abnormal behavior detection system provided in an embodiment of this application.

[0020] Figure 3 This is a schematic diagram of the process for calculating quantization parameters provided in an embodiment of this application.

[0021] Labeling Explanation: 210, Image Acquisition and Tracking Module; 220, Vector Extraction Module; 230, Feature Statistics Module; 240, Pattern Recognition Module; 250, Early Warning Output Module. Detailed Implementation

[0022] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0023] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0024] To better understand the technical solutions disclosed in this application, detailed descriptions will be provided below through specific embodiments. It should be noted that these embodiments are merely illustrative of the technical concept of this application and are not intended to limit the scope of protection of this application.

[0025] Firstly, referring to Figure 1 This application proposes a method for detecting abnormal behavior in smart campus security based on image recognition. The method includes the following steps: S1. Acquire a continuous image sequence of the monitoring area, and identify and track the motion of multiple targets in the image sequence to extract the motion trajectory of each target; S2. Extract the motion vectors corresponding to each target based on the motion trajectory; S3. Obtain the layout information of the monitoring area, and delineate at least one target area within the monitoring area based on the layout information. Perform vector synthesis operation on all motion vectors within the target area to obtain quantitative parameters used to characterize the consistency of the target group's movement direction. S4. Compare the quantification parameters with the preset orderliness threshold to identify whether the target group's behavior pattern is organized. S5. When it is determined that the behavior pattern of the target group is not organized, obtain the real-time status parameters in the target area, verify the real-time status parameters with the corresponding anomaly judgment benchmark, and output an anomaly warning signal based on the verification result.

[0026] In the context of this application, organization refers to the non-random, inherently regular, or shared-goal-oriented behavior of a majority of individuals within a group. This characteristic can manifest as a unified direction of movement, such as a group of people heading towards the same exit; or as a unified focus of attention, such as a group of people facing a stage or the center of an event. Organized group activities are usually normal and predictable, while unorganized group behavior is more likely to be associated with unusual events.

[0027] The quantification parameter is an indicator introduced in this application to transform the abstract concept of organization into a calculable and comparable numerical value. This parameter is calculated by comprehensively analyzing the movement directions of all individuals in the group to obtain a value that reflects the degree of consistency in the overall group's movement direction. The higher the value, the more unified the group's movement direction and the stronger its organization; conversely, the lower the value, the more chaotic the group's movement direction and the weaker its organization.

[0028] In practice, the first step is to acquire a continuous sequence of images of the monitored area. One or more wide-angle, high-definition network cameras can be installed at the highest point of the teaching building, ensuring their field of view completely covers the entire plaza area. These cameras continuously acquire video data at a set frame rate, such as 25 frames per second, forming a temporally continuous image sequence, which is then transmitted to a backend analysis server.

[0029] Next, the analysis server processes the received image sequences to identify and track multiple targets, primarily humanoid targets. This process is fundamental to all subsequent analysis, and its accuracy is crucial.

[0030] In a specific implementation, the process can be divided into two stages: human target identification and continuous tracking of movement trajectory.

[0031] For human-shaped targets, deep learning-based target detection algorithms can be used. For example, a pre-trained YOLOv5 model can be deployed. This model has been trained on large public datasets (such as the COCO dataset) and inherently possesses the ability to recognize people. To improve recognition accuracy in specific campus scenarios, the model can be fine-tuned using video clips collected on campus to better adapt to students' unique appearances (such as school uniforms and backpacks) and imaging characteristics under different lighting and weather conditions. When processing each frame of the image, the model can quickly outline all identified human-shaped targets in the image and provide the bounding box coordinates for each target.

[0032] After identifying human targets, it's necessary to correlate these targets across consecutive image frames to form each person's motion trajectory. A prediction correction algorithm, such as a combination of a Kalman filter and the Hungarian algorithm, can be used. Specifically, for each person detected in frame N, the Kalman filter predicts their possible location in frame N+1 based on their current position and velocity. Once the detection results for frame N+1 are available, the distance or appearance similarity between the predicted location of each tracked target in frame N and the locations of all newly detected targets in frame N+1 is calculated, forming a cost matrix. Then, the Hungarian algorithm is used to solve this cost matrix, matching each target in frame N to its corresponding target in frame N+1 based on the lowest overall cost. By repeatedly performing this loop, a time-varying sequence of coordinate points—the motion trajectory—can be generated for each student in the square.

[0033] After obtaining the motion trajectory of each target, the next step is to extract the motion vector of each target within a short period of time based on these trajectories. The motion vector visually represents the direction and velocity of the target's movement at a given instant. The calculation method can be as follows: for a given target, take its position coordinates (xt, yt) at time point t and its position coordinates (xt+Δt, yt+Δt) at a later time point t+Δt. Then, its motion vector within this time interval Δt (which can be set according to actual needs, such as a preset multiple of the image frame sampling period) can be represented as (xt+Δt-xt, yt+Δt-yt). This vector contains information about the direction and distance of the target's movement.

[0034] Subsequently, these motion vectors need to be used to calculate the quantitative parameters mentioned earlier, which characterize the consistency of the target group's movement direction. Before performing the calculation, the calculation area, i.e., the target region, needs to be known. This can be done by manually drawing on the electronic map of the monitoring screen to delineate one or more target areas that require special attention. For example, the main part of the plaza in front of the entire teaching building can be designated as a polygonal target region.

[0035] After the target area is determined, a vector synthesis operation is performed on the motion vectors of all targets within that area. In a preferred embodiment, this operation process is as follows: First, the motion vectors of each target are converted into unit direction vectors. This step aims to eliminate the interference of individual speed differences on the judgment of directional consistency. For example, a student who runs very fast and a student who walks slowly should contribute equally to the group's organization if they are moving in the same direction. Specifically, the motion vector (Δx, Δy) of each target is divided by its own magnitude, i.e., sqrt(Δx² + Δy²), resulting in a unit vector of length 1 but with an unchanged direction.

[0036] Next, vector synthesis is performed on all these unit direction vectors within the target area. This involves adding the x-components of all unit vectors to obtain a synthesized x-component; and adding all y-components to obtain a synthesized y-component. This yields a final synthesized vector.

[0037] Next, obtain the total number of targets within the target area, denoted as N.

[0038] Finally, the magnitude of the synthesized vector is calculated, and the ratio of this magnitude to the total number of targets N is used as the final quantization parameter. This ratio ranges from 0 to 1.

[0039] For example, suppose there are 50 students in a square, all walking towards building B. Their unit direction vectors will be almost identical, and the magnitude of the composite vector will be very close to 50. In this case, the calculated quantization parameter will be close to 50 / 50 = 1. This indicates a highly consistent and well-organized group movement. Conversely, if these 50 students move haphazardly in various directions, their unit direction vectors will cancel each other out during composition, resulting in a very small magnitude of the composite vector, close to 0. In this case, the calculated quantization parameter will also be close to 0, indicating extremely disorganized group behavior.

[0040] After obtaining the quantified parameters, they are compared with a preset orderliness threshold to identify whether the target group's behavior patterns are organized. This orderliness threshold is an empirical value that can be set by the administrator according to the actual situation, for example, 0.6.

[0041] If the calculated quantification parameter is higher than the orderliness threshold (e.g., a calculated value of 0.8), the behavior pattern of the current target group is determined to be organized activity, exhibiting organizational characteristics. For example, this could very well correspond to a group of students walking together across the square towards teaching building B. In this case, the system will consider it normal activity and will not perform further anomaly checks, thus avoiding false alarms.

[0042] If the calculated quantification parameter is lower than or equal to the orderliness threshold (e.g., a calculated value of 0.3), the system determines that the target group's behavior pattern is a potential unorganized gathering, lacking organization. This could be students chatting and wandering around in small groups in a square, or it could be a sudden event causing a disorderly crowd to gather. This state is a candidate abnormal state that requires further analysis.

[0043] The next verification process will be initiated only if the target group's behavior pattern is determined to be disorganized. At this point, a series of real-time status parameters within the target area will be immediately acquired and verified against the corresponding anomaly detection criteria.

[0044] In one specific implementation, the real-time state parameters include at least the population distribution density, the average movement speed of the target group, and the duration of the current state. The anomaly detection criteria correspondingly include preset density thresholds, preset speed thresholds, and preset duration thresholds.

[0045] Population density can be calculated by dividing the number of people in the target area by the area of ​​that area (e.g., square meters).

[0046] The average movement rate of the target group can be obtained by calculating the average of the magnitudes of the movement vectors of all targets within the region.

[0047] The duration of the current state is determined by a timer, which starts counting from the moment the region is determined to have entered a state of disorganization and stops counting from the moment the state of disorganization is determined to have ended. This duration is the duration of the current state.

[0048] The verification process checks whether the following three conditions are met simultaneously: whether the current population density exceeds a preset density threshold (e.g., more than 0.8 people per square meter), whether the average movement rate of the current target group is lower than a preset speed threshold (e.g., less than 0.2 meters per second), and whether the duration of this disorganized state has reached a preset duration threshold (e.g., 60 seconds).

[0049] Only when all three conditions mentioned above are met simultaneously—that is, a group not only moves erratically but also exhibits high density and is almost stationary, and this state persists for a period of time—will the system ultimately determine that the verification has passed and trigger an abnormal warning signal. This signal can be a prominent warning window popping up on the security center's monitoring screen, displaying real-time video footage of the area, highlighting the target group with a red box, and accompanied by text prompts and sound alarms indicating suspected abnormal gathering, requesting security personnel to pay immediate attention and take action.

[0050] Through the aforementioned two-stage judgment mechanism, the technical solution of this application enables refined management of crowd activities on campus. Orderly, rapid pedestrian flows through the plaza, due to their high quantification parameters, are identified as organized activity in the first stage and ignored, thus not triggering an alarm. Similarly, normal social groups with lower quantification parameters but low density, or those only briefly staying, cannot simultaneously meet the three conditions for subsequent verification and will also not trigger an alarm. Only groups exhibiting truly disorganized, high-density, and prolonged stays are identified, thereby significantly improving the accuracy of early warnings, reducing false alarm rates, and making the security system more intelligent and efficient.

[0051] Furthermore, to make the calculation of quantization parameters more accurate, the steps of performing vector synthesis operations on all motion vectors within the target area to obtain quantization parameters characterizing the consistency of the target group's movement direction include: Convert the motion vectors of each target into unit direction vectors; The composite vector is obtained by combining all unit direction vectors within the target area; Obtain the total number of targets within the target area; The ratio of the magnitude of the synthesized vector to the total number of targets within the target area is calculated and used as a quantization parameter.

[0052] As mentioned above, this specific calculation method eliminates the interference of individual speed differences through normalization and makes the final quantization parameters fall within a standardized [0,1] interval. This not only makes the physical meaning of the parameters clearer (1 represents complete order and 0 represents complete disorder), but also greatly simplifies the setting and debugging of subsequent order thresholds, and enhances the stability and universality of the entire scheme.

[0053] Furthermore, to make the conditions for anomaly detection more stringent, the real-time status parameters include at least the population distribution density, the average movement speed of the target group, and the duration of the current state; the anomaly detection criteria include preset density thresholds, preset speed thresholds, and preset duration thresholds. The steps of verifying real-time status parameters against the corresponding anomaly judgment benchmarks and outputting anomaly warning signals based on the verification results include: Compare the population distribution density, the average movement rate of the target group, and the duration of the current state with the corresponding thresholds in the anomaly judgment criteria. When the population density exceeds a preset density threshold, the average movement rate of the target group is lower than a preset speed threshold, and the duration of the current state reaches a preset duration threshold, the verification is deemed successful, and an abnormal warning signal is triggered.

[0054] As mentioned earlier, this judgment logic based on population density, average movement speed, and duration of activity provides a more complete and reliable basis for identifying abnormal clustering patterns. High density alone might indicate normal crowding during breaks, low speed alone might indicate normal queuing, and long duration alone might indicate loose social activity. Combining these three characteristics effectively identifies potentially risky individuals who warrant the highest level of security personnel attention, thus ensuring a high detection rate while minimizing interference from normal circumstances.

[0055] Based on the above-described embodiments, in order to further improve the adaptability and accuracy of the detection method, this application also proposes a series of optimized technical solutions.

[0056] There are significant differences in normal pedestrian traffic and activity intensity across different areas and at different times on campus. For example, the plaza in front of the teaching building is typically crowded during breaks, but should be relatively empty during class hours. If a fixed set of anomaly detection criteria (such as density thresholds and speed thresholds) is applied to all times and locations, it will inevitably lead to a large number of false alarms during peak hours or insufficient sensitivity to abnormal gatherings during off-peak hours.

[0057] To address this issue, this application further proposes that, prior to the step of verifying the real-time status parameters against the corresponding anomaly determination benchmark, the following steps are also included: Obtain the timestamp information for the current time period, and retrieve the corresponding historical normal feature database based on the timestamp information; Extract the expected density distribution and expected rate interval that match the current time period from the historical normal feature database; The anomaly detection criteria are modified based on the expected density distribution and the expected rate range.

[0058] In practice, a historical normal characteristic database needs to be pre-built. The construction process can be as follows: In the initial stage after deployment (e.g., the first two weeks), a learning mode is entered, continuously collecting and analyzing the population distribution density and average movement rate of each monitored area in different time segments (e.g., in 15-minute units), and calculating the statistical mean and standard deviation for each time segment. This data, with the area ID, time segment, expected density mean, expected density standard deviation, expected rate mean, and expected rate standard deviation stored in a one-to-one correspondence in the database, forms the historical normal characteristic database.

[0059] Once in full operation mode, whenever an anomaly verification is required, the current system time (e.g., Wednesday, 10:05 AM) is first retrieved. Based on this timestamp, the historical normal characteristic database is queried to find the historical data corresponding to this time slice. Assume the expected average density found is 0.7 people / square meter, and the expected average speed is 1.0 meter / second.

[0060] Next, these historical data are used to revise the basic anomaly detection criteria. For example, the basic density threshold is set at 0.8 people / square meter, but considering that the current period is already a high-density period, the revised density threshold can be adjusted to: basic threshold + k * expected average density (where k is an adjustable coefficient that can be preset according to different periods). If k is 0.5 for a certain period, then 0.8 + 0.5 * 0.7 = 1.15 people / square meter. Similarly, the speed threshold can also be adjusted accordingly.

[0061] Conversely, if the current time is during school hours (e.g., Wednesday, 11:00 AM), historical data shows that the expected density for that time is only 0.1 people per square meter. Therefore, the revised density threshold might be lowered, making it more sensitive to any small gatherings that occur at that time.

[0062] Furthermore, the aforementioned methods primarily rely on the target's motion vector to determine the group's organization. However, in some scenarios, while the group may exhibit high levels of organization, its overall movement can be very slow or even static. A typical example is an open-air lecture or a club performance in a public square, where a large number of students spontaneously form a circle to watch. Behaviorally, they share a common focus, exhibiting typical organized activity. However, because they are almost entirely still, their motion vector approaches zero, resulting in very low calculated quantification parameters. This could easily lead to misjudgment as a potentially disorganized and anomalous cluster.

[0063] Furthermore, to ensure the accuracy of target recognition and tracking, the steps of identifying and tracking multiple targets in the image sequence to extract the motion trajectory of each target include: Extract local features from each image frame in the image sequence, and generate corresponding feature descriptors based on the local features; A pre-defined classifier is used to classify and identify feature descriptors in order to determine human-shaped targets in image frames; Based on the visual characteristics and spatial location of the human-shaped target, a prediction and correction algorithm is used to continuously track the movement trajectory of the human-shaped target.

[0064] This scheme clarifies the technical path for the crucial data preprocessing stage, from the original image to the target trajectory. In the recognition stage, the image frame is first preprocessed with grayscale conversion, illumination correction, and Gaussian denoising. The image is divided into equal pixel cells, and the pixel gradient magnitude and orientation histogram within each cell are calculated as basic local features. Adjacent cells are combined into feature blocks and L2 normalized. After sliding through the entire image, all feature block vectors are concatenated, and dimensionality reduction is performed to obtain a standardized one-dimensional feature descriptor. After generating the one-dimensional hand-crafted feature descriptor, all descriptors are input into a pre-trained SVM classifier. Based on the human-shaped sample feature distribution learned during training, the classifier performs binary classification on the feature vectors within the image's sliding window, distinguishing human targets from non-human regions such as background, scenery, and clutter. It outputs the bounding box coordinates of all human targets, completing the recognition of human targets within the image frame. In the tracking stage, predictive correction algorithms (such as Kalman filtering) effectively handle situations where the target is occluded or missed by the detector in a short period, ensuring the continuity and smoothness of the trajectory.

[0065] Furthermore, in one implementation, to make the logic for determining organization clearer, the step of comparing quantitative parameters with a preset orderliness threshold to identify whether the behavioral patterns of a target group are organized includes: If the quantification parameter is higher than the orderliness threshold, the behavior pattern of the target group is determined to be organized activity and has organizational characteristics. If the quantification parameter is lower than or equal to the orderliness threshold, the behavior pattern of the target group is determined to be a potential unorganized aggregation, lacking organization.

[0066] This step simplifies complex group behavior patterns into a clear decision point, automatically diverting crowd activity to two different processing paths: if determined to be organized, it is considered normal and the process ends; if determined to be disorganized, it is considered suspicious and proceeds to the next stage of in-depth verification. This design makes the entire method's logical flow very clear, easy to implement and manage.

[0067] To compensate for the shortcomings of relying solely on motion vector analysis, in another embodiment, this application further proposes that, before obtaining the quantitative parameters used to characterize the consistency of the target group's movement direction, the method further includes: Determine whether the average movement speed of the target group is lower than the preset critical speed value; If the average movement speed is lower than the critical speed value, then extract the body orientation vector of each target within the target area; Then, the steps of comparing the quantified parameters with a preset orderliness threshold to identify whether the target group's behavioral patterns are organized include: The quantitative parameters are compared with the preset orderliness threshold, and combined with the body orientation vector of each target, the behavior pattern of the target group is identified as organized.

[0068] This scheme introduces a combined static and dynamic analysis strategy. First, the overall motion state of the group is determined. The critical velocity value can be set to a very low value, such as 0.1 m / s. When the monitored average motion velocity of the group is below this value, the group is considered to be in a quasi-static state. At this point, analyzing motion vectors alone is insufficient to determine its organization, and supplementary analysis methods, namely analyzing the body orientation of individuals, need to be initiated.

[0069] The extraction of the body orientation vector can be achieved through deeper image analysis techniques, such as human pose estimation algorithms. Models like OpenPose or MediaPipe Pose can be used, which can identify multiple key points in a human image, such as the shoulders, hips, and nose. Based on the positions of these key points, a vector representing the orientation of the human torso can be calculated, for example, a vector pointing from the center of the hips to the center of the shoulders.

[0070] Specifically, the steps of comparing quantified parameters with preset orderliness thresholds and combining this with the body orientation vectors of each target to identify whether the behavioral patterns of the target group are organized include: Calculate the intersection distribution characteristics of the extension lines of the body orientation vector within the target area; Assess the visual focus convergence of the target group based on the characteristics of the intersection distribution; If the visual focus convergence conforms to the preset geometric distribution pattern and the quantification parameter is higher than the orderliness threshold, the target group is determined to be in an organized and clustered state of common concern, and thus possesses organization. Otherwise, the target group is deemed to lack organization.

[0071] Intersection distribution characteristics refer to the set of spatial distribution statistical characteristics of all intersection points formed by the pairwise intersections of the extended rays of the body orientation vectors of all static human figures within the target area, within the two-dimensional monitoring imaging plane. It is a general quantitative indicator for quantifying the degree of convergence of people's line of sight / body orientation.

[0072] In the aforementioned performance viewing scenario, although the students' bodies remain stationary, their bodies and heads are mostly facing the center of the performance. After extracting the orientation vector of each student's body, these vectors are treated as rays on a two-dimensional plane and extended. If the crowd is indeed collectively focused on a particular focal point, then these extended lines would theoretically converge within a very small area—the area where the performer is located.

[0073] Visual focus convergence can be calculated by counting the intersections of all these extensions and analyzing the spatial distribution of these intersections. If more than a preset percentage (e.g., 70%) of the intersections are found to be concentrated within a circle of a preset radius (e.g., 2 meters), the visual focus convergence of the group is considered to be high, conforming to the preset geometric distribution law of centripetal aggregation.

[0074] At this point, even if the quantization parameters calculated based on motion vectors are low, as long as the visual focus convergence is high, it can be determined that the group is in an organized and focused state, exhibiting organizational characteristics, thus avoiding misjudging it as abnormal. Conversely, if a group of people are aimlessly lingering in the square, their body orientations will be random, and the intersection points of the extensions of their orientation vectors will be scattered, failing to form an effective convergence. Based on this, it can be determined that they lack organizational characteristics.

[0075] Finally, the technical solution of this application not only focuses on the identification of unorganized gatherings, but also further considers risk monitoring of groups that have been identified as organized. This is because an orderly group can also instantly transform into a high-risk abnormal state under certain stimuli, such as panic scattering and causing stampedes.

[0076] In this regard, this application further proposes that if the quantization parameter is higher than the orderliness threshold, the method also includes: Real-time monitoring of the rate of change in the average movement speed of the target group; When the rate of change exceeds the preset jump threshold within a preset time threshold, the risk assessment procedure is initiated. The risk assessment process includes: Retrieve historical normal information for the current time period; Monitor the space occupancy rate of people within the target area and the average spatial distance between individuals; If the space occupancy rate rises above the preset warning threshold and the average space distance shrinks below the preset safe distance, it is determined that there is an abnormal flow in the target area. The average movement rate is compared with the historical peak rate in the historical normal information. If the average movement rate is higher than the historical peak rate, it is determined that an abnormal synchronous escape has occurred in the target area, triggering an emergency warning.

[0077] Once a group is identified as engaging in organized activity, it is not left unattended. Instead, its macro-dynamics are continuously monitored with relatively low resource consumption, particularly the rate of change in average movement speed. When the rate of change in the average movement speed of an orderly group exceeds a preset threshold within a short-term window, a risk assessment procedure is immediately initiated. This procedure first retrieves historical routine information, which consists of routine pedestrian flow statistics collected and stored during the initial learning phase, categorized by monitoring area and time slice. This data includes the expected density of normal pedestrian flow during the corresponding time period, the range of normal movement speeds, and historical peak speeds. This information provides a dynamic benchmark for the current scenario, avoiding misjudgments caused by fixed thresholds.

[0078] Subsequently, two spatial congestion characteristics were collected simultaneously: the space occupancy rate of people in the target area and the average spatial distance between individuals. The space occupancy rate of people refers to the ratio of the total area occupied by all human figures in the area to the total area of ​​the monitored target area, which directly reflects the degree of crowd congestion.

[0079] If the monitored area shows a space occupancy rate exceeding a preset warning threshold and an average distance between individuals falling below a preset safe distance, an abnormal flow of people, indicating crowding and pushing, is identified. The real-time average movement rate of the group is then compared to historical peak rates in historical data. If the real-time rate exceeds the historical peak, an abnormal synchronous escape is detected, and the highest-level emergency warning is issued. This warning signal can trigger the activation of all emergency exits, highlight alarms in monitoring pop-ups, and push notifications to security terminals, proactively intervening in high-risk events that could lead to stampedes, such as panic at gatherings, rushing to exits, and sudden conflicts, thus buying valuable time for campus emergency response.

[0080] Suppose that after an event, a large number of students are orderly flowing out of the auditorium, moving in the same direction with high quantitative parameters, and are judged as a normal, organized evacuation. Suddenly, someone in the crowd screams, causing panic. Within a mere 2 seconds, the average movement speed of the crowd is detected to surge from 1.2 m / s to 3.0 m / s, a rate of change far exceeding the preset threshold. A risk assessment procedure is immediately initiated.

[0081] The program first retrieved historical normal information and found that the historical peak evacuation rate during normal evacuation of the auditorium never exceeded 1.8 meters per second.

[0082] Meanwhile, the program analyzes the spatial parameters of the crowd in real time and finds that due to pushing and shoving, the space occupancy rate of people near the exit has soared from 70% to 95%, and the average spatial distance between individuals has been compressed to less than 0.4 meters, which is far below the safe distance.

[0083] Based on the above information: a drastic change in speed, extremely crowded spatial parameters, and a current speed far exceeding historical peak values, a final judgment can be made: an abnormal synchronous escape has occurred in the target area, immediately triggering the highest level of emergency warning, notifying security personnel to rush to the scene, and possibly coordinating with other security equipment, such as opening all emergency exits and playing reassuring broadcasts, thereby buying valuable time to deal with stampedes or other serious incidents.

[0084] Secondly, referring to Figure 2 This application also provides a smart campus security abnormal behavior detection system based on image recognition, the system comprising: The image acquisition and tracking module 210 is used to acquire a continuous image sequence of the monitoring area and to identify and track multiple targets in the image sequence in order to extract the motion trajectory of each target. Vector extraction module 220 is used to extract the motion vectors corresponding to each target based on the motion trajectory; The feature statistics module 230 is used to acquire the layout information of the monitoring area, and delineate at least one target area in the monitoring area according to the layout information, and perform vector synthesis operation on all motion vectors in the target area to obtain quantitative parameters for characterizing the consistency of the movement direction of the target group. The pattern recognition module 240 is used to compare the quantification parameters with the preset orderliness threshold to identify whether the behavior pattern of the target group is organized. The early warning output module 250 is used to obtain real-time status parameters in the target area when the behavior pattern of the target group is determined to be disorganized, verify the real-time status parameters with the corresponding anomaly judgment benchmark, and output an anomaly early warning signal based on the verification result.

[0085] In actual deployment, the system can be a software program deployed on a campus security center server, or it can be an edge computing device that integrates high-performance computing units (such as an embedded motherboard with a GPU).

[0086] The image acquisition and tracking module 210 corresponds in hardware to the network cameras deployed throughout the campus, and in software to the program process on the server responsible for receiving video streams and running target detection and tracking algorithms (such as the aforementioned YOLOv5 and Kalman filter combination scheme). The output of this module is to assign a unique ID to each identified person and continuously output the real-time coordinate sequence corresponding to that ID.

[0087] The vector extraction module 220 receives trajectory data from the image acquisition and tracking module 210 and calculates the instantaneous motion vector of each target at each time point by performing differential operations on continuous coordinate points.

[0088] The feature statistics module 230 first needs to load the preset area layout information, that is, the coordinates of the electronic fences of each monitoring area defined by the administrator. Then, it filters out the motion vectors of all targets falling into a specific target area, and performs a series of operations such as normalization, vector synthesis, modulus calculation and normalization, and finally outputs the key quantization parameters.

[0089] The pattern recognition module 240 receives the quantization parameters output by the feature statistics module 230 and compares them with the orderliness threshold stored in the configuration, thereby making a preliminary judgment on whether the behavior pattern of the group is organized or not.

[0090] The early warning output module 250 is only activated upon receiving an unorganized judgment signal from the pattern recognition module 240. Upon activation, it immediately collects status parameters such as density, speed, and duration of the current area and performs a final verification against anomaly judgment criteria (which may have been corrected using historical data). Once all conditions are met, the module calls the corresponding interface to trigger various forms of alarms, such as audible, visual, and electrical alarms. For example, it may push alarm information to security personnel's mobile app via the network or highlight the alarm area on the electronic map in the monitoring center.

[0091] The above are merely embodiments of this application and are not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for detecting abnormal behavior in smart campus security based on image recognition, characterized in that, The method includes the following steps: A continuous image sequence of the monitoring area is acquired, and multiple targets in the image sequence are identified and their motion is tracked to extract the motion trajectory of each target; Extract the motion vectors corresponding to each target based on the motion trajectory; The layout information of the monitoring area is obtained, and at least one target area is delineated in the monitoring area according to the layout information. Vector synthesis operation is performed on all the motion vectors in the target area to obtain a quantitative parameter for characterizing the consistency of the movement direction of the target group. The quantification parameters are compared with a preset orderliness threshold to identify whether the behavior patterns of the target group are organized. When it is determined that the behavior pattern of the target group is not organized, real-time status parameters of the target area are obtained, the real-time status parameters are verified with the corresponding anomaly judgment benchmark, and an anomaly warning signal is output based on the verification result.

2. The method for detecting abnormal behavior in smart campus security based on image recognition according to claim 1, characterized in that, The step of performing vector synthesis on all motion vectors within the target area to obtain quantitative parameters characterizing the consistency of the target group's movement direction includes: The motion vectors of each target are converted into unit direction vectors; A composite vector is obtained by combining all unit direction vectors within the target area. Obtain the total number of targets within the target area; The ratio of the magnitude of the synthesized vector to the total number of targets within the target region is calculated and used as the quantization parameter.

3. The method for detecting abnormal behavior in smart campus security based on image recognition according to claim 1, characterized in that, The real-time status parameters include at least the population distribution density, the average movement speed of the target group, and the duration of the current status; The anomaly detection criteria include preset density threshold, preset speed threshold, and preset duration threshold; The step of verifying the real-time status parameters against the corresponding anomaly determination benchmark and outputting an anomaly warning signal based on the verification result includes: The population distribution density, the average movement rate of the target group, and the duration of the current state are compared with the corresponding thresholds in the anomaly determination criteria. When the population density exceeds a preset density threshold, the average movement rate of the target group is lower than a preset speed threshold, and the duration of the current state reaches a preset duration threshold, the verification is deemed successful, and the abnormal warning signal is triggered.

4. The method for detecting abnormal behavior in smart campus security based on image recognition according to claim 3, characterized in that, Before the step of verifying the real-time status parameters with the corresponding anomaly determination benchmark, the method further includes: Obtain the timestamp information for the current time period, and retrieve the corresponding historical normal feature database based on the timestamp information; Extract the expected density distribution and expected rate interval that match the current time period from the historical normal feature database; The anomaly determination criteria are modified based on the expected density distribution and the expected rate range.

5. The method for detecting abnormal behavior in smart campus security based on image recognition according to claim 1, characterized in that, The step of identifying and tracking the motion of multiple targets in the image sequence to extract the motion trajectory of each target includes: Local features of each image frame in the image sequence are extracted, and corresponding feature descriptors are generated based on the local features; The feature descriptors are classified and identified using a preset classifier to determine the human-shaped targets in the image frame; Based on the visual characteristics and spatial location of the humanoid target, a prediction and correction algorithm is used to continuously track the movement trajectory of the humanoid target.

6. The method for detecting abnormal behavior in smart campus security based on image recognition according to claim 1, characterized in that, The step of comparing the quantification parameters with a preset orderliness threshold to identify whether the behavioral patterns of the target group are organized includes: If the quantification parameter is higher than the orderliness threshold, the behavior pattern of the target group is determined to be organized activity and has organizational characteristics. If the quantification parameter is lower than or equal to the orderliness threshold, the behavior pattern of the target group is determined to be a potential unorganized aggregation, lacking organization.

7. The method for detecting abnormal behavior in smart campus security based on image recognition according to claim 1, characterized in that, Before the step of obtaining the quantitative parameters used to characterize the consistency of the target group's movement direction, the method further includes: Determine whether the average movement speed of the target group is lower than a preset critical speed value; If the average movement speed is lower than the critical speed value, then the body orientation vector of each target within the target area is extracted; The step of comparing the quantification parameters with a preset orderliness threshold to identify whether the behavioral patterns of the target group are organized includes: The quantification parameters are compared with a preset orderliness threshold, and combined with the body orientation vectors of each target, the behavior patterns of the target group are identified as organized.

8. The method for detecting abnormal behavior in smart campus security based on image recognition according to claim 7, characterized in that, The step of comparing the quantification parameters with a preset orderliness threshold and combining the body orientation vectors of each target to identify whether the behavior pattern of the target group is organized includes: Calculate the intersection distribution characteristics of the extension lines of the body orientation vector within the target area; The visual focus convergence of the target group is assessed based on the aforementioned intersection distribution characteristics; If the visual focus convergence conforms to a preset geometric distribution pattern and the quantification parameter is higher than the orderliness threshold, it is determined that the target group is in an organized and clustered state of common concern and has organizational characteristics. Otherwise, the target group is deemed to lack organization.

9. The method for detecting abnormal behavior in smart campus security based on image recognition according to claim 6, characterized in that, If the quantization parameter is higher than the orderliness threshold, the method further includes: Real-time monitoring of the rate of change of the average movement speed of the target group; When the rate of change exceeds a preset jump threshold within a preset time threshold, a risk assessment procedure is initiated. The risk assessment procedure includes: Retrieve historical normal data for the current time period; Monitor the space occupancy rate of people within the target area and the average spatial distance between individuals; If the space occupancy rate rises above the preset warning threshold and the average space distance shrinks below the preset safe distance, it is determined that there is an abnormal flow in the target area. The average movement rate is compared with the historical rate peak in the historical normal information. If the average movement rate is higher than the historical rate peak, it is determined that an abnormal synchronous escape has occurred in the target area, triggering an emergency warning.

10. A smart campus security abnormal behavior detection system based on image recognition, characterized in that, include: The image acquisition and tracking module is used to acquire a continuous image sequence of the monitoring area, and to identify and track the motion of multiple targets in the image sequence in order to extract the motion trajectory of each target; The vector extraction module is used to extract the motion vectors corresponding to each of the targets based on the motion trajectory. The feature statistics module is used to acquire the layout information of the monitoring area, and delineate at least one target area in the monitoring area according to the layout information, and perform vector synthesis operation on all the motion vectors in the target area to obtain a quantitative parameter for characterizing the consistency of the movement direction of the target group. The pattern recognition module is used to compare the quantification parameters with a preset orderliness threshold to identify whether the behavior pattern of the target group is organized. The early warning output module is used to obtain real-time status parameters in the target area when it is determined that the behavior pattern of the target group is not organized, verify the real-time status parameters with the corresponding anomaly judgment benchmark, and output an anomaly early warning signal based on the verification result.

Images