Computer vision auxiliary-based park safety situation awareness method and system

Abstract

The application provides a park security situation awareness method and system based on computer vision assistance, relates to the technical field of data analysis, and comprises the following steps: collecting multi-source sensing data in a park in real time; receiving and executing light inference through an edge computing node; uploading local situation awareness information generated by each edge computing node to a cloud server, constructing a space-time trajectory graph, and making a situation judgment according to a preset hierarchical early warning model; when the judgment result reaches a warning triggering condition, generating warning information and pushing the warning information to a mobile terminal and a command center; and responding to voice intercom and shouting instructions to perform voice intervention on the scene. The application solves the technical problem that, in the prior art, due to the lack of continuous description and deep understanding of cross-space-time behavior, it is difficult to discover hidden illegal routes and potential risk patterns, which affects the security of the park, improves the identification ability of hidden illegal behavior and potential risk patterns, and improves the security of the park.

Description

Technical Field

[0001] This application relates to the field of data analysis technology, specifically to a computer vision-assisted method and system for campus security situation awareness. Background Technology

[0002] Park video surveillance systems are widely deployed, achieving basic coverage of the park through cameras distributed at various key locations, enabling preliminary real-time monitoring and recording of personnel and vehicle traffic. However, most park video surveillance systems currently remain in a single-point, isolated monitoring mode. Each camera independently captures and displays images of its designated local area, requiring manual observation of multiple fragmented feeds simultaneously. The video data from different locations lacks inherent correlation and collaborative analysis capabilities. Essentially, park video surveillance systems serve only as tools for post-event tracing, failing to provide valuable situational assessment support during incidents. Due to the fragmented distribution of information on various security elements within the park, the appearance of the same person under different cameras is recorded as isolated, discrete points, unable to be automatically linked into a complete and continuous movement trajectory, nor can the behavioral patterns and intentions of personnel or vehicles be depicted from a global perspective. For concealed violations, visual observation of multiple feeds is almost insufficient to detect them in time, resulting in a large amount of highly valuable information being lost in massive amounts of video data, severely hindering park security and prevention.

[0003] In summary, existing technologies suffer from the problem that security personnel struggle to detect hidden violations and identify potential risk patterns because each security element within the park is monitored at a single point, lacking continuous characterization and in-depth understanding of cross-temporal and spatial behaviors, which further impacts the security of the park. Summary of the Invention

[0004] The purpose of this application is to provide a computer vision-assisted method and system for campus security situational awareness, in order to solve the technical problems in the existing technology where each security element in the campus is in a single-point monitoring state, lacking continuous characterization and in-depth understanding of cross-temporal and spatial behavior, which makes it difficult for security personnel to discover hidden illegal movement lines and identify potential risk patterns, thus further affecting the security of the campus.

[0005] To achieve the above objectives, this application provides a computer vision-assisted method and system for campus security situation awareness.

[0006] Firstly, this application provides a computer vision-assisted method for campus security situation awareness. This method is implemented through a computer vision-assisted campus security situation awareness system. The method includes: real-time acquisition of multi-source sensor data containing video images and audio signals within the campus; receiving the multi-source sensor data through edge computing nodes, performing lightweight inference, and forming local situation awareness information; uploading the local situation awareness information generated by each edge computing node to a cloud server; the cloud server aggregating situation awareness information from the entire campus area and constructing a spatiotemporal trajectory map; performing situation analysis based on the spatiotemporal trajectory map according to a preset hierarchical early warning model; and generating early warning information and pushing it to a mobile terminal and command center when the analysis results meet the early warning triggering conditions, responding to voice intercom and announcement commands from the command center or mobile terminal, and conducting voice intervention on-site through front-end deployed audio output devices.

[0007] Optionally, existing video surveillance cameras within the park can be reused, and microphones can be installed at the locations of the cameras to collect multi-source sensor data, including video images and audio signals, in real time.

[0008] Optionally, target detection is performed on the video images to extract people and vehicles from the video frame; for people, human skeleton key point tracking is used to identify abnormal behavior; voice keyword wake-up detection and sound intensity detection are performed on the audio signals collected by the microphone to identify abnormal sounds; combined with the abnormal behavior and abnormal sounds, abnormal targets are located; for abnormal targets, cross-camera trajectory association is performed on the same abnormal target in video images from different cameras to generate cross-camera trajectory association data; the local situational awareness information is generated using the abnormal behavior, cross-camera trajectory association data, and abnormal sounds.

[0009] Optionally, facial recognition is used to identify abnormal targets in video images and extract their appearance features. When the same abnormal person is detected in video images from different cameras, the spatiotemporal locations of the same abnormal person under different cameras are associated based on the similarity matching between the identification results and appearance features, forming a continuous movement trajectory of the corresponding abnormal person in the park. The cross-camera trajectory association data is generated using the continuous movement trajectory of each abnormal person.

[0010] Optionally, the cloud server receives local situational awareness information uploaded by each edge computing node, merges and deduplicates the location information of the same abnormal target reported by different edge nodes within adjacent time windows and adjacent spatial areas, forming complete spatiotemporal trajectory data of each abnormal target within the park; using the park's electronic map as a base map, the complete spatiotemporal trajectory data of each abnormal target is overlaid on the base map to construct the spatiotemporal trajectory map; for each person or vehicle in the spatiotemporal trajectory map, single-point rule analysis, time-series pattern analysis, and correlation analysis are performed in parallel; when any analysis level triggers an early warning condition, the corresponding early warning type and early warning level are determined, and the early warning information is generated.

[0011] Optionally, the spatiotemporal trajectory map includes the location coordinates, dwell time, direction of travel, and sequence of points passed by each person and vehicle at each time point in the park.

[0012] Optionally, the single-point rule analysis specifically involves: determining the spatial relationship between the current location of a person or vehicle and preset multiple spatial areas; when the location of a person or vehicle falls into any preset spatial area, a corresponding type of single-point warning condition is triggered; wherein, the preset multiple spatial areas include at least prohibited areas, key monitoring areas, and temporary control areas.

[0013] Optionally, the timing pattern analysis specifically involves: performing timing analysis on the trajectory sequence of personnel or vehicles within a preset time window, and triggering a timing warning condition when the trajectory sequence matches a preset abnormal timing pattern; the preset abnormal timing pattern includes at least an abnormal occurrence pattern, an abnormal stay pattern, an overdue stay pattern, a violation passage pattern, and an absence pattern.

[0014] Optionally, the correlation analysis specifically involves: performing spatiotemporal overlap analysis on the trajectories of multiple individuals or vehicles, and triggering a correlation warning condition when the trajectories of two or more individuals or vehicles overlap within a preset spatiotemporal window.

[0015] Secondly, this application also provides a computer vision-assisted park security situation awareness system for executing the computer vision-assisted park security situation awareness method described in the first aspect. The computer vision-assisted park security situation awareness system includes: a data acquisition module for real-time acquisition of multi-source sensor data containing video images and audio signals within the park; a lightweight inference module for receiving the multi-source sensor data through edge computing nodes, performing lightweight inference, and forming local situation awareness information; a situation analysis module for uploading the local situation awareness information generated by each edge computing node to a cloud server, whereby the cloud server aggregates situation awareness information from the entire park area, constructs a spatiotemporal trajectory map, and performs situation analysis based on the spatiotemporal trajectory map according to a preset hierarchical early warning model; and an early warning information push module for generating early warning information and pushing it to a mobile terminal and command center when the analysis result meets the early warning triggering conditions, responding to voice intercom and announcement commands from the command center or mobile terminal, and providing voice intervention to the site through front-end deployed audio output devices.

[0016] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0017] By collecting multi-source sensor data, including video images and audio signals, in real time within the park; receiving the multi-source sensor data through edge computing nodes, performing lightweight inference, and forming local situational awareness information; uploading the local situational awareness information generated by each edge computing node to a cloud server, the cloud server aggregates situational awareness information from the entire park area, constructs a spatiotemporal trajectory map, and performs situational analysis based on the spatiotemporal trajectory map according to a preset hierarchical early warning model; when the analysis result meets the early warning triggering conditions, an early warning message is generated and pushed to the mobile terminal and the command center, responding to voice intercom and announcement commands from the command center or mobile terminal, and conducting voice intervention on site through audio output devices deployed at the front end. In other words, by integrating video images and audio signals for collaborative perception, it enables proactive 24 / 7 detection of diverse security incidents; by utilizing real-time edge inference and cloud-based cross-camera trajectory correlation, it accurately reconstructs personnel's intentions and automatically determines risk levels; at the same time, it establishes a reverse control link for front-end audio output devices, enabling the command center and on-site security personnel to promptly initiate remote voice announcements and deterrence, shifting the security defense line from post-incident tracing to timely intervention during the incident, allowing high-risk incidents to be strongly intervened and suppressed in their nascent stage, significantly reducing the emergency response window, and thus effectively enhancing the park's security capabilities.

[0018] The above description is merely an overview of the technical solution of this application. To better understand the technical means of this application and to facilitate its implementation according to the description, and to make the above and other objects, features, and advantages of this application more apparent, specific embodiments of this application are described below. It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent through the following description. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the computer vision-assisted campus security situational awareness method proposed in this application.

[0021] Figure 2 This is a schematic diagram of the structure of the computer vision-assisted campus security situation awareness system of this application.

[0022] Figure labeling: Data acquisition module 11, lightweight inference module 12, situation analysis module 13, early warning information push module 14. Detailed Implementation

[0023] This application provides a computer vision-assisted method and system for campus security situational awareness. It addresses the technical problem in existing technologies where single-point monitoring of various security elements within the campus lacks continuous characterization and in-depth understanding of cross-temporal and spatial behavior, making it difficult for security personnel to detect concealed violations and identify potential risk patterns, thus further impacting campus security. By integrating video images and audio signals for collaborative perception, it achieves 24 / 7 proactive detection of diverse security events. Utilizing real-time edge inference and cloud-based cross-camera trajectory correlation, it accurately reconstructs personnel's intentions and automatically determines risk levels. Simultaneously, it establishes a reverse control link for front-end audio output devices, enabling command centers and on-site security personnel to promptly initiate remote voice announcements and deterrence. This shifts the security defense line from post-event tracing to real-time intervention, allowing high-risk events to be strongly intervened and suppressed at their nascent stage, significantly reducing the emergency response window and effectively enhancing campus security capabilities.

[0024] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. It should be understood that this application is not limited to the exemplary embodiments described herein. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application. It should also be noted that, for ease of description, only the parts related to this application are shown in the accompanying drawings, not all of them.

[0025] Example 1, please refer to the appendix. Figure 1 This application provides a computer vision-assisted campus security situation awareness method, wherein the computer vision-assisted campus security situation awareness method is applied to a computer vision-assisted campus security situation awareness system, and the computer vision-assisted campus security situation awareness method specifically includes the following steps:

[0026] It collects multi-source sensor data, including video images and audio signals, in real time within the park.

[0027] Furthermore, this application also includes the following steps: reusing existing video surveillance cameras in the park and installing microphones at the locations of the cameras to collect multi-source sensor data containing video images and audio signals in real time.

[0028] Specifically, a register of all existing video surveillance cameras within the park was compiled, recording the brand, model, IP address, installation pole number, GPS coordinates, orientation, coverage area, and current network access status of each camera. Based on key security areas within the park, usable camera locations were selected, ensuring these locations have access to a power network and can stably output video streams. For older cameras that do not support standard protocols, video encoders were installed to convert their analog signals into digital IP streams before connecting them to the monitoring system.

[0029] For each selected camera location, technicians choose an omnidirectional microphone with appropriate sensitivity and wind and dust resistance, based on its installation height and ambient noise levels. During on-site installation, workers use stainless steel clamps or specialized mounting brackets to securely install the microphone on the same pole directly below the camera. The installation height is set at 2.5m to 3m above the ground to prevent damage from low installations and to avoid loss of ground sound details from high installations. The microphone's orientation must be carefully adjusted during installation to ensure its main pickup axis is aligned with the camera's lens optical axis, with a directional deviation controlled within 15°. This ensures that sound from the center of the camera's image is received by the microphone with maximum sensitivity.

[0030] Since the microphone is a newly added, independently installed hardware device, the weak audio signals collected are transmitted via cable to the nearest edge computing device. Construction workers laid dedicated audio transmission lines with an outer layer of metal shielding along the poles and the park's low-voltage cable trays to resist electromagnetic interference from nearby power lines and frequency converters, ensuring that the sound is not mixed with noise during transmission over distances of tens of meters. During the laying process, the cables were physically protected by being run through galvanized steel conduits, and junction boxes were installed at corners to prevent the cables from being cut by sharp angles. Simultaneously, the audio lines maintained a parallel distance of at least 30cm from the park's power supply cables, and when crossing them, they were perpendicular to each other to minimize power frequency interference.

[0031] During the system debugging phase, edge computing devices are connected to a unified time reference server within the park. The local clocks of the devices are automatically calibrated at regular intervals to ensure that the timestamps of the video streams from all cameras and the audio packets from all microphones are based on the same precise timeline. All raw video and audio data are temporarily stored in a local circular buffer for real-time inference; the raw audio and video streams are never uploaded to the cloud server, saving network bandwidth and fundamentally protecting the privacy and security of personnel within the park.

[0032] The multi-source sensor data is received by the edge computing node, lightweight inference is performed, and local situational awareness information is formed.

[0033] Furthermore, this application also includes the following steps: performing target detection on video images to extract people and vehicles from the video frame; tracking people using key points of the human skeleton to identify abnormal behavior; performing voice keyword wake-up detection and sound intensity detection on audio signals collected by the microphone to identify abnormal sounds; combining the abnormal behavior and abnormal sounds to locate abnormal targets; for abnormal targets, performing cross-camera trajectory association on the same abnormal target in video images from different cameras to generate cross-camera trajectory association data; and generating the local situational awareness information using the abnormal behavior, cross-camera trajectory association data, and abnormal sounds.

[0034] Furthermore, this application also includes the following steps: using facial recognition to identify abnormal targets in video images and extracting the appearance features of the abnormal targets; when the same abnormal person is detected in video images from different cameras, based on the similarity matching between the identification results and the appearance features, associating the spatiotemporal locations of the same abnormal person under different cameras to form the continuous movement trajectory of the corresponding abnormal person in the park; generating the cross-camera trajectory association data based on the continuous movement trajectory of each abnormal person.

[0035] Specifically, multi-source sensor information, including video images and audio signals, is received through edge computing nodes. Each frame of the video image is scanned, and the bounding boxes of all targets belonging to the categories of people and vehicles in the image are output along with their confidence scores. Each bounding box is also accompanied by a temporary number.

[0036] For each human bounding box, pose estimation is used to identify the coordinates of key points such as the head and limbs in a 2D image. Human behavior is determined by analyzing the spatial relationships and temporal changes of these points. Human skeleton keypoint tracking extracts the coordinates of several key joints from the human body area in the video frame, such as the head, left and right shoulders, left and right elbows, left and right wrists, left and right hips, left and right knees, and left and right ankles. The positional changes of these points are continuously tracked across multiple frames to determine the human posture and movement. For example, when the center point of the hip moves rapidly downward while the ankle remains relatively stationary, a fall can be identified.

[0037] Based on geometric criteria corresponding to different actions, abnormal behaviors are identified, such as climbing, falling, rapid movement, and holding dangerous objects. Climbing is indicated when the wrist keypoint continuously rises above the head, and both feet leave the ground or alternately move upwards; a fall is indicated when the vertical coordinate of the hip center keypoint rapidly drops by more than 0.8m in a very short time, and the subsequent ankle keypoint remains basically horizontal; a running state is indicated when the displacement rate of the human body center point exceeds a preset speed threshold, such as 2m / s; and the detection of an additional object contour separated from the human body contour near the wrist area, with the object's shape showing a high degree of matching with a preset dangerous object template. Once any abnormal behavior is triggered, the corresponding behavior label and trigger time are assigned to the target person.

[0038] In parallel with video stream processing, a sliding time window is maintained, such as a 2-second duration. Voice keyword wake-up detection and sound intensity detection are performed on the audio data within the window. Voice keyword wake-up detection involves real-time analysis of the sound signal collected by the microphone to identify preset specific words or phrases. Sound intensity detection measures the volume of the audio signal in real-time, calculating its sound pressure level (SPL) or full-scale digital level in decibels. When the sound intensity suddenly jumps above a certain threshold exceeding the normal ambient background noise, it is determined to be an abnormal sound event. Speech segments are segmented from the audio signal and fed into a lightweight keyword recognition model, outputting the probability of each word appearing in a pre-set security keyword list. When the probability of keywords such as "help," "fire," and "explosion" exceeds a preset threshold (e.g., 0.85), the corresponding keyword alarm is triggered. The average sound intensity of the audio signal within the current time window is calculated and dynamically compared with the average sound intensity of the ambient background noise over the previous 10 seconds. When the real-time sound intensity exceeds the background noise by a certain difference (e.g., more than 15dB) and the duration exceeds 200ms, it is determined to be an abnormal sound event, and the abnormal sound type and its timestamp are recorded.

[0039] Abnormal behavior and abnormal sound are spatiotemporally aligned and fused. Using the timestamp as the primary key, matching is performed within a ±500ms time window. If a person or target exhibits a fall at time t, and an impact sound or a cry for help is detected in the audio link within the same time window, then the target has a high anomalous confidence level. The anomalous confidence level is obtained by a weighted sum of the two. Simultaneously, the pixel coordinates of the target in the image at that time are mapped to two-dimensional coordinates in the physical space of the park, thus obtaining the precise spatial location of the abnormal target within the park. After the fusion determination is completed, a list of abnormal targets is output. Each abnormal target includes its unique identifier, spatial coordinates, abnormal behavior label, associated abnormal sound type, and corresponding unified timestamp. For example, the video link output showing user 068 falling at 09:22:33.520 is weighted and fused with the audio link output showing a cry for help and an abnormally high sound intensity at 09:22:33.900. The weights for abnormal behavior and abnormal sound are 0.6 and 0.4 respectively. The resulting comprehensive abnormal score is 0.89 × 0.6 + 0.94 × 0.4 = 0.91, exceeding the set abnormal threshold of 0.75. Therefore, user 068 is officially marked as an abnormal target. Using the camera's perspective transformation calibration matrix, the camera is pre-calibrated using the Zhang Zhengyou calibration method, with a reprojection error of 0.32 pixels. The pixel coordinates (464, 390) of user 068 at the moment of the fall are mapped to the park's physical coordinates (X = 127.3m, Y = 88.6m), achieving a positioning accuracy of ±0.35m.

[0040] For each person or vehicle target that has been marked as abnormal, initiate an identification process. If it is a person, perform person identification and appearance extraction; if it is a vehicle, perform vehicle identification and appearance extraction.

[0041] For targets marked as anomalous, edge nodes extract a facial image patch from the video frame corresponding to the moment the anomalous behavior is triggered. Based on the target's bounding box position, the patch is extracted from the upper region of the bounding box, typically from the top of the bounding box to about one-third of its height below, corresponding to the head position. This facial image patch undergoes a multi-dimensional quality assessment, including sharpness scoring, skew angle evaluation, and illumination uniformity evaluation. The Laplacian variance of the image is calculated; a score higher than a preset threshold (e.g., >15.0) indicates sharpness, while a score lower than the threshold indicates blurriness. Facial pose is determined by detecting facial key points (eyes, nose tip, and corners of the mouth, a total of 5 key points). A frontal or near-frontal view is only considered if the ratio of the horizontal offset difference between the left and right eye key points to the face width is ≤0.15 and the vertical position of the nose tip is within a reasonable range directly below the line connecting the two eyes. The standard deviation of pixel brightness distribution within the facial region is calculated; a standard deviation within a reasonable range (e.g., 20-80 gray levels) indicates suitable illumination. If any of the clarity score, deflection angle evaluation, or illumination uniformity evaluation is not met, the node will backtrack or extend several frames forward or backward, up to ±30 frames, and select the frame that is closest in time to the anomaly trigger moment and passes the quality evaluation as the final recognition image. If no usable face image is found after continuous backtracking and extension, the face recognition step is skipped, and subsequent cross-camera tracking is performed solely based on appearance features, but the identity information is marked as unknown.

[0042] For face images that pass the quality assessment, the edge nodes normalize their size to a uniform size, such as 112×112 pixels, and feed them into a lightweight face recognition model for forward inference, outputting a 128-dimensional face feature vector. Each dimension is a 32-bit floating-point number, normalized using the L2 norm. This 128-dimensional feature vector is then compared 1:N with a pre-distributed campus personnel feature database containing facial feature templates of all employees and students. The cosine similarity between the query vector and each template in the database is calculated. The result with the highest similarity is selected as the candidate match. If the highest similarity value exceeds a preset identity verification threshold, such as 0.75, the match is considered successful, and the candidate's identity information, including name, student ID / employee ID, and college / position, is output. If the highest similarity is below the threshold, the match fails, and the identity information is marked as unrecognized.

[0043] Regardless of whether face recognition is successful or not, the appearance feature extraction pipeline is executed independently. The image of the abnormal target within the bounding box of the trigger frame is used as input and fed into the global appearance feature extraction model. This model is typically pre-trained on a large-scale person re-identification dataset and outputs a 512-dimensional feature vector. This 512-dimensional vector is a digital encoding of the person's visual appearance. Clustering algorithms are used to extract the dominant color and its RGB values ​​within the human body region. For example, the dominant color of the upper body (RGB(195,42,38)) is red, accounting for 67%; the dominant color of the lower body (RGB(38,42,58)) is dark blue, accounting for 82%. The width-to-height ratio of the human body bounding box and the ratio of shoulder to hip width are encoded into several continuous numerical dimensions. If the person walks in multiple consecutive frames, gait swing period features are extracted, i.e., the frequency and amplitude of alternating left and right leg movements. The final 512-dimensional feature vector, after L2 norm normalization, serves as the visual fingerprint of the abnormal person and is stored along with the identity recognition result in the target's temporary data record.

[0044] For vehicles flagged as anomalous, the license plate area image is extracted based on the vehicle's bounding box location, typically at the lower edge of the front or rear of the vehicle. This image is then fed into a lightweight license plate detection and optical character recognition cascade model, outputting a license plate number string. Simultaneously, the complete vehicle image within the bounding box is fed into a vehicle appearance feature extraction model. This model is a publicly available model trained on a large-scale vehicle re-identification dataset, outputting a 512-dimensional appearance feature vector. The encoded content includes the vehicle's primary color tone and its RGB values; vehicle shape descriptions such as aspect ratio and contour corner features, distinguishing between sedans, SUVs, MPVs, trucks, etc.; and vehicle brand-related visual features such as headlight shape and grille texture. The license plate number string and the 512-dimensional appearance feature vector are stored together in a temporary data record for the anomalous vehicle.

[0045] Each successfully extracted abnormal target record is stored, and a cache expiration period is set. When an abnormal target is detected in the view of a camera under the jurisdiction of the current node, the target's appearance feature vector is used as the query vector to perform a similarity search in the appearance feature cache shared by adjacent nodes. If a historical record with an appearance similarity exceeding a preset threshold is found (e.g., cosine distance exceeds 0.82) and the timing is reasonable (i.e., the later appearance time is later than the earlier appearance time), it is determined that the same target has reappeared under the adjacent camera. The location coordinates of this appearance are arranged in chronological order with the location coordinates of the previous appearances to form a continuous trajectory point sequence for the abnormal target. When the same abnormal target appears at more than 3 different camera locations, its trajectory point sequence is sufficient to support path fitting. Each time a new trajectory point is added, the node updates the cross-camera trajectory association data of that target locally.

[0046] At a fixed reporting cycle or when a new anomaly is detected, the processing results of all anomalies within the current window are packaged together. This package includes the anomaly's unique identifier and identification result, anomaly behavior tag and timestamp of the behavior occurrence, the target's current spatial physical coordinates, the target's cross-camera trajectory point sequence over a past period, and the type and intensity data of the anomalous sound associated with the current anomaly. The local situational awareness information is the final output structured data packet, which includes anomaly behavior tags and their corresponding timestamps and locations; the anomaly's continuous cross-camera trajectory; and the type and time of the anomalous sound event. This data is uploaded to the cloud server in structured text format.

[0047] The local situational awareness information generated by each edge computing node is uploaded to the cloud server. The cloud server aggregates the situational awareness information of the entire park area and constructs a spatiotemporal trajectory map. Based on the spatiotemporal trajectory map, situational assessment is performed according to a preset hierarchical early warning model.

[0048] Furthermore, this application also includes the following steps: the cloud server receives local situational awareness information uploaded by each edge computing node, merges and deduplicates the location information of the same abnormal target reported by different edge nodes within adjacent time windows and adjacent spatial areas to form complete spatiotemporal trajectory data of each abnormal target within the park; using the park's electronic map as a base map, the complete spatiotemporal trajectory data of each abnormal target is superimposed on the base map to construct the spatiotemporal trajectory map; for each person or vehicle in the spatiotemporal trajectory map, single-point rule analysis, time-series pattern analysis, and correlation analysis are performed in parallel, and when any analysis level triggers an early warning condition, the corresponding early warning type and early warning level are determined, and the early warning information is generated.

[0049] Furthermore, this application also includes the following steps: the spatiotemporal trajectory map includes the location coordinates, dwell time, direction of travel, and sequence of points passed by each person and vehicle at each time point in the park.

[0050] Furthermore, this application also includes the following steps: the single-point rule analysis specifically involves: determining the spatial relationship between the current location of a person or vehicle and a preset multi-type spatial area; when the location of a person or vehicle falls into any preset spatial area, the corresponding type of single-point warning condition is triggered; wherein, the preset multi-type spatial area includes at least a restricted area, a key monitoring area, and a temporary control area.

[0051] Furthermore, this application also includes the following steps: the timing pattern analysis specifically involves: performing timing analysis on the trajectory sequence of personnel or vehicles within a preset time window, and triggering a timing warning condition when the trajectory sequence matches a preset abnormal timing pattern; the preset abnormal timing pattern includes at least an abnormal occurrence pattern, an abnormal stay pattern, an overtime stay pattern, a violation passage pattern, and an absence pattern.

[0052] Furthermore, this application also includes the following steps: the correlation analysis specifically involves: performing spatiotemporal overlap analysis on the trajectories of multiple personnel or vehicles, and triggering a correlation warning condition when the trajectories of two or more personnel or vehicles overlap within a preset spatiotemporal window.

[0053] Specifically, the cloud server continuously monitors the standard network data receiving port, receiving local situational information data packets uploaded by all edge computing nodes within the park at fixed reporting intervals. Each data packet is in structured text format, containing a list of all targets marked as abnormal within the coverage area of ​​the cameras under the node's jurisdiction. Each abnormal target includes a unique temporary target number, category (personnel / vehicle), identification result, abnormal behavior tag, current spatial physical coordinates, a sequence of local trajectory points over a recent period, and associated abnormal sound type. The cloud server parses each received data packet, extracts the abnormal target information, and maintains a global record in the cloud for each target.

[0054] Since the same anomalous target may be reported by multiple edge computing nodes during its movement, the cloud server needs to identify which records belong to the same target from the massive number of reported records and merge them into the same global trajectory. For each newly received anomalous target record, its appearance feature vector is extracted and compared with the appearance feature vectors of all active anomalous targets currently maintained by the cloud server. If the appearance similarity between an existing active target and the newly reported target exceeds a preset threshold, such as 0.82, and the two are in a reasonable time order (i.e., the timestamp of the new record is later than the timestamp of the latest trajectory point of the existing target), it is determined to be a new trajectory point of the same target. The new location record is then appended to the end of the target's existing trajectory sequence.

[0055] In actual operation, different edge nodes may generate multiple duplicate or highly redundant records for the same target at similar times and locations due to overlapping camera coverage areas and interleaved reporting times. Before appending new records to the target trajectory, the cloud server performs a merging and deduplication algorithm. Specifically, for the trajectory sequence of an abnormal target, the most recently received trajectory points are examined. Multiple points meeting the conditions of a time difference less than the threshold of an adjacent time window (e.g., 5 seconds) and a spatial distance less than the threshold of an adjacent spatial region (e.g., 50 meters) are identified as duplicate records. Only the two points with the earliest and latest timestamps are retained, serving as markers for entering and leaving the overlapping region, respectively. Intermediate redundant points are discarded. If multiple edge nodes report almost identical location coordinates of the target near the same time, the point with coordinates closest to the average of all reported values ​​is retained, and the rest are discarded. After this merging and deduplication process, the trajectory point sequence left by each abnormal target is evenly distributed in the spatiotemporal dimension, with no redundant repetitions.

[0056] After merging and deduplication, a complete trajectory sequence is maintained for each active anomalous target. Each trajectory point in the sequence includes physical coordinates (X-value, Y-value), a precise timestamp, and the source camera number. As the target continues to move within the park and is captured and reported by more edge nodes, the cloud server continuously appends new trajectory points to the end of the sequence, forming a spatiotemporal trajectory line that extends over real time. When no new reporting points appear for a target within a considerable period, the cloud server marks the target as disappeared, and no new points are added to its trajectory sequence.

[0057] For each active, abnormal target's complete trajectory sequence, the cloud server automatically calculates the associated attribute information of each trajectory point after each update. Starting from the first point in the trajectory sequence, consecutive trajectory points are scanned. If the spatial distance between multiple consecutive points is less than the set dwell radius (e.g., 5m), and the time span of these points exceeds the set dwell time threshold (e.g., 30s), these consecutive points are grouped into a dwell segment. The dwell time is equal to the timestamp of the last point minus the timestamp of the first point. All points within a dwell segment are marked with the same dwell time value, while the dwell time of points outside the dwell segment is recorded as 0. For each trajectory point in the trajectory sequence except the last point, the positional difference between it and the next trajectory point is taken, and the azimuth angle from the current point to the next point is calculated mathematically, with true north as 0°, rotating clockwise. The azimuth angle is the instantaneous direction of travel at that trajectory point. Since the last trajectory point has no subsequent points, its direction of travel is marked as unknown. The cloud server pre-marks all key locations with clear geographical semantics on the park's electronic map, such as the installation location of each camera, each entrance / exit, each intersection, and the gate of each building. Each key location's name and physical coordinates are recorded. For each trajectory point in the trajectory sequence, the straight-line distance between that point and all pre-marked key locations is calculated. The key location closest to it and smaller than the associated judgment radius (e.g., 3m) is taken as the point passed by that trajectory point. All key locations associated with each trajectory point are arranged in chronological order, and consecutively repeated point names are merged to ultimately form the sequence of points passed by the abnormal target.

[0058] Using a high-precision electronic map of the park as a base map, the complete spatiotemporal trajectory data of each anomalous target is overlaid and rendered onto the base map to form a visualized spatiotemporal trajectory map. Different colored curves are used to draw the trajectory lines of different anomalous targets, with colors dynamically assigned according to the target's risk level (e.g., red for high risk, orange for medium risk, and yellow for low risk). An arrow symbol indicates the direction of travel at each trajectory point, pointing to the calculated azimuth angle at that point. A semi-transparent circular heat map marks the location of a loitering area; the radius of the circle is positively correlated with the loitering duration—the longer the loitering, the larger the circle. The start and end points of the trajectory lines are marked with green and red dots, respectively. Key information about the target, including its number, identity, and type of anomalous behavior, is labeled next to the trajectory lines. The final spatiotemporal trajectory map can be pushed to the command center's large screen for full-screen display, allowing on-duty personnel to intuitively grasp the real-time situational distribution and historical movement patterns of anomalous targets throughout the park.

[0059] At each fixed analysis cycle, such as every 2 seconds, or when a new trajectory point is added to the trajectory of an abnormal target, the complete trajectory data and associated attributes of the target are read from the spatiotemporal trajectory map database, including the coordinates of each point, timestamp, dwell time, direction of travel, and sequence of points passed. The system loads the rule configuration parameters required for single-point rule analysis, time-series pattern analysis, and correlation analysis, including the spatial region polygon coordinate data required for single-point rules, such as the list of polygon vertices for restricted areas, key monitoring areas, and temporary control areas; various threshold parameters required for time-series patterns, such as the start and end times of restricted periods, maximum dwell time at each point, required departure time, start and end times of attendance periods, and a list of restricted areas; and spatiotemporal window parameters required for correlation analysis, such as proximity distance threshold, co-occurrence duration threshold, and analysis time window length.

[0060] Single-point rule analysis is a real-time judgment logic based on spatial location. It focuses only on the current location of an abnormal target, regardless of its historical trajectory or future trend. It checks the real-time location coordinates of the target against various spatial regions pre-mapped on an electronic map to determine their point-polygon inclusion relationships. Once a target falls into a region, a corresponding single-point alert is triggered. For each abnormal target's latest location—the coordinates of the last trajectory point in the spatiotemporal trajectory map—it performs mathematical determination of point-polygon inclusion relationships against all preset spatial regions. Traverse the polygon boundaries of all restricted areas and determine whether the current point is inside any of the restricted area polygons. Restricted areas on campus include, but are not limited to, laboratory hazardous chemical storage rooms and surrounding buffer zones, power distribution rooms, data center computer rooms, finance offices, exam paper security rooms, broadcasting station control rooms, and other closed management areas involving core campus security or major property security. If so, immediately trigger a single-point warning for intrusion into a restricted area. The warning level is determined according to the preset security level of the specific area intruded into. For example, the laboratory hazardous chemical storage area is red (highest level), the power distribution room and data center computer room are orange (second highest level), and the exam paper security room is temporarily upgraded to red during the examination period. If no entry warning is triggered, the system continues to iterate through all key monitoring areas to determine if the current point falls within any of them. Key monitoring areas on campus include, but are not limited to, the school gate and its surrounding buffer zone, playground, canteen, library entrances and exits, dormitory lobbies, teaching building corridors and stairwells, stadium, and intersections of main campus roads—places with high population density or important activities. If so, a single-point warning for entering a key monitoring area is triggered. Instead of directly generating an alarm push, the target is added to the key monitoring list, increasing its trajectory recording priority and trajectory point reporting frequency, such as from every 2 seconds to every 0.5 seconds. If no warning is triggered, the system continues to iterate through all currently valid temporary control areas. Temporary control areas on campus include, but are not limited to, closed areas around examination rooms during examinations, construction area enclosures on campus, temporary control areas during large-scale events, and temporary isolation areas designated during emergencies. The system reads the effective time window of the temporary control areas in real time and only judges areas that are currently within their validity period. If the current point falls within an area, a single-point warning for entering a temporary control area is triggered, with the warning level equivalent to the level specified when the corresponding temporary control was set up. The severity of the situation is preset by the administrator. After the triple spatial judgment is completed, if no area is hit, the single-point rule judgment result for the target is normal.

[0061] Temporal pattern analysis is a behavioral pattern analysis logic based on the time dimension, focusing on what anomaly targets do over a period of time, rather than just their instantaneous location. By matching the target's trajectory sequence with multiple preset abnormal behavior patterns, it identifies hidden risks that are not detectable in single-point spatial judgment and require time to accumulate before becoming apparent. It reads the restricted access time configuration corresponding to the current date, such as 23:00 to 05:00 the next day on weekdays, or all day on weekends. It iterates through all trajectory points in the anomaly target's trajectory sequence located near the entrances and exits of closed areas, checking whether the timestamps of these trajectory points fall within the restricted access time. If students or off-campus personnel appear at campus entrances and exits, around the perimeter of walls, or in non-open areas during nighttime closed hours; or if outsiders appear around the girls' dormitory building, the restricted area of ​​the laboratory building, etc., during non-open hours, an anomaly occurrence pattern warning is triggered. It iterates through the dwell time attribute value of each trajectory point in the anomaly target's trajectory sequence, comparing it with the preset maximum allowed dwell time threshold for the area where that point is located. The preset maximum allowed dwell time threshold is usually set by relevant personnel based on the importance of the area. If the dwell time of a trajectory point exceeds the preset maximum allowed dwell time threshold, such as a student lingering at the library entrance for more than 1 hour without entering, someone loitering in the laboratory corridor for more than 20 minutes without entering any room, or a food delivery rider staying at the school gate for more than 15 minutes without delivering, an abnormal dwelling mode warning is triggered. The warning level is positively correlated with the multiple of the dwelling timeout; the longer the timeout, the higher the level. The system reads the configured departure time slot for the current date. For example, if the designated late return time for students in the dormitory area is 22:00, it checks whether the target trajectory sequence contains trajectory points with timestamps later than the departure time, and whether the number of these trajectory points later than the departure time exceeds the preset minimum detection threshold (e.g., at least 3 trajectory points at different times), proving that the student was indeed active within the campus and not just due to a false alarm. If the above conditions are met, an overtime dwelling mode warning is triggered. Obtain the category label of the target, i.e., personnel or vehicle. If it is a vehicle, read the coordinates of each trajectory point in the vehicle's trajectory sequence and determine the point-polygon containment relationship with a preset list of restricted areas. If a trajectory point falls inside a restricted area, a violation warning is triggered. At the start of a preset attendance period, initiate a timed check task to check whether each person who should be present during that period has been reported by at least one edge node. If a person has no trajectory point recorded during that period, an absence warning is triggered, such as if a student is not captured by the school building area cameras during the designated class attendance period (e.g., 8:00 AM to 12:00 PM). Summarize and record the warning types and levels corresponding to all triggered modes in the target's time-series analysis results.

[0062] Association analysis is a judgment logic based on group behavior analysis. It does not focus on the absolute position or temporal behavior of a single target, but rather on the relative spatial positional relationships and temporal synchronicity between multiple targets. By calculating the degree of overlap in the spatiotemporal dimensions of multiple abnormal target trajectories, it identifies potentially risky behaviors that require multiple people to coordinate. For example, three or more people converging from different directions in a short period of time in areas such as playgrounds, squares, or rooftops may indicate a premeditated group incident; an off-campus individual following a student at a fixed distance (5m to 15m) under multiple surveillance cameras may indicate stalking; multiple people entering from different gates and then converging in the same secluded location (such as the rooftop of the laboratory building or behind the gymnasium) may indicate a premeditated violation of rules or laws; multiple people appearing at different locations around the perimeter of the school at the same time at night may indicate a group outing. All currently active abnormal targets are traversed in pairs, and the degree of spatiotemporal overlap in the trajectories between each pair is calculated. For example, taking the trajectory sequences of targets A and B within a preset analysis time window, such as the most recent 5 minutes, and aligning them on the time axis, the Euclidean distance between A and B is calculated for each identical or close time point. If the spatial distance between them at a certain moment is less than a preset proximity threshold, such as 3 meters, a close contact event is recorded. If the cumulative number of close contact events between A and B within the same time window exceeds a preset number threshold, such as 3 times, or the duration of a single close contact exceeds a preset duration threshold, such as 5 minutes, a significant spatiotemporal correlation between A and B is determined, triggering a correlation warning. The warning includes the target identifiers of both parties and the time point at which the correlation was first discovered. Furthermore, if the trajectories of A and B are highly synchronized in time, meaning that the spatial distance between them remains small at multiple time points and their movement directions are consistent, even if the duration of a single contact does not reach the threshold, but there is a sustained accompanying movement for a relatively long period, such as more than 2 minutes, it is also determined to be a correlation, triggering a cooperative accompaniment subtype warning.

[0063] The outputs of single-point rule analysis, time-series pattern analysis, and correlation analysis are summarized to form a comprehensive situational assessment conclusion for the abnormal target within the current period. If none of the three threads trigger any alerts, the target's status is marked as normal, and monitoring continues without generating alert pushes. If any thread triggers an alert, the base level of this alert is determined according to a preset risk level mapping table. If multiple threads trigger simultaneously, the alert levels are cumulatively increased, such as single-point trigger (yellow) + time-series trigger (yellow) = orange, and single-point trigger (orange) + time-series trigger (orange) = red. The final alert information includes the alert type, specific pattern name, alert level, trigger time, and associated target information. This alert information is written to the cloud alert record database and simultaneously pushed to the notification module for distribution.

[0064] The system offers comprehensive analysis dimensions, covering not only location but also behavior and relationships, effectively identifying hidden risks that traditional security systems cannot detect, such as prolonged stays and multi-person conspiracy. By defining absence and violation access patterns, it achieves data-level integration between the security system and campus attendance and daily access management processes, enabling a single video surveillance system to simultaneously serve the dual functions of security and daily management, thus improving the precision of campus management. Utilizing a streaming computing framework for parallel processing of the three types of analysis ensures that even with tens of thousands of concurrent targets, the warning delay remains within milliseconds. Multi-dimensional cross-validation significantly reduces the false alarm rate; for example, alarms are only triggered when both location and time are abnormal, allowing security personnel to focus on truly high-risk events and shifting campus security from passive defense to proactive prediction.

[0065] When the assessment results meet the early warning trigger conditions, an early warning message is generated and pushed to the mobile terminal and the command center. In response to the voice intercom and announcement instructions from the command center or the mobile terminal, voice intervention is carried out on site through the audio output device deployed at the front end.

[0066] Specifically, after completing the analysis, the cloud server determines the overall situation of each abnormal target. If any analysis thread outputs a warning and its level reaches yellow or above, it is considered to meet the warning triggering conditions. The system reads complete information about the target, including identification results, real-time spatial coordinates, abnormal behavior tags, and trajectory sequences within the last 60 seconds. It retrieves keyframes from the corresponding edge nodes at the time of the anomaly trigger, assembles all data into a structured warning information package according to a preset push data template, assigns a unique number to the warning, and writes it to the cloud warning record database.

[0067] Through the park's internal LAN, using standard network communication protocols, early warning information is transmitted in real time to the command center's visualization platform. Upon receiving the early warning information, the command center's large screen immediately triggers strong visual and auditory cues, such as highlighting and flashing the target location on the map and playing warning audio in different tones and repetition frequencies. Via the internet, mobile push services are used to distribute the early warning information to the mobile applications of all online security personnel in the park. Upon receiving the push notification, the mobile device reads a summary of the warning content aloud using synthesized speech, such as "Orange Warning: Unknown personnel overstaying their permitted time at the East Gate." The two push channels operate in parallel without blocking each other; if a push fails, an automatic retry mechanism is implemented, with a maximum of three retries at 2-second intervals to ensure that the early warning information reaches at least one valid terminal.

[0068] While pushing out early warning information, the cloud server calculates the Euclidean distance between the target's location and all online audio devices within the park based on the target's real-time physical coordinates. It then filters out the closest devices to the target. If multiple devices are close, the top three closest are selected as candidates, and the one with the most recent online status is chosen from these. If the nearest found device exceeds the preset maximum effective broadcast radius (e.g., 500m), it is determined that there are no available audio output devices nearby. A log indicating no available broadcast device is recorded, and the voice intervention step is skipped; only the push notification is executed. If the search is successful, the device's unique identifier, IP address, volume, and channel configuration parameters are written to the pending command queue.

[0069] After receiving a warning notification on the large screen, the command center staff checks the details. If remote voice intervention is needed for the abnormal target, they click the "Speak" button on the interface. A start playback command is sent to the target's audio output device, containing the audio stream's receiving port number, decoding parameters, and playback volume percentage. Upon receiving the command, the audio output device immediately opens the corresponding network port to receive the audio stream. After internal decoding and power amplification, the stream is amplified and played back into the surrounding space through speaker units.

[0070] Once the audio output device starts playing the announcement, it continuously collects and transmits real-time audio streams, enabling real-time two-way communication where the speaker can listen simultaneously. When the command center turns off its microphone, it sends a stop playback command to the audio output device via the signaling channel. The device immediately terminates audio reception and playback, returning to standby mode.

[0071] If multiple audio output devices are available for the same warning target, multiple adjacent devices will be selected for synchronous playback according to the preset configuration. When multiple devices are synchronized, the cloud will copy and distribute the audio stream to each device, and add a small delay compensation to eliminate echo. The delay compensation value is calculated based on the distance difference between each device and the target, and is usually 3ms / m. During the voice intervention process, the cloud server records the entire voice operation log.

[0072] Through parallel dual-push notifications on the command center's large screen and mobile devices, early warning information is ensured to reach both fixed duty posts and mobile patrol posts within one second. Even if one channel is temporarily interrupted due to network failure, the other channel can still independently complete the early warning delivery, achieving redundancy backup of the push link. The cloud automatically retrieves and selects the nearest available audio output device for broadcasting based on the real-time physical coordinates of the abnormal target, avoiding ineffective intervention by broadcasting in empty areas. It automatically avoids offline or faulty devices, ensuring that the target device will respond and play the message after it is sent.

[0073] In summary, the computer vision-assisted campus security situational awareness method provided in this application has the following technical effects: It collects multi-source sensor data, including video images and audio signals, within the campus in real time; it receives the multi-source sensor data through edge computing nodes, performs lightweight inference, and forms local situational awareness information; it uploads the local situational awareness information generated by each edge computing node to a cloud server, which aggregates situational awareness information from the entire campus area, constructs a spatiotemporal trajectory map, and performs situational analysis based on the spatiotemporal trajectory map according to a preset hierarchical early warning model; when the analysis result meets the early warning triggering conditions, it generates early warning information and pushes it to the mobile terminal and command center, responding to voice intercom and announcement commands from the command center or mobile terminal, and using audio output devices deployed at the front end to provide voice intervention on-site. In other words, by integrating video images and audio signals for collaborative perception, it enables proactive 24 / 7 detection of diverse security incidents; by utilizing real-time edge inference and cloud-based cross-camera trajectory correlation, it accurately reconstructs personnel's intentions and automatically determines risk levels; at the same time, it establishes a reverse control link for front-end audio output devices, enabling the command center and on-site security personnel to promptly initiate remote voice announcements and deterrence, shifting the security defense line from post-incident tracing to timely intervention during the incident, allowing high-risk incidents to be strongly intervened and suppressed in their nascent stage, significantly reducing the emergency response window, and thus effectively enhancing the park's security capabilities.

[0074] Example 2: Based on the same inventive concept as the computer vision-assisted campus security situation awareness method in Example 1, this application also provides a computer vision-assisted campus security situation awareness system. Please refer to the appendix. Figure 2 The computer vision-assisted campus security situation awareness system includes:

[0075] The data acquisition module 11 is used to collect multi-source sensor data, including video images and audio signals, within the park in real time; the lightweight inference module 12 is used to receive the multi-source sensor data through edge computing nodes, perform lightweight inference, and form local situational awareness information; the situational assessment module 13 is used to upload the local situational awareness information generated by each edge computing node to the cloud server, the cloud server aggregates the situational awareness information of the entire park area, constructs a spatiotemporal trajectory map, and performs situational assessment based on the spatiotemporal trajectory map according to a preset hierarchical early warning model; the early warning information push module 14 is used to generate early warning information and push it to the mobile terminal and the command center when the assessment result reaches the early warning triggering condition, responding to voice intercom and shouting commands from the command center or mobile terminal, and conducting voice intervention on site through the audio output device deployed at the front end.

[0076] Furthermore, the data acquisition module 11 in the computer vision-assisted park security situation awareness system is also used to: reuse existing video surveillance cameras in the park and install microphones at the locations of the cameras to collect multi-source sensor data containing video images and audio signals in real time.

[0077] Furthermore, the lightweight inference module 12 in the computer vision-assisted campus security situation awareness system is also used for: target detection of video images to extract personnel and vehicles from the video frame; tracking of personnel using human skeleton key points to identify abnormal behavior; voice keyword wake-up detection and sound intensity detection of audio signals collected by the microphone to identify abnormal sounds; locating abnormal targets by combining the abnormal behavior and abnormal sounds; for abnormal targets, performing cross-camera trajectory association on the same abnormal target in video images from different cameras to generate cross-camera trajectory association data; and generating the local situation awareness information using the abnormal behavior, cross-camera trajectory association data, and abnormal sounds.

[0078] Furthermore, the lightweight inference module 12 in the computer vision-assisted park security situation awareness system is also used to: identify abnormal targets in video images using facial recognition and extract the appearance features of the abnormal targets; when the same abnormal person is detected in video images from different cameras, based on the similarity matching between the identification results and the appearance features, associate the spatiotemporal locations of the same abnormal person under different cameras to form the continuous movement trajectory of the corresponding abnormal person in the park; and generate the cross-camera trajectory association data based on the continuous movement trajectory of each abnormal person.

[0079] Furthermore, the situation assessment module 13 in the computer vision-assisted park security situation awareness system is also used for: receiving local situation awareness information uploaded by each edge computing node from the cloud server, merging and deduplicating the location information of the same abnormal target reported by different edge nodes within adjacent time windows and adjacent spatial areas to form complete spatiotemporal trajectory data of each abnormal target in the park; using the park's electronic map as the base map, overlaying the complete spatiotemporal trajectory data of each abnormal target onto the base map to construct the spatiotemporal trajectory map; for each person or vehicle in the spatiotemporal trajectory map, performing single-point rule assessment, time-series pattern assessment, and correlation assessment in parallel, and determining the corresponding warning type and warning level when any assessment level triggers the warning condition, and generating the warning information.

[0080] Furthermore, the situation assessment module 13 in the computer vision-assisted park security situation awareness system is also used for: the spatiotemporal trajectory map includes the location coordinates, dwell time, direction of travel, and sequence of points passed by each person and vehicle at each time point in the park.

[0081] Furthermore, the situation assessment module 13 in the computer vision-assisted park security situation awareness system is also used for: single-point rule assessment specifically: determining the spatial relationship between the current location of personnel or vehicles and preset multi-type spatial areas, and triggering the corresponding type of single-point early warning condition when the location of personnel or vehicles falls into any preset spatial area; wherein, the preset multi-type spatial areas include at least prohibited areas, key monitoring areas and temporary control areas.

[0082] Furthermore, the situation analysis module 13 in the computer vision-assisted park security situation awareness system is also used for: time sequence pattern analysis specifically: performing time sequence analysis on the trajectory sequence of personnel or vehicles within a preset time window, and triggering a time sequence warning condition when the trajectory sequence meets a preset abnormal time sequence pattern; the preset abnormal time sequence pattern includes at least an abnormal occurrence pattern, an abnormal stay pattern, an overstay pattern, a violation passage pattern, and an absence pattern.

[0083] Furthermore, the situation assessment module 13 in the computer vision-assisted park security situation awareness system is also used for: correlation assessment specifically: spatiotemporal overlap analysis of the trajectories of multiple personnel or vehicles, and triggering correlation warning conditions when the trajectories of two or more personnel or vehicles overlap within a preset spatiotemporal window.

[0084] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The computer vision-assisted campus security situation awareness method and specific examples in the aforementioned Embodiment 1 are also applicable to the computer vision-assisted campus security situation awareness system of this embodiment. Through the foregoing detailed description of the computer vision-assisted campus security situation awareness method, those skilled in the art can clearly understand the computer vision-assisted campus security situation awareness system of this embodiment. Therefore, for the sake of brevity, it will not be described in detail here.

[0085] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0086] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of this application and its equivalents, this application also intends to include such modifications and variations.

Claims

1. A computer vision-assisted method for campus security situation awareness, characterized in that, include: Real-time acquisition of multi-source sensor data, including video images and audio signals, within the park; The multi-source sensor data is received by the edge computing node, lightweight inference is performed, and local situational awareness information is formed. Local situational awareness information generated by each edge computing node is uploaded to the cloud server. The cloud server aggregates situational awareness information from the entire park area and constructs a spatiotemporal trajectory map. Based on the spatiotemporal trajectory map, situational assessment is performed according to a preset hierarchical early warning model. When the assessment results meet the early warning trigger conditions, an early warning message is generated and pushed to the mobile terminal and the command center. In response to the voice intercom and announcement instructions from the command center or the mobile terminal, voice intervention is carried out on site through the audio output device deployed at the front end.

2. The computer vision-assisted campus security situation awareness method as described in claim 1, characterized in that, Real-time acquisition of multi-source sensor data within the park, including video images, audio signals, and thermal imaging data, including: The existing video surveillance cameras in the park are reused, and microphones are added at the locations of the cameras to collect multi-source sensor data, including video images and audio signals, in real time.

3. The computer vision-assisted campus security situation awareness method as described in claim 1, characterized in that, The multi-source sensor data is received through edge computing nodes, lightweight inference is performed, and local situational awareness information is formed, including: Perform target detection on video images to extract people and vehicles from the video frame; Using key points of the human skeleton to track personnel and identify abnormal behavior; The audio signal collected by the microphone is subjected to voice keyword wake-up detection and sound intensity detection to identify abnormal sounds; Based on the aforementioned abnormal behavior and sounds, the abnormal target can be located; For abnormal targets, cross-camera trajectory correlation is performed on the same abnormal target in video images from different cameras to generate cross-camera trajectory correlation data; The local situational awareness information is generated using the abnormal behavior, cross-camera trajectory correlation data, and abnormal sound.

4. The computer vision-assisted campus security situation awareness method as described in claim 3, characterized in that, For anomalous targets, cross-camera trajectory correlation is performed on the same anomalous target in video images from different cameras to generate cross-camera trajectory correlation data, including: Facial recognition is used to identify abnormal targets in video images and extract their appearance features. When the same abnormal person is detected in video images from different cameras, the spatiotemporal locations of the same abnormal person under different cameras are associated based on the similarity matching between the identity recognition results and appearance features, forming the continuous movement trajectory of the corresponding abnormal person in the park. The cross-camera trajectory association data is generated from the continuous movement trajectory of each abnormal person.

5. The computer vision-assisted campus security situation awareness method as described in claim 1, characterized in that, Local situational awareness information generated by each edge computing node is uploaded to a cloud server. The cloud server aggregates situational awareness information from the entire park area, constructs a spatiotemporal trajectory map, and performs situational assessment based on the spatiotemporal trajectory map according to a preset hierarchical early warning model, including: The cloud server receives local situational awareness information uploaded by each edge computing node, merges and deduplicates the location information of the same abnormal target reported by different edge nodes within adjacent time windows and adjacent spatial areas, and forms complete spatiotemporal trajectory data of each abnormal target within the park. Using the park's electronic map as a base map, the complete spatiotemporal trajectory data of each abnormal target are overlaid onto the base map to construct the spatiotemporal trajectory map. For each person or vehicle in the spatiotemporal trajectory map, single-point rule analysis, temporal pattern analysis, and correlation analysis are performed in parallel. When any analysis level triggers an early warning condition, the corresponding early warning type and level are determined, and the early warning information is generated.

6. The campus security situation awareness method based on computer vision assistance as described in claim 5, characterized in that, The spatiotemporal trajectory map includes the location coordinates, dwell time, direction of travel, and sequence of points passed by each person and vehicle at each time point in the park.

7. The computer vision-assisted campus security situation awareness method as described in claim 5, characterized in that, The specific rules for single-point rule analysis are as follows: The spatial relationship between the current location of a person or vehicle and multiple preset spatial areas is determined. When the location of a person or vehicle falls into any preset spatial area, the corresponding type of single-point warning condition is triggered. The preset multiple spatial areas include at least restricted areas, key monitoring areas, and temporary control areas.

8. The computer vision-assisted campus security situation awareness method as described in claim 5, characterized in that, The specific steps of time series pattern analysis are as follows: Perform time-series analysis on the trajectory sequences of personnel or vehicles within a preset time window, and trigger time-series warning conditions when the trajectory sequence meets the preset abnormal time-series pattern; The preset abnormal timing patterns include at least the abnormal occurrence pattern, abnormal stay pattern, timeout stay pattern, illegal passage pattern, and absence pattern.

9. The computer vision-assisted campus security situation awareness method as described in claim 5, characterized in that, The specific steps for determining the relationship are as follows: Spatiotemporal overlap analysis is performed on the trajectories of multiple people or vehicles. When the trajectories of two or more people or vehicles overlap within a preset spatiotemporal window, an associated early warning condition is triggered.

10. A campus security situation awareness system based on computer vision assistance, characterized in that, The steps for implementing the computer vision-assisted campus security situation awareness method according to any one of claims 1 to 9, wherein the computer vision-assisted campus security situation awareness system comprises: The data acquisition module is used to collect multi-source sensor data, including video images and audio signals, in real time within the park; The lightweight inference module is used to receive the multi-source sensor data through the edge computing node, perform lightweight inference, and form local situational awareness information; The situation assessment module is used to upload the local situation awareness information generated by each edge computing node to the cloud server. The cloud server aggregates the situation awareness information of the entire park area, constructs a spatiotemporal trajectory map, and performs situation assessment based on the spatiotemporal trajectory map according to the preset hierarchical early warning model. The early warning information push module is used to generate early warning information and push it to the mobile terminal and the command center when the judgment results meet the early warning trigger conditions. It responds to the voice intercom and shouting instructions of the command center or mobile terminal and conducts voice intervention on site through the audio output device deployed at the front end.

Images