A campus safety early warning method based on multi-modal fusion and AI vision

Abstract

The application discloses a kind of campus safety early warning method based on multi-modal fusion and AI vision, it is related to computer vision and intelligent security technology field, comprising the following steps: S1, access control passage perception dataset is constructed;S2, based on access control passage perception dataset identification passage object entering order and passage behavior characteristics;S3, facing relationship expression, in combination with gate occupancy state and authorized behavior record, matching relationship discrimination is carried out;S4, according to gate occupancy state and accompanying behavior characteristics, risk-driven evaluation is carried out;S5, according to risk response condition, linkage control behavior is executed and passage process record and strategy adjustment are completed.The problem that the same direction accompanying borrow line under the passage peak is not associated with the opening and closing rhythm of access control, leading to the problem that the trailing mixed in is difficult to identify in advance is solved.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent security technology, specifically a campus security early warning method based on multimodal fusion and AI vision. Background Technology

[0002] With the development of image detection technology, deep learning-based target detection algorithms have been widely used in video surveillance, intelligent security, and personnel behavior recognition. Image detection, by locating and classifying targets in video frames, enables the automatic perception of people, objects, and their motion states. It has gradually evolved from single-frame detection to multi-frame correlation and temporal analysis, and has formed a foundation for routine applications in scenarios such as campus entrance and exit management and access control monitoring.

[0003] For example, the invention patent with announcement number CN116563797B discloses a monitoring and management system for smart campuses, including an intelligent door lock system, a video surveillance system, an intelligent early warning notification module, an intelligent lighting system, and a monitoring and management system. The video surveillance system includes a first intelligent camera, a second intelligent camera, and a third intelligent camera, all of which are connected to a face recognition module and a posture safety recognition module. Through intelligent campus monitoring and management, the system identifies the posture and facial information of students when they are locked in classrooms and transmits this information to on-campus security personnel, teachers, and parents. It also intelligently adjusts the campus lighting, ensuring that students can safely leave the campus with the knowledge and accompaniment of security personnel, parents, and teachers, thus solving the problem that monitoring and management cannot adapt and respond when students are locked in classrooms.

[0004] For example, invention patent CN119339318B discloses an artificial intelligence-based campus abnormal behavior detection system, belonging to the field of abnormal behavior detection technology. This system includes: connecting to a campus monitoring platform to determine a first monitoring blind zone and a first monitoring neighborhood; establishing a first standard clustering feature set for the first monitoring blind zone; acquiring first monitoring video data; identifying and labeling personnel to generate a first monitoring object time sequence; performing clustering feature recognition to generate a first real-time clustering feature; performing abnormal clustering feature recognition to generate a first abnormal probability; and generating a first abnormal warning message when the first abnormal probability is greater than a preset warning probability. This invention solves the technical problem in existing technologies where traditional campus monitoring struggles to cover blind zones and accurately judges abnormal clustering behavior at different times, achieving the technical effect of effectively detecting and warning of abnormal behavior in campus monitoring blind zones, and improving the level of campus safety management.

[0005] Existing image-based access control methods typically combine card swipe records or facial recognition results to verify the identity of passersby, supplemented by video surveillance for post-event review or simple anomaly detection. These methods largely rely on single-trigger information or single-frame detection results, lacking correlation analysis of continuous multi-target behavior within a door opening time interval. They fail to comprehensively model changes in distance between people, consistency of movement direction, and gate occupancy status. In dense or continuous passage scenarios, they are prone to difficulty distinguishing between normal congestion and accompanying abnormal behavior, resulting in insufficient overall recognition stability and early warning accuracy.

[0006] To address the above issues, there is an urgent need for a campus security early warning method based on multimodal fusion and AI vision. Summary of the Invention

[0007] Technical problems to be solved

[0008] To address the shortcomings of existing technologies, this invention provides a campus security early warning method based on multimodal fusion and AI vision, which solves the problem that when people travel in the same direction during peak hours, the lack of correlation constraints between the method and the opening and closing rhythm of access control makes it difficult to identify tailgating in advance.

[0009] Technical solution

[0010] To achieve the above objectives, this invention provides the following technical solution: a campus security early warning method based on multimodal fusion and AI vision, comprising the following steps: S1, collecting access control data directly generated during access control, constructing an access control perception dataset, and preprocessing the access control perception dataset; S2, identifying the entry sequence and access behavior characteristics of access objects based on the access control perception dataset, forming an accompanying relationship expression; S3, based on the accompanying relationship expression, performing matching relationship discrimination by combining gate occupancy status and authorized behavior records to distinguish access behavior types; S4, performing risk-driven assessment based on gate occupancy status and accompanying behavior characteristics, forming risk change characteristics and determining risk response conditions; S5, executing linkage control behavior based on risk response conditions and completing access process recording and strategy adjustment, constructing a closed loop for risk event handling.

[0011] Furthermore, the specific steps for collecting access control data directly generated during the access control process and constructing the access control perception dataset are as follows: Collect access control data directly generated during the access control process, including access control trigger time, door magnetic status signal, gate area video frames, video frame timestamps, and the position of human target detection boxes in the video frames; perform alignment processing on the access control trigger time and door magnetic status signal based on the timestamp correspondence, and uniformly map them to the video frame timestamps; arrange the video frame timestamps according to the execution order, establish a correspondence between the positions of human target detection boxes in adjacent video frames and generate object identifiers, and establish a joint index structure of time index and object index based on the timestamps and object identifiers to construct the access control perception dataset.

[0012] Furthermore, the specific steps for preprocessing the access control perception dataset are as follows: Constructing door opening interval constraints based on the access control trigger time and door magnetic status signal, and filtering corresponding passage time segments; For the correlation relationship of human target detection box positions in video frames under continuous time markers, using the Hungarian algorithm to perform cross-frame matching, and constructing a matching cost matrix based on the intersection-union ratio of detection boxes and the distance to the center point to establish a stable object association structure; For the time offset between multi-source time series, using the dynamic time warping method to perform cross-sequence alignment; For the numerical data in the access control perception dataset, using the range standardization method to perform standardization processing; For the standardized data, using the min-max normalization method to perform normalization processing.

[0013] Further, the specific steps for identifying the entry sequence and passage behavior characteristics of access objects based on the access control perception dataset are as follows: Based on the access control trigger time, locate the time marker of the door magnetic state signal entering the open state and determine it as the door opening start time; locate the time marker of the door magnetic state signal changing from open to closed within the same trigger cycle and determine it as the door opening end time; extract the video frame time markers between the door opening start time and the door opening end time, count the video frames of the corresponding gate area frame by frame to obtain the window sampling frame number; extract all time markers between the door opening start time and the door opening end time, and arrange them in chronological order to form a window time index set; read the door magnetic state signal corresponding to each time marker in the window time index set and execute the opening... The process involves several steps: First, determining the gate's open status and obtaining its status marker. Then, iterating through the video frames of the gate area corresponding to the window's time index set, performing deduplication and statistical analysis on the human target detection box positions according to the object index to obtain the number of objects passing through. Next, subtracting the opening start time from the closing time to obtain the opening duration. Finally, multiplying the gate's open status marker corresponding to each time marker in the window's time index set by the number of objects passing through, yields the single-time-marker passage occupancy product. Summing all single-time-marker passage occupancy products in the window's time index set yields the cumulative window passage occupancy. Dividing the cumulative window passage occupancy by the number of window sampling frames yields the average window passage occupancy. Finally, multiplying the opening duration by the average window passage occupancy yields the gate opening and gate holding intensity value.

[0014] Further, the specific steps for forming the accompanying relationship expression are as follows: The door opening and window holding strength value is compared with the window holding judgment threshold in real time; when the door opening and window holding strength value is greater than or equal to the window holding judgment threshold, the door opening and window holding strength value is written into the window state buffer; the video frame sequence corresponding to the time marker between the door opening start time and the door opening end time is continuously retained according to the video frame retention length; the door opening state marker is updated time-by-time according to the time marker; the object numbering of the human detection results corresponding to the time marker is kept continuous and consistent; the horizontal and vertical position coordinates of the same object are continuously updated under continuous time markers; when the door opening and window holding strength value is less than the window holding judgment threshold, the door opening time interval is marked as a non-retention interval; only the door opening start frame and end frame are retained according to the video frame retention length of the time interval; the numbering update of human targets within the time interval is stopped; and the detection results with a trajectory length less than the trajectory length threshold are eliminated.

[0015] Furthermore, the specific steps for determining the matching relationship based on the expression of the accompanying relationship and the gate occupancy status and authorized behavior records are as follows: Obtain the window time index set; extract the human target detection box positions in continuous video frames under the constraints of the time index and the corresponding object indices of the first and second objects, construct a position change sequence, calculate the direction of adjacent position changes and perform angle conversion to obtain the motion direction angle variables of the first and second objects; read the human target detection box positions of the first and second objects corresponding to the time markers, extract coordinates along the lateral direction of the gate, and obtain the lateral position coordinate variables of the first and second objects; select the human target detection box positions of the first and second objects under continuous time markers, calculate the position change along the passage direction and perform a difference operation based on the time marker intervals to obtain the longitudinal velocity variables of the first and second objects. Within the window time index set, select object combinations where the first and second objects are different; subtract the motion direction angle of the second object from the motion direction angle of the first object, take the cosine, and add one to obtain the numerator of direction consistency; multiply the absolute value of the difference between the lateral position coordinates of the first and second objects by the lateral proximity adjustment coefficient to obtain the lateral proximity constraint; multiply the absolute value of the difference between the longitudinal velocity of the first and second objects by the longitudinal synchronization adjustment coefficient to obtain the longitudinal synchronization constraint; add one to the lateral proximity constraint and the longitudinal synchronization constraint to obtain the denominator of the accompanying constraint; divide the numerator of direction consistency by the denominator of the accompanying constraint to obtain the accompanying ratio of the object combination; take the maximum value of the accompanying ratio of all object combinations under the same time mark to obtain the maximum accompanying ratio of a single time mark; take the maximum value of the maximum accompanying ratio of all single time marks within the window time index set to obtain the sticking and accompanying discrimination value.

[0016] Further, the specific steps for distinguishing traffic behavior types are as follows: The proximity-accompaniment discrimination value is compared with the accompanying threshold in real time. The accompanying threshold includes a first-level accompanying threshold and a second-level accompanying threshold. When the proximity-accompaniment discrimination value is less than the first-level accompanying threshold, the proximity-accompaniment discrimination value is written into the accompanying relationship buffer. The detection boxes of the object pair are updated according to time markers, and the horizontal position coordinates and vertical velocity are updated under continuous time markers. When the proximity-accompaniment discrimination value is greater than or equal to the first-level accompanying threshold and less than the second-level accompanying threshold, the retention length of the video frames corresponding to the object pair is extended. The horizontal position of the left and right boundaries of the detection boxes of the object pair is updated time-by-time. The difference in horizontal spacing between the object pairs is calculated and recorded time-by-time. When the proximity-accompaniment discrimination value is greater than or equal to the second-level accompanying threshold, the object pair is numbered and locked. The sampling update frequency of the detection area corresponding to the object pair is increased. The difference in horizontal position and vertical velocity of the left and right boundaries of the object pair is calculated and recorded under each time marker. The video frames within the continuous time marker range corresponding to the object pair are completely retained.

[0017] Further, the specific steps of performing risk-driven assessment based on gate occupancy status and accompanying behavioral characteristics are as follows: Under the constraints of time stamps and object indexes, retrieve the human target detection box positions in the video frames of the gate area; read the horizontal coordinates of the left and right boundaries of the detection box to obtain the horizontal positions of the left and right boundaries of the object, respectively; filter the time stamp set within the time index range that only detects a single human target; extract the horizontal coordinates of the left and right boundaries of the corresponding human target detection box positions and calculate the difference sequence; perform statistical averaging on the difference sequence to obtain the single-person occupancy benchmark; extract the door-opening window-blocking strength value based on the window state buffer; extract the adjacency discrimination value based on the accompanying relationship buffer; obtain the window sampling frame number, window time index set, door opening status stamp, and passing object. The following steps are performed: Quantity; Subtract the horizontal position of the left boundary of the object from the horizontal position of the right boundary to obtain the horizontal width of a single object; Sum the horizontal widths of all objects passing through at the same time marker to obtain the actual width of the gate; Multiply the single-person width reference by the number of passing objects to obtain the person reference width; Subtract the person reference width from the actual width of the gate to obtain the width deviation; Multiply the width deviation by the gate opening status marker to obtain the effective width deviation of a single time marker; Sum the effective width deviations of all single time markers within the window time index set, take the absolute value, and divide by the number of window sampling frames to obtain the average width deviation; Multiply the accompanying enhancement adjustment coefficient by the accompanying discrimination value and add one to obtain the accompanying enhancement factor; Multiply the average width deviation by the accompanying enhancement factor to obtain the gate alarm drive value.

[0018] Further, the specific steps for forming risk change characteristics and determining risk response conditions are as follows: Write the gate alarm drive value into the risk execution register; calculate the difference between the gate alarm drive value corresponding to the current time marker and the gate alarm drive value corresponding to the previous time marker, and take the absolute value of the difference to obtain the absolute value of the difference; adjust the duration of the prompt information, the prompt trigger frequency, and the video frame retention length before and after the door opening time interval based on the absolute value of the difference; adjust the human body detection frame update frequency according to the gate alarm drive value, and update the horizontal boundary position and vertical movement status under the time marker; when the difference is positive, keep the object number and trajectory data within the current time interval from being overwritten; when the difference is negative, restore the video frame retention length and detection update frequency to the initial set values.

[0019] Furthermore, the specific steps for executing the linkage control behavior based on the risk response conditions and completing the passage process recording and strategy adjustment to construct a closed loop for risk event handling are as follows: Based on the adjusted duration and trigger frequency of the prompt information, trigger the access control terminal prompt information and voice reminder; based on the adjusted video frame retention length, perform retention processing on the video frames and video segments within the corresponding time interval; under the constraints of time stamps and object indexes, mark the objects with a proximity discrimination value greater than or equal to the first-level accompaniment threshold and record the object trajectory data under continuous time stamps; extract the time stamps, window time index sets, object trajectory data, and access control authorization status triggered by the risk response conditions and perform recording and storage; based on the frequency of risk events corresponding to the same access control trigger position in the recorded storage, update the door opening duration and prompt trigger frequency, and limit the range of the update results based on the statistical distribution of the frequency of risk events, write the update results into the access control parameter area, and participate in the construction of door opening interval constraints in the subsequent access control triggering process.

[0020] Beneficial effects

[0021] The present invention has the following beneficial effects:

[0022] (1) This invention constructs a unified time index of access control trigger time, door magnetic status signal and video frame time mark, and forms a window time index set under the constraints of door opening start time and door opening end time. Combined with the door opening window strength value, the passage time interval is filtered so that the door opening status mark and the number of passage objects are continuously expressed under the unified time mark. Compared with the method of relying only on single trigger or single frame recognition, it realizes the continuous characterization of the passage process within the effective door opening interval, and improves the temporal consistency and data validity of passage behavior analysis.

[0023] (2) This invention extracts the position change of the human target detection box under continuous time markers, constructs the first object motion direction angle variable, horizontal position coordinate variable and vertical velocity variable, and introduces the horizontal proximity adjustment coefficient and the vertical synchronization adjustment coefficient to form the sticking companion discrimination value, so that the direction consistency, spatial proximity and motion synchronization relationship between objects are uniformly quantified. Compared with the method of judging only based on position or single feature, it realizes the coupled expression of multi-dimensional behavioral features and improves the ability to distinguish behavioral features in complex traffic scenarios.

[0024] (3) This invention constructs the absolute value of the width offset based on the single person width occupancy benchmark, the number of passing objects and the horizontal width occupancy of the objects, and combines it with the accompanying discrimination value to form the accompanying reinforcement value, and further obtains the gate joint alarm driving value, so that the gate occupancy status and accompanying behavior characteristics are fused and expressed under the same calculation framework. Compared with the method of independently analyzing the number of people or space occupancy, it realizes the unified characterization of the group passage structure and enhances the overall assessment capability of the multi-target passage status.

[0025] (4) This invention writes the gate alarm drive value into the risk execution register area, and continuously adjusts the duration of the prompt information, the prompt trigger frequency and the video frame retention length based on the differential absolute value. On this basis, it updates the door opening duration and prompt trigger frequency in combination with the frequency of risk events and writes them back to the access control parameter area, so that the passage process record and control parameters form a feedback association in the same link. Compared with the fixed parameter control method, it realizes the dynamic adjustment and continuous optimization of the access control response strategy.

[0026] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0027] Figure 1 This is a flowchart of a campus security early warning method based on multimodal fusion and AI vision according to the present invention;

[0028] Figure 2 This is a trend chart of the adhesion and accompaniment discrimination value of the present invention;

[0029] Figure 3 This is the event grid diagram of the access window in this invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] Please see Figures 1-3This invention provides a technical solution: a campus security early warning method based on multimodal fusion and AI vision, comprising the following steps: S1, collecting access control data directly generated during access control, constructing an access control perception dataset, and preprocessing the access control perception dataset; S2, identifying the entry order and access behavior characteristics of access objects based on the access control perception dataset, forming an accompanying relationship expression; S3, based on the accompanying relationship expression, performing matching relationship discrimination by combining the gate occupancy status and authorized behavior records to distinguish access behavior types; S4, performing risk-driven assessment based on the gate occupancy status and accompanying behavior characteristics, forming risk change characteristics and determining risk response conditions; S5, executing linkage control behavior based on risk response conditions and completing access process recording and strategy adjustment, constructing a closed loop for risk event handling.

[0032] Specifically, the steps for collecting access control data directly generated during access control and constructing an access control perception dataset are as follows: A Hall effect magnetic sensor is placed at the access control trigger position to output a magnetic door status signal; the relay trigger edge is collected in the access control circuit to form the access control trigger time; an industrial network camera is installed above the gate to collect video frames of the gate area and simultaneously generate video frame timestamps; in the video processing unit, human target detection is performed on the gate area video frames based on computer vision algorithms, and the human target detection box position is output as a pixel coordinate pair consisting of the coordinates of the upper left and lower right corners of the rectangle. The pixel coordinates establish a two-dimensional image coordinate system based on the video frame image resolution; based on the timestamp pairs... The system aligns the access control trigger time with the door magnetic status signal and maps them to video frame timestamps to form a unified timeline. Based on the video frame timestamps, the system establishes a correspondence between the human target detection box positions in adjacent video frames based on spatial overlap and center point displacement continuity, generating object identifiers. A joint index structure of time index and object index is constructed based on the timestamps and object identifiers. Abnormal frames are removed using a discrimination method based on the median absolute deviation. Missing detection box intervals are filled in using piecewise cubic interpolation while maintaining trajectory continuity. Finally, an access control perception dataset is constructed for subsequent access window construction and associated relationship parsing.

[0033] In this implementation scheme, the access control trigger time, the door magnetic status signal, and the video frames of the gate area form a stable correspondence under a unified time axis. The video frame time stamps and the human target detection box positions are uniformly organized. After the joint structure of time index and object index is established, the continuous association expression of multi-source information is realized. Abnormal frames and trajectory breaks are effectively constrained. The passage behavior is traceable and computable in both temporal and spatial dimensions, providing a consistent data foundation for the subsequent construction of passage windows and the analysis of accompanying relationships.

[0034] Specifically, the preprocessing steps for the access control perception dataset are as follows: First, door opening interval constraints are constructed based on the access control trigger time and door magnetic sensor status signal, and corresponding passage time segments are selected. The door opening interval is defined by continuous open state segments in the door magnetic sensor status signal and forms a time boundary constraint with the access control trigger time. Second, regarding the association relationship of human target detection box positions in video frames under continuous time markers, the Hungarian algorithm is used to perform cross-frame matching, and a matching cost matrix is ​​constructed based on the cross-union ratio of detection boxes and the Euclidean distance of the center point. A unified measurement is performed on the spatial overlap and positional change amplitude between different detection boxes to establish a stable object association structure. Third, a dynamic time warping method is used to perform cross-sequence matching to address the time offset between multi-source time series. Column alignment and dynamic time warping path search constraints are limited to the time window formed by the access control trigger time and the video frame time stamp. This is used to eliminate the inconsistency between the door magnetic signal sampling period and the video frame sampling period, as well as the nonlinear time deviation caused by clock drift. The numerical data in the access control perception dataset are standardized using the range standardization method to uniformly map time quantities and spatial location quantities of different dimensions to the same numerical interval to achieve dimension elimination. The standardized data are then normalized using the min-max normalization method to transform all data involved in the calculation into a dimensionless proportional expression and maintain the consistency of numerical distribution. This provides a unified scale data foundation for subsequent adjoint relationship analysis and width coupling discrimination.

[0035] In this implementation scheme, the door opening interval constraint and the passage time segment are consistently divided, the multi-source time series are uniformly aligned under the conditions of time offset and sampling difference, the human target detection box position association structure remains continuous and stable, the cross-frame object relationship has traceable expression, the numerical data is transformed into a dimensionless unified scale after standardization and normalization, and the overall data structure is uniformly organized in the time and space dimensions, providing a stable input foundation for subsequent adjoint relationship analysis and width coupling discrimination.

[0036] Specifically, the steps for identifying the entry sequence and behavior characteristics of access objects based on the access control perception dataset are as follows: Based on the access control trigger time, locate the time marker of the door magnetic status signal entering the open state and determine it as the door opening start time. The door magnetic status signal uses a binary encoding form to represent the door's open and closed state, with the open state recorded as 1 and the closed state as 0. Locate the time marker of the door magnetic status signal changing from open to closed within the same trigger cycle and determine it as the door opening end time. Extract the video frame time markers between the door opening start time and the door opening end time, count the video frames of the corresponding gate area frame by frame to obtain the window sampling frame number. Extract all time markers between the door opening start time and the door opening end time and arrange them in chronological order to form a window time index set. Read the door magnetic status signal corresponding to each time marker in the window time index set, perform the open state determination, and obtain the door opening state marker. The door opening state marker takes a value of 0 or 1. Traverse the gate area video frames corresponding to the window time index set and detect human targets by object index. The frame location is deduplicated to obtain the number of passing objects. The number of passing objects is the effective target count result after deduplication by object index under each time mark. The opening duration is obtained by subtracting the opening start time from the opening end time. The door opening status mark corresponding to each time mark in the window time index set is multiplied by the number of passing objects to obtain the single time mark passage occupancy product. The passage occupancy products of all single time marks in the window time index set are summed to obtain the cumulative passage occupancy of the window. The cumulative passage occupancy of the window is divided by the number of window sampling frames to obtain the average passage occupancy of the window. The door opening duration is multiplied by the average passage occupancy of the window to obtain the door opening window strength value. The door opening status mark is a binary dimensionless encoded quantity, the number of passing objects is a discrete dimensionless count, and the door opening window strength value retains the dimensionless attribute. The door opening window strength value is jointly determined by the cumulative relationship between the door opening duration interval and the effective occupancy degree within the window. It is used to characterize the actual carrying level of passage behavior during the door opening stage and limit the effective analysis range of subsequent identification.

[0037] The specific calculation method for the strength value of doors and windows when open is as follows:

[0038]

[0039] In the formula, This indicates the strength value of the door and window when open. Indicates the start time of door opening. Indicates the time when the door opens and closes. Indicates the number of frames sampled in the window. Represents the set of window time indices. Indicates the door's open / closed status. Indicates the number of objects allowed to pass.

[0040] In this implementation scheme, the start and end times of door opening and the window time index set form a stable correspondence. The door opening status marker and the number of passing objects are consistently expressed in the time dimension. The entry order of passing objects and the characteristics of passing behavior have a continuous characterization basis. The cumulative structure under the constraint of the number of window sampling frames maintains stable statistical characteristics. The door opening window intensity value forms a unified quantitative expression and maintains dimensionless properties, providing a reliable basis for subsequent passage window determination and accompanying relationship analysis.

[0041] Specifically, the steps for forming the accompanying relationship expression are as follows: The door opening and window stopping strength value is compared with the window stopping judgment threshold in real time; when the door opening and window stopping strength value is greater than or equal to the window stopping judgment threshold, the door opening and window stopping strength value is written into the window state buffer; the video frame sequence corresponding to the time marker between the door opening start time and the door opening end time is continuously retained according to the video frame retention length, which is determined based on the statistical quantile of the duration distribution of continuous targets in the historical valid passage window and remains consistent within the control period; the door opening status marker is updated time-by-time according to the time marker, and the object number of the human detection result corresponding to the time marker is kept continuous and consistent. Under continuous time marking, the horizontal and vertical position coordinates of the same object are continuously updated to form a stable trajectory representation. When the door opening window intensity value is less than the window judgment threshold, the door opening time interval is marked as a non-resident interval. For the video frame sequence of the time interval, only the door opening start frame and end frame are retained according to the video frame retention length. The numbering update of human targets within the time interval is stopped. The detection results with trajectory length less than the trajectory length threshold are eliminated. The trajectory length threshold is determined based on the lower quantile statistical results of the distribution of the number of consecutive occurrences of objects in historical normal passage samples. It is used to distinguish between valid passage trajectories and short-term false detection trajectories, thereby ensuring that the association relationship representation is only based on the construction of stable trajectories.

[0042] In this implementation scheme, the door opening interval is effectively filtered under the window determination constraint, the video frame sequence forms a continuous retention structure in the time dimension, the door opening status mark is updated in a consistent manner with the object number, the horizontal position coordinates and the vertical position coordinates form a continuous trajectory expression, short-term unstable detection results are eliminated, the passage behavior presents stable temporal correlation characteristics within the window, and the accompanying relationship expression has clear boundaries and a reliable supporting foundation.

[0043] Specifically, for the expression of accompanying relationships, the matching relationship judgment is carried out by combining the gate occupancy status and authorized behavior records as follows: Obtain the window time index set; extract the human target detection box positions in continuous video frames under the constraints of time index and the corresponding object indexes of the first and second objects, construct the position change sequence, and establish the pixel coordinate system of the human target detection box positions based on the video frames of the gate area, and map them to the ground coordinate system through gate plane perspective correction to form a unified spatial reference; calculate the direction of change of adjacent positions and perform angle conversion to obtain the motion direction angle variables of the first object and the second object, and limit the value range of the motion direction angle variables to the range of negative π to positive π; read the first object and the second object corresponding to the time mark. The human target detection box positions are determined by extracting coordinates along the lateral direction of the gate to obtain the lateral position coordinate variables of the first and second objects. The lateral direction is defined as the gate width direction perpendicular to the passage direction. The human target detection box positions of the first and second objects are selected under continuous time markers. The position change is calculated along the passage direction, and a difference operation is performed based on the time marker intervals to obtain the longitudinal velocity variables of the first and second objects. The passage direction is defined as the gate entry / exit direction and is orthogonal to the lateral direction. Objects where the first and second objects are different are selected from the window time index set. The motion direction angle of the first object is subtracted from the motion direction angle of the second object, the cosine is taken, and one is added to obtain the numerator of consistent direction. The absolute value of the difference between the lateral position coordinates of the first object and the second object is multiplied by a lateral proximity adjustment coefficient to obtain a lateral proximity constraint term. This coefficient is determined by statistical analysis of the lateral spacing distribution of passing objects in historical normal passage samples, combined with the gate width range, and ranges from 0.5 to 3.0. The absolute value of the difference between the longitudinal speeds of the first and second objects is multiplied by a longitudinal synchronization adjustment coefficient to obtain a longitudinal synchronization constraint term. This coefficient is obtained by statistical analysis of the longitudinal speed difference distribution in normal continuous passage and abnormal close-proximity passage scenarios, and ranges from 0.5 to 2.5. The lateral proximity constraint term and the longitudinal synchronization constraint term are then added together to obtain... The denominator of the accompanying constraint is obtained; the numerator of the direction consistency is divided by the denominator of the accompanying constraint to obtain the object combination accompanying ratio; the maximum value of the accompanying ratio of all object combinations under the same time mark is taken to obtain the maximum accompanying ratio of a single time mark; the maximum value of the maximum accompanying ratio of all single time marks within the window time index set is taken to obtain the close-to-accompany discrimination value; the close-to-accompany discrimination value is composed of the ratio of the direction consistency numerator and the accompanying constraint denominator, and each participating quantity is derived from the proportional expression of the angle difference, position difference and velocity difference, and the whole maintains the dimensionless property; the direction consistency numerator represents the consistency of the object's motion direction, and the accompanying constraint denominator integrates the lateral closeness difference and the longitudinal synchronization difference, and the ratio of the two is used to characterize the strength of the close-to-accompany relationship between objects.The close - following discriminant value is used to characterize the degree of close - following and the tendency of synchronous entry formed by multiple passing objects within the same door - opening window.

[0044] The close - following discriminant value takes the consistency of the movement directions between objects as the main constraint, and the horizontal position proximity degree and the vertical speed synchronization degree as the auxiliary constraints, to comprehensively quantify the collaborative behavior between different objects. When two objects maintain a similar movement direction, the horizontal distance remains continuously small, and the vertical speed changes tend to be consistent, the close - following discriminant value gradually increases; when there are obvious direction deviations, horizontal separations or speed differences between objects, the close - following discriminant value gradually decreases. The close - following discriminant value can reflect whether there is a close - following relationship between objects that exceeds the normal queuing - passing characteristics, and extract the pair of objects with the highest degree of close - following from all object combinations as the analysis result, so as to highlight the most significant close - following behavior characteristics and provide a behavioral correlation basis for subsequent following - borrowing recognition, risk - driven assessment, and linkage warning triggering.

[0045] The specific calculation method of the close - following discriminant value is as follows:

[0046]

[0047] In the formula, represents the close - following discriminant value, represents the set of window - time indexes, represents the movement - direction angle of the first object, represents the movement - direction angle of the second object, represents the horizontal - position coordinate of the first object, represents the horizontal - position coordinate of the second object, represents the vertical speed of the first object, represents the vertical speed of the second object, represents the horizontal - proximity adjustment coefficient, represents the vertical - synchronization adjustment coefficient.

[0048] Table 1 shows the data table of mating and accompaniment discrimination values ​​provided in the embodiments of this application. For criterion 1, the motion direction angle of the first object is set to 0.36, the motion direction angle of the second object is set to 0.92, the lateral position coordinate of the first object is set to 2.18, the lateral position coordinate of the second object is set to 1.26, the longitudinal velocity of the first object is set to 0.61, the longitudinal velocity of the second object is set to 0.86, the lateral proximity adjustment coefficient is set to 0.82, and the longitudinal synchronization adjustment coefficient is set to 0.79. For criterion 2, the motion direction angle of the first object is set to 0.88, the motion direction angle of the second object is set to 0.41, the lateral position coordinate of the first object is set to 1.56, the lateral position coordinate of the second object is set to 1.21, the longitudinal velocity of the first object is set to 1.08, the longitudinal velocity of the second object is set to 0.94, the lateral proximity adjustment coefficient is set to 0.91, and the longitudinal synchronization adjustment coefficient is set to 0.73. For criterion 3, the motion direction angle of the first object is set to 0.47, the motion direction angle of the second object is set to 0.95, the lateral position coordinate of the first object is set to 2.04, and the lateral position coordinate of the second object is set to 0.73. For discrimination 4, the first object's longitudinal speed is set to 1.37, the second object's longitudinal speed is set to 1.52, the second object's longitudinal speed is set to 1.05, the lateral proximity adjustment coefficient is set to 0.77, and the longitudinal synchronization adjustment coefficient is set to 0.88. For discrimination 5, the first object's motion direction angle is set to 1.02, the second object's motion direction angle is set to 0.63, the first object's lateral position coordinate is set to 1.42, the second object's lateral position coordinate is set to 1.16, the first object's longitudinal speed is set to 1.03, the second object's longitudinal speed is set to 0.97, the lateral proximity adjustment coefficient is set to 0.95, and the longitudinal synchronization adjustment coefficient is set to 0.69. For discrimination 5, the first object's motion direction angle is set to 0.59, the second object's motion direction angle is set to 0.84, the first object's lateral position coordinate is set to 1.88, the second object's lateral position coordinate is set to 1.29, the first object's longitudinal speed is set to 0.41, the second object's longitudinal speed is set to 0.88, the lateral proximity adjustment coefficient is set to 0.84, and the longitudinal synchronization adjustment coefficient is set to 0.81.

[0049] Table 1. Data Table of Adhesion Accompanying Discriminant Values

[0050]

[0051] like Figure 2The image shows a trend chart of the adjacency discrimination value provided in this embodiment of the application. According to the data in the image and table, the set first-level adjacency threshold is 0.90, and the second-level adjacency threshold is 1.20. The adjacency discrimination values ​​corresponding to the five groups of discriminations fluctuate between 0.79 and 1.49, exhibiting an overall fluctuating and alternating characteristic. The adjacency discrimination value of discrimination 1 is 0.79, lower than the first-level adjacency threshold; the adjacency discrimination value of discrimination 2 is 1.33, higher than the second-level adjacency threshold; the adjacency discrimination value of discrimination 3 is 0.98, between the first-level and second-level adjacency thresholds; the adjacency discrimination value of discrimination 4 is 1.49, significantly higher than the second-level adjacency threshold; and the adjacency discrimination value of discrimination 5 is 1.02, between the two thresholds. It is evident that there are significant differences in the degree of close proximity between different criteria. Criteria 2 and 4 correspond to values ​​in the higher range, while criterion 1 is in the lower range, and the remaining criteria are in the middle range. This can intuitively reflect the changes in the close proximity relationship between passing objects, providing a basis for subsequent risk level classification and coordinated response.

[0052] In this implementation scheme, the directional consistency and spatial relative position relationship between objects are uniformly characterized. The lateral position coordinates and longitudinal velocity are continuously expressed in the time series. The sticking companion discrimination value achieves stable quantitative output and maintains dimensionless properties. The object pair selection process is limited by the maximum value screening mechanism, and the computational scale is effectively controlled. Under occlusion and crowding conditions, it maintains a high ability to distinguish abnormal sticking companion behavior, providing a reliable input basis for subsequent occupancy coupling discrimination.

[0053] Specifically, the steps for distinguishing traffic behavior types are as follows: The proximity-accompaniment discrimination value is compared with the accompanying threshold in real time. The accompanying threshold includes a primary accompanying threshold and a secondary accompanying threshold. Traffic behavior types are divided into normal traffic type, suspected tailing type, and confirmed tailing type. When the proximity-accompaniment discrimination value is less than the primary accompanying threshold, the traffic behavior type is marked as normal traffic type, and the proximity-accompaniment discrimination value is written to the accompanying relationship cache. The detection boxes of the object pair are updated according to time markers. Under continuous time markers, the horizontal position coordinates and vertical velocity are updated. The detection update process is controlled by a fixed detection update cycle parameter. When the proximity-accompaniment discrimination value is greater than or equal to the primary accompanying threshold and less than the secondary accompanying threshold, the traffic behavior type is marked as suspected tailing type. The corresponding detection boxes of the object pair are updated according to time markers. The video frame retention length is extended, the horizontal position of the left and right boundaries of the detection box of the object pair is updated by time marker, the difference between the horizontal spacing of the object pair is calculated and recorded by time marker, and the detection update cycle is shortened to increase the number of detection updates per unit time. When the adjacency discrimination value is greater than or equal to the secondary adjacency threshold, the passage behavior type is marked as a determined tailing type, the object pair is numbered and locked, the sampling update frequency of the corresponding detection area of ​​the object pair is increased, the sampling update frequency is defined as the reciprocal of the detection update cycle, and the detection update cycle is shortened in high-risk conditions. The difference between the horizontal position and the vertical velocity of the left and right boundaries of the object pair is calculated and recorded at each time marker, and the video frames within the continuous time marker range corresponding to the object pair are completely retained.

[0054] In this implementation scheme, the passage behavior is classified into levels under the constraint of the accompanying threshold. Normal passage type, suspected tailing type and confirmed tailing type are clearly distinguished. The object pair detection box and the horizontal position coordinate and vertical velocity are continuously updated. The video frame retention length and the detection update cycle are dynamically adjusted. The numbering locking and high-frequency sampling mechanism ensure the complete recording of information under abnormal conditions. The overall discrimination process has a stable hierarchical expression capability.

[0055] Specifically, the risk-driven assessment based on the gate occupancy status and accompanying behavior characteristics follows these steps: Under time stamp and object index constraints, retrieve the human target detection box positions in the gate area video frames; read the left and right boundary horizontal coordinates of the detection box; the human target detection box positions are expressed based on the image pixel coordinate system and mapped through the gate plane to form a unified horizontal reference, obtaining the left and right boundary horizontal positions of the object respectively; within the time index range, filter the time stamp set where only a single human target is detected, extract the left and right boundary horizontal coordinates from the corresponding human target detection box positions, and calculate the difference sequence; perform statistical averaging on the difference sequence to obtain the single-person occupancy benchmark, which characterizes the horizontal occupancy range of a single object when passing through the gate; extract the door opening and window blocking strength value based on the window state buffer; extract the adhering association discrimination value based on the association relationship buffer; obtain the window sampling frame number, window time index set, door opening status stamp, and number of passing objects; subtract the left boundary horizontal position from the right boundary horizontal position of the object to obtain the single-object horizontal occupancy; for the same time... The actual width of the gate is obtained by summing the horizontal width occupied by all single objects under the marker; the width per person is multiplied by the number of objects passing through to obtain the width per person reference; the width per person reference is subtracted from the actual width of the gate to obtain the width deviation; the width deviation is multiplied by the gate opening status marker to obtain the effective width deviation of a single time marker; the absolute value of the summation of the effective width deviations of all single time markers in the window time index set is divided by the number of window sampling frames to obtain the average width deviation; the accompanying enhancement adjustment coefficient is multiplied by the aggression discrimination value and then added to obtain the accompanying enhancement adjustment coefficient. The gate joint warning driving value is obtained by multiplying the average width deviation by the accompanying enhancement factor. The accompanying enhancement adjustment coefficient is obtained by performing a discrimination analysis on the distribution of the adjacency discrimination values ​​of historical tailing events and normal passage events and determining the optimal discrimination ratio. The gate joint warning driving value is composed of the average width deviation and the accompanying enhancement factor. The average width deviation reflects the degree of deviation of the gate occupancy state from the normal passage state, and the accompanying enhancement factor reflects the degree of enhancement of the adjacency behavior. The two together constitute the basis for the calculation of the gate joint warning driving value.

[0056] The gate linkage driving value is used to comprehensively characterize the coupling relationship between the degree of abnormal gate occupancy and the intensity of accompanying close-fitting behavior. The gate linkage driving value not only considers the current occupancy status of people within the open doorway but also introduces a close-fitting discrimination value to strengthen the expression of abnormal accompanying behavior, thus forming a unified quantitative result for gate space occupancy characteristics and object behavior characteristics. When the deviation between the actual gate width and the width corresponding to the normal number of people increases, it indicates that there are phenomena such as multiple people passing in parallel, abnormal squeezing, obstruction and overlap, or shared passage during passage. When the close-fitting discrimination value increases synchronously, it indicates that there are strong characteristics of same-direction close-fitting and coordinated entry among objects. The gate linkage driving value integrates the width deviation relationship and the accompanying behavior relationship in calculation, distinguishing between simple congested passage and abnormal passage states with the risk of tailgating. The larger the gate linkage driving value, the higher the degree of superposition between abnormal space occupancy and abnormal accompanying behavior within the gate area; the smaller the gate linkage driving value, the closer the passage state is to normal independent passage characteristics. The gate-based joint alarm driving value serves as the core input for risk linkage analysis, supporting subsequent risk level determination and linkage response triggering.

[0057] The specific calculation method for the gate alarm drive value is as follows:

[0058]

[0059] In the formula, Indicates the gate alarm drive value. Indicates the number of frames sampled in the window. Represents the set of window time indices. Indicates the door's open / closed status. Indicates the number of objects allowed to pass. Indicates the horizontal position of the left boundary of the object. Indicates the horizontal position of the right boundary of the object. Indicates the width of a single person's footprint. Indicates the adjacency discriminant value. This indicates the accompanying enhancement regulation coefficient.

[0060] In this implementation plan, the gate occupancy status and accompanying behavioral characteristics form a unified driving expression. The lateral occupancy width and occupancy width offset are continuously characterized in the time series. The gate opening status mark effectively constrains the calculation range. The parallel operation of multiple users and the occupancy situation are clearly distinguished. The cumulative structure and the accompanying reinforcement relationship achieve synergy. The gate joint alarm driving value maintains the dimensionless attribute and has stable quantification capability, providing a direct basis for risk level determination.

[0061] Specifically, the steps for forming risk change characteristics and determining risk response conditions are as follows: Write the gate alarm drive value into the risk execution register and form a sequence record under continuous time markers; calculate the difference between the gate alarm drive value corresponding to the current time marker and the gate alarm drive value corresponding to the previous time marker, and take the absolute value of the difference to obtain the absolute value of the difference; adjust the duration of the prompt information, the prompt trigger frequency, and the video frame retention length before and after the gate opening time interval based on the absolute value of the difference. The duration of the prompt information is mapped based on the median and interquartile range of the historical normal passage phase prompt duration sequence and scaled proportionally with the absolute value of the difference. The prompt trigger frequency is based on the sliding statistical average of the number of risk triggers per unit time and scaled proportionally with the absolute value of the difference. The retention length is determined based on the upper quantile value of the video backtracking frame distribution corresponding to historical risk events, and increases stepwise according to the quantile interval with the absolute value of the difference. The update frequency of the human detection box is adjusted according to the gate joint police driving value. The detection box update frequency is expressed as the reciprocal of the detection update cycle. The detection update cycle is performed by performing an inverse proportional mapping based on the change rate of the gate joint police driving value sequence under continuous time markers and is limited to the change range corresponding to the historical stable interval. The horizontal boundary position and vertical movement status are continuously updated under the time markers. When the difference component is positive, the object number and trajectory data in the current time interval are kept from being overwritten and the continuous recording structure is maintained. When the difference component is negative, the video frame retention length and detection update frequency are restored to the corresponding interval range obtained from the statistics of the historical stable passage stage.

[0062] In this implementation plan, the gate-linked alarm driving value forms a continuous change expression in the time series, the risk change characteristics have a stable characterization basis, the duration of the prompt information and the prompt triggering frequency are adaptively adjusted with the intensity of change, the video frame retention length and the detection box update frequency form a coordinated adjustment relationship, the object number and trajectory data maintain a continuous recording structure, and the risk response conditions have clear trigger boundaries and dynamic adaptation capabilities.

[0063] Specifically, the steps for constructing a closed loop for risk event handling, including executing coordinated control actions based on risk response conditions and completing passage process recording and strategy adjustments, are as follows: Based on the adjusted duration and trigger frequency of the prompt information, trigger the access control system's prompt information and voice reminders. The prompt information is simultaneously output within the access control area in both audio-visual and voice broadcast formats. Based on the adjusted video frame retention length, perform retention processing on video frames and video segments within the corresponding time interval. Video frames are stored continuously according to time stamp order to form traceable video segments. Under time stamp and object index constraints, mark objects with an adjacency discrimination value greater than or equal to the first-level adjacency threshold and record object trajectory data under continuous time stamps. The trajectory data is generated from the horizontal position... The coordinate and vertical position coordinate sequence constitute and maintains temporal continuity; the time stamp, window time index set, object trajectory data and access control authorization status triggered by the risk response conditions are extracted and recorded and stored. The access control authorization status is formed by the authorization verification result returned by the access control system; based on the frequency of risk events corresponding to the same access control trigger position in the recorded storage, the door opening duration and prompt trigger frequency are updated. The frequency of risk events is obtained by statistically analyzing the event count results within the sliding time window, and the update results are range-limited based on the statistical distribution of the frequency of risk events. The range boundary is determined by the quantile interval of the historical stable passage stage. The update results are written into the access control parameter area and participate in the construction of door opening interval constraints in subsequent access control triggering processes.

[0064] In this implementation plan, prompts and voice reminders form an instant linkage response; video frames and video segments are continuously retained and traceable; abnormal objects form a stable trajectory record within the time series; risk event-related information is structurally stored; the frequency of risk event occurrences participates in the dynamic adjustment of door opening duration and prompt trigger frequency; the access control parameter area is continuously updated and participates in the construction of subsequent door opening interval constraints, forming a closed-loop processing mechanism.

[0065] like Figure 3The image shown is a grid diagram of the passage window event provided in an embodiment of this application. According to the image and data, the horizontal axis represents the window time index, and the vertical axis represents the object number. A grid is used to mark the existence state of the passing objects under each time index. The gate opening interval covers time indices 2 to 11. Within this interval, objects gradually enter and form a superimposed occupancy structure. Object 1 exists continuously from time indices 2 to 5, object 2 appears continuously from time indices 3 to 7, object 3 remains occupied from time indices 4 to 8, object 4 forms a continuous distribution from time indices 6 to 10, and object 5 gradually enters from time indices 8 to 11 and continues into the later part. The corresponding number of passing objects gradually increases from 0 to 3 over time, maintains a stable high occupancy state from time indices 4 to 8, and then gradually decreases to 0. This diagram, through a combination of grid distribution and time evolution, intuitively depicts the entire process of objects entering, overlapping, and exiting within the passage window, providing a basis for identifying unidirectional contacting behaviors and analyzing the gate occupancy structure.

[0066] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0067] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A campus security early warning method based on multimodal fusion and AI vision, characterized in that, Includes the following steps: S1. Collect access control data directly generated during the access control process, construct an access control perception dataset, and preprocess the access control perception dataset; S2, based on the access control perception dataset, identifies the entry order and access behavior characteristics of access objects, forming an accompanying relationship expression; S3, oriented towards the expression of accompanying relationships, combines the gate occupancy status and authorized behavior records to perform matching relationship discrimination and distinguish the type of passage behavior; S4. Perform risk-driven assessment based on the gate occupancy status and accompanying behavioral characteristics to form risk change characteristics and determine risk response conditions; S5 executes coordinated control actions based on risk response conditions and completes the recording of passage processes and strategy adjustments to build a closed loop for risk event handling.

2. The campus security early warning method based on multimodal fusion and AI vision according to claim 1, characterized in that: The specific steps for collecting access control data directly generated during the access control process and constructing an access control perception dataset are as follows: Collect access control data directly generated during the access control process, including access control trigger time, door magnetic status signal, gate area video frame, video frame timestamp, and human target detection box position in the video frame; perform alignment processing on access control trigger time and door magnetic status signal based on timestamp correspondence, and uniformly map them to video frame timestamp; Based on the execution order of video frame time stamps, the corresponding relationship between the human target detection box positions in the video frames and the adjacent video frames is established and object identifiers are generated. Based on the time stamps and object identifiers, a joint index structure of time index and object index is established to construct the access control perception dataset.

3. The campus security early warning method based on multimodal fusion and AI vision according to claim 1, characterized in that: The specific steps for preprocessing the access control and access sensing dataset are as follows: The system constructs door opening interval constraints based on access control trigger time and door magnetic status signals, and filters corresponding passage time segments. For the correlation of human target detection box positions in video frames under continuous time markers, the Hungarian algorithm is used to perform cross-frame matching, and a matching cost matrix is ​​constructed based on the intersection-union ratio of the detection boxes and the distance to the center point to establish a stable object association structure. Dynamic time warping is used to perform cross-sequence alignment to address time offsets between multi-source time series. Range standardization is used to standardize the numerical data in the access control perception dataset. Finally, min-max normalization is used to normalize the standardized data.

4. The campus security early warning method based on multimodal fusion and AI vision according to claim 1, characterized in that: The specific steps for identifying the entry order and access behavior characteristics of access control objects based on the access control perception dataset are as follows: Based on the access control trigger time, the time marker of the door magnetic status signal entering the open state is located and determined as the door opening start time; the time marker of the door magnetic status signal changing from open to closed within the same trigger cycle is located and determined as the door opening end time; the video frame time markers between the door opening start time and the door opening end time are extracted, and the video frames of the corresponding gate area are counted frame by frame to obtain the window sampling frame number; all time markers between the door opening start time and the door opening end time are extracted and arranged in chronological order to form a window time index set; the door magnetic status signal corresponding to each time marker in the window time index set is read, the open state determination is performed, and the door opening state marker is obtained; the gate area video frames corresponding to the window time index set are traversed, and the human target detection box positions are deduplicated and counted according to the object index to obtain the number of passing objects; Subtracting the opening start time from the opening end time gives the opening duration; multiplying the door opening status flag corresponding to each time marker in the window time index set by the number of passing objects gives the single time marker passage occupancy product; summing all single time marker passage occupancy products in the window time index set gives the window passage occupancy cumulative amount; dividing the window passage occupancy cumulative amount by the window sampling frame number gives the window average passage occupancy; multiplying the opening duration by the window average passage occupancy gives the door opening window strength value.

5. The campus security early warning method based on multimodal fusion and AI vision according to claim 1, characterized in that: The specific steps for forming the adjoint relation expression are as follows: The door opening and window closure strength value is compared with the window closure judgment threshold in real time; When the door opening and window holding strength value is greater than or equal to the window holding judgment threshold, the door opening and window holding strength value is written into the window state buffer. The video frame sequence corresponding to the time mark between the door opening start time and the door opening end time is continuously retained according to the video frame retention length. The door opening state mark is updated time-by-time according to the time mark. The human body detection results corresponding to the time mark are kept to keep the object number continuous and consistent. The horizontal and vertical position coordinates of the same object are continuously updated under continuous time marks. When the door opening and window holding intensity value is less than the window holding determination threshold, the door opening time interval is marked as a non-holding interval. For the video frame sequence of the time interval, only the door opening start frame and end frame are retained according to the video frame retention length. The numbering update of human targets within the time interval is stopped, and the detection results with trajectory length less than the trajectory length threshold are removed.

6. The campus security early warning method based on multimodal fusion and AI vision according to claim 1, characterized in that: The specific steps for determining the matching relationship based on the association relationship expression, combined with the gate occupancy status and authorized behavior records, are as follows: Obtain the window time index set; extract the human target detection box positions in consecutive video frames under the constraints of time index and corresponding object index of the first and second objects, construct the position change sequence, calculate the direction of change of adjacent positions and perform angle conversion to obtain the motion direction angle variables of the first object and the second object; read the human target detection box positions of the first and second objects corresponding to the time markers, extract the coordinates along the horizontal direction of the gate, and obtain the horizontal position coordinate variables of the first object and the horizontal position coordinate variables of the second object; Select the human target detection box positions of the first and second objects under continuous time markers, calculate the position change along the passage direction, and perform differential operation in combination with the time marker interval to obtain the longitudinal velocity variables of the first and second objects. Select a group of objects whose first and second objects are different from the window time index set; subtract the motion direction angle of the second object from the motion direction angle of the first object, take the cosine and add one to obtain the numerator with the same direction. Multiply the absolute value of the difference between the lateral position coordinates of the first object and the lateral position coordinates of the second object by the lateral proximity adjustment coefficient to obtain the lateral proximity constraint term; multiply the absolute value of the difference between the longitudinal velocity of the first object and the longitudinal velocity of the second object by the longitudinal synchronization adjustment coefficient to obtain the longitudinal synchronization constraint term. Add the first constraint to the horizontal proximity constraint and the vertical synchronization constraint to obtain the denominator of the accompanying constraint. Divide the numerator of the direction-consistent term by the denominator of the adjoint constraint term to obtain the object combination adjoint ratio; take the maximum value of all object combination adjoint ratios under the same time mark to obtain the maximum adjoint ratio of a single time mark; take the maximum value of all single time mark maximum adjoint ratios within the window time index set to obtain the close adjoint discrimination value.

7. A campus security early warning method based on multimodal fusion and AI vision according to claim 1, characterized in that: The specific steps for distinguishing traffic behavior types are as follows: The adhering accompaniment discrimination value is compared with the accompaniment threshold in real time. The accompaniment threshold includes a first-level accompaniment threshold and a second-level accompaniment threshold. When the sticking and accompaniment discrimination value is less than the first-level accompaniment threshold, the sticking and accompaniment discrimination value is written into the accompaniment relationship cache area, and the detection box of the object pair is updated according to the time mark. Under the continuous time mark, the horizontal position coordinate and vertical velocity are updated. When the adjacency discrimination value is greater than or equal to the first-level adjacency threshold and less than the second-level adjacency threshold, the length of the video frame corresponding to the object pair is extended, the horizontal position of the left and right boundaries of the detection box of the object pair is updated by time-by-time marker, and the horizontal spacing between the object pairs is calculated and recorded by time-by-time marker difference. When the adjacency discrimination value is greater than or equal to the secondary adjacency threshold, the object pair is numbered and locked, the sampling update frequency of the corresponding detection area of ​​the object pair is increased, the lateral position difference and longitudinal velocity difference of the left and right boundaries of the object pair are calculated and recorded at each time mark, and the video frames within the corresponding continuous time mark range of the object pair are completely preserved.

8. A campus security early warning method based on multimodal fusion and AI vision according to claim 1, characterized in that: The specific steps for performing risk-driven assessment based on gate occupancy status and accompanying behavioral characteristics are as follows: Under the constraints of time stamp and object index, the position of human target detection box in the video frame of the gate area is retrieved. The horizontal coordinates of the left and right boundaries of the detection box are read to obtain the horizontal position of the left and right boundaries of the object, respectively. Within the time index range, the time stamp set that only detects a single human target is filtered. The horizontal coordinates of the left and right boundaries of the corresponding human target detection box position are extracted and the difference sequence is calculated. The statistical average of the difference sequence is calculated to obtain the single person width reference. Extract the door opening and window holding strength value based on the window state buffer; extract the sticking and companion discrimination value based on the companion relationship buffer; Get the number of window sampling frames, the set of window time indices, the door opening status flag, and the number of passing objects; Subtract the horizontal position of the left boundary of the object from the horizontal position of the right boundary of the object to obtain the horizontal width of a single object; sum the horizontal widths of the single objects corresponding to the number of all passing objects under the same time mark to obtain the actual width of the gate; multiply the single person width reference by the number of passing objects to obtain the person reference width; subtract the person reference width from the actual width of the gate to obtain the width deviation. Multiply the width deviation by the gate opening status mark to obtain the effective width deviation of a single time mark; sum all the effective width deviations of single time marks in the window time index set, take the absolute value, and then divide by the number of window sampling frames to obtain the average width deviation; multiply the accompanying enhancement adjustment coefficient by the accompanying discrimination value and add one to obtain the accompanying enhancement factor; multiply the average width deviation by the accompanying enhancement factor to obtain the gate alarm drive value.

9. A campus security early warning method based on multimodal fusion and AI vision according to claim 1, characterized in that: The specific steps for forming risk change characteristics and determining risk response conditions are as follows: Write the gate alarm drive value into the risk execution register area; calculate the difference between the gate alarm drive value corresponding to the current time mark and the gate alarm drive value corresponding to the previous time mark, and take the absolute value of the difference to obtain the absolute value of the difference; The duration of the prompt message, the prompt trigger frequency, and the length of video frames retained before and after the door opening time interval are adjusted based on the differential absolute value; the update frequency of the human body detection box is adjusted according to the gate alarm linkage drive value, and the horizontal boundary position and vertical movement status are updated under the time mark; When the difference component is positive, the object number and trajectory data within the current time interval are kept from being overwritten; when the difference component is negative, the video frame retention length and detection update frequency are restored to the initial settings.

10. A campus security early warning method based on multimodal fusion and AI vision according to claim 1, characterized in that: The specific steps for executing coordinated control actions based on risk response conditions, recording passage processes, and adjusting strategies to construct a closed loop for risk event handling are as follows: Based on the adjusted duration and trigger frequency of the prompt message, trigger the access control terminal to send prompt messages and voice reminders; based on the adjusted video frame retention length, perform retention processing on video frames and video segments within the corresponding time interval; Under time stamp and object index constraints, object pairs with a matching discrimination value greater than or equal to the first-level matching threshold are marked and their trajectory data is recorded under continuous time stamps. The time stamps, window time index sets, object trajectory data, and access control authorization status triggered by risk response conditions are extracted and recorded. Based on the frequency of risk events corresponding to the same access control trigger location in the record storage, the door opening duration and prompt trigger frequency are updated, and the update results are range-limited based on the statistical distribution of the frequency of risk events. The update results are written into the access control parameter area and participate in the construction of door opening interval constraints in subsequent access control triggering processes.

Images