Campus security management and control method and system integrating multiple systems

Abstract

The invention discloses a campus security management and control method and system fusing multiple systems, and relates to the technical field of artificial intelligence, and the method comprises the steps: collecting a video stream and an audio signal; adopting a human body posture estimation model based on an attention mechanism to obtain human body posture information, calculating abnormal behavior data, and designing an abnormal event judgment function for judging whether an abnormal event occurs or not; the method comprises the following steps: constructing a campus security knowledge graph, constructing a graph neural network model, and when an abnormal event occurs, learning embedded representation of an abnormal event entity by using the graph neural network model, and analyzing a reasoning reason of the abnormal event; a sensor combination is obtained based on the abnormal audio signals, the signal similarity of each sensor pair in the sensor combination is calculated, and a candidate sound source position with the highest matching degree score is obtained through calculation according to the signal similarity and serves as the occurrence position of an abnormal event; sending the occurrence position of the abnormal event and the reasoning reason of the abnormal event to campus security personnel; and campus personnel safety monitoring based on multi-system data is realized.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a campus security management and control method and system integrating multiple systems. Background Art

[0002] As the degree of campus intelligence continues to improve, the application of video surveillance, audio monitoring and sensor technology in campus security has received widespread attention. As a campus environment with dense personnel flow and frequent activities, it is particularly important to ensure the safety of students, faculty and staff. In this context, real-time detection and precise positioning of abnormal events have become the key to ensuring campus safety.

[0003] A Chinese patent application with publication number CN118153953A discloses a campus safety incident warning method based on spatiotemporal analysis and video AI. The method includes constructing a spatial data model, building a road network, deploying cameras, and generating a warning model. By combining the campus's specific schedule, three-dimensional spatial structure characteristics, and real-time video intelligent analysis results, a three-dimensional, intelligent warning system is constructed, thereby effectively improving the foresight and accuracy of campus safety management.

[0004] Existing campus security management and control methods mainly rely on single video surveillance data, which makes it difficult to fully utilize the data sources of multiple systems on campus, resulting in limited perception and analysis capabilities of abnormal events. Summary of the invention

[0005] The present application aims to solve at least one of the technical problems in the related art to a certain extent. To this end, one purpose of the present application is to propose a campus safety management and control method and system integrating multiple systems, so as to realize campus personnel safety monitoring based on multi-system data.

[0006] One aspect of the present application provides a campus security management method integrating multiple systems, including:

[0007] Step S100: deploying high-definition cameras and sensor arrays in the campus area to collect video streams and audio signals. When an abnormal audio signal is captured, the video streams before and after the abnormality occurs are obtained;

[0008] Step S200: using a human posture estimation model based on an attention mechanism to obtain human posture information in a video stream, calculating abnormal behavior data, and designing an abnormal event judgment function for judging whether an abnormal event occurs based on the abnormal behavior data;

[0009] Step S300: construct a campus security knowledge graph based on campus multi-system data, and construct a graph neural network model based on the campus security knowledge graph. When an abnormal event occurs, use the graph neural network model to learn the embedded representation of the abnormal event entity in the abnormal event and analyze the reasoning cause of the abnormal event;

[0010] Step S400: obtaining a sensor combination based on the abnormal audio signal, calculating the signal similarity between each sensor pair in the sensor combination, and obtaining a candidate sound source position with the highest matching score according to the signal similarity calculation, and taking it as the occurrence position of the abnormal event;

[0011] Step S500: sending the location of the abnormal event and the reasoning cause of the abnormal event to campus security personnel;

[0012] The specific method of arranging high-definition cameras and sensor arrays in the campus area to collect video streams and audio signals, and when an abnormal audio signal is captured, obtaining the video streams before and after the abnormality occurs is as follows: deploying high-definition cameras and sensor arrays in the campus area, arranging the sensor arrays in a uniformly distributed topological structure near the cameras, and the sensor spacing of the sensor arrays satisfy: , where c is the speed of sound, is the maximum frequency of the audio signal; when the sensor array is abnormal When an abnormal audio signal is captured, obtain the time when the abnormality occurs Duration Video stream;

[0013] The specific method for obtaining the human posture estimation model based on the attention mechanism is:

[0014] Step S210: Use the human body posture estimation model based on the attention mechanism to estimate the human body posture, extract the spatial coordinates of the key points of the human body, and introduce the attention mechanism in the time dimension to weightedly aggregate the human body posture features of different frames. The attention weight of the tth frame is ;

[0015] Step S220: In the spatial dimension, an attention mechanism is introduced to extract feature vectors of different parts of the human body. The attention weight of part j1 is ;

[0016] Step S230: The human posture estimation model based on the attention mechanism uses the Attention-LSTM network as the initial network, including a CNN backbone network, an attention mechanism and an LSTM timing module, obtains continuous frames of human body images based on the video stream as input data, uses the human body key points of each frame of the human body image as output data, and obtains human body posture information estimated by the Attention-LSTM network according to the spatial position of the human body key points, the relative position and distance between the human body key points, and the motion trajectory of the human body key points. The motion trajectory of the human body key points is obtained by connecting the coordinates of the same human body key points in human body images of different frames;

[0017] The training method of the human posture estimation model is:

[0018] The Attention-LSTM network is used as the initial network. The end of the Attention-LSTM network is connected to the fully connected layer. The hidden state output by the LSTM timing module is mapped to the two-dimensional coordinates of the key points of the human body. The output of the entire model is the coordinate sequence of the key points of the human body corresponding to each frame of the human body image.

[0019] Collect a human posture data set, the data set includes a human body image and corresponding human body key point coordinate annotations, take the human body image as input data, take the corresponding human body key point coordinate annotations as output data, constitute a training sample, use the training sample to train the Attention-LSTM network, use the mean square error of the human body key point coordinates as the loss function for training the human posture estimation model, and take minimizing the value of the loss function as the training goal. When the loss function reaches convergence, the training is completed, and a trained human posture estimation model is obtained;

[0020] The specific method of obtaining the human body posture information in the video stream, calculating the abnormal behavior data, and designing an abnormal event judgment function for judging whether an abnormal event occurs based on the abnormal behavior data is:

[0021] Step S240: the abnormal behavior data includes the static time of the human body, the height difference of the center of gravity, the descending speed and acceleration of the center of gravity, the change of the angle of the human body falling and the angular velocity, the average key point density, and the average movement amplitude of all key points of the human body; the abnormal events include abnormal fainting events, abnormal gathering events, and abnormal limb movements, and the abnormal event judgment function includes abnormal fainting event judgment function, abnormal gathering event judgment function, and abnormal limb movement judgment function;

[0022] Step S250: Setting the static threshold , calculate the inter-frame displacement of human key points, and for the kth human key point, calculate its inter-frame displacement between the tth frame and the t+1th frame When the inter-frame displacement of a human key point is less than the static threshold, the human key point is considered to be in a static state. Keep still within the frame for a period of , and the static time exceeds the preset time threshold , then the person is considered to be in a stationary state, where is the frame interval time;

[0023] Step S260: Calculate the coordinates of the center of gravity of the human body according to the coordinates of the key points of the human body , calculate the height difference between the center of gravity of the human body in the tth frame and the t-1th frame , calculate the center of gravity descent speed and acceleration , calculate the change in the angle of the human body falling to the ground and angular velocity ;

[0024] Step S270: Divide the video stream into Grid, calculate the density of key points of the human body in each grid , get the average key point density of the entire picture , set the density threshold , define the abnormal aggregation event judgment function as: when the time threshold When the internal average key point density is continuously greater than the density threshold, it is determined that an abnormal aggregation event has occurred;

[0025] Step S280: Calculate the kth human key point in the continuous Average motion within a frame , calculate the average movement amplitude of all human key points , set the amplitude threshold ,The abnormal limb movement judgment function is defined as: when the average movement amplitude of all human key points is greater than the amplitude threshold, it is determined that an abnormal limb movement has occurred;

[0026] Step S290: Setting the center of gravity height change threshold, center of gravity descent speed threshold and acceleration threshold, falling angle change threshold and angular velocity threshold, setting condition 1 as follows: The center of gravity height difference within the frame is continuously less than the negative center of gravity height change threshold. Condition 2 is: The center of gravity descent speed within the frame is continuously greater than the center of gravity descent speed threshold and the acceleration is continuously greater than the acceleration threshold. Condition three is: The falling angle change in the frame is continuously greater than the falling angle change threshold and the angular velocity is continuously greater than the angular velocity threshold. Condition 4 is: All human key points are continuously The frame remains still and the stillness duration exceeds the preset time threshold ; Design an abnormal fainting event judgment function including the above conditions one, two, three, and four. When any three of the four conditions are met, it is determined that an abnormal fainting event has occurred;

[0027] The specific method of constructing a campus security knowledge graph based on campus multi-system data and constructing a graph neural network model based on the campus security knowledge graph is:

[0028] Step S310: define the top-level concepts of the campus security knowledge graph to form an ontology architecture;

[0029] Step S320: extract different entities, entity attributes, and relationships between entities from the multi-system database of the campus according to the top-level concepts, use the entities as nodes of the graph and the relationships as edges of the graph, and construct a campus security knowledge graph;

[0030] Step S330: Build a graph neural network model based on the campus security knowledge graph. The graph neural network model includes a graph convolution layer and a graph attention layer. The graph convolution layer is used to perform weighted summation on the embedded representations of the node's neighbors. The weight of the weighted summation is determined by the edge weights between the nodes. Decide;

[0031] Step S340: Introduce the attention mechanism based on the graph convolution layer to obtain the graph attention layer, and calculate the attention weights of different neighbor nodes j to node i , based on the attention weight, we get the embedding representation of node i ;

[0032] Step S350: train the graph neural network model in an unsupervised manner, construct a triplet (head node, relationship, tail node) based on the relationship between nodes in the campus security knowledge graph, use the triplet as a positive sample, construct a negative sample through negative sampling, use the positive sample and the negative sample as training data, use an unsupervised training method, use the nodes in the campus security knowledge graph as input, and use the embedded representation of the node as output;

[0033] Step S360: Taking maximizing the likelihood probability of positive samples and minimizing the likelihood probability of negative samples as the training objectives, the loss function of the training graph neural network model is obtained based on the likelihood probability of positive samples and the likelihood probability of negative samples. , train the graph neural network model and minimize the loss function. When the loss function converges, the trained graph neural network model is obtained;

[0034] When an abnormal event occurs, the specific method of using the graph neural network model to learn the embedded representation of the abnormal event entity in the abnormal event and analyze the reasoning cause of the abnormal event is:

[0035] Step S370: when an abnormal event occurs, extract the abnormal event entity of the abnormal event and input it into the graph neural network model, and output the embedded representation of the abnormal event entity;

[0036] Step S380: In the campus security knowledge graph, with the node where the abnormal event entity is located as the center, search for other entities with the smallest inner product with the abnormal event entity in the embedding space of the campus security knowledge graph, select the first K* entities with the smallest inner product as candidate causes, and calculate their semantic relevance probability based on the embedding vectors of the abnormal event entity and the candidate cause. , integrating semantically relevant probabilities and corresponding candidate causes into the inferred causes of abnormal events;

[0037] The specific method of obtaining a sensor combination based on the abnormal audio signal, calculating the signal similarity between each sensor pair in the sensor combination, and obtaining the candidate sound source position with the highest matching score according to the signal similarity calculation, and using it as the occurrence position of the abnormal event is as follows:

[0038] Step S410: Obtain the short-term energy of each sensor when an abnormal event occurs and signal-to-noise ratio , select sensors whose short-time energy is greater than the energy threshold and whose signal-to-noise ratio is greater than the signal-to-noise ratio threshold as the sensor combination;

[0039] Step S420: For the sensor pair in the sensor combination , the signal similarity between sensor pairs is calculated based on their generalized cross-correlation function at the time of abnormal event ;

[0040] Step S430: Calculate the matching score of the candidate sound source position P by accumulating the signal similarity of each pair of sensors ;

[0041] Step S440: taking the candidate sound source position P with the largest matching score as the location where the abnormal event occurred.

[0042] One aspect of the present application provides a campus security management and control system integrating multiple systems, including:

[0043] The video and audio acquisition module is used to arrange high-definition cameras and sensor arrays in the campus area to collect video streams and audio signals. When an abnormal audio signal is captured, the video streams before and after the abnormality occurs are obtained;

[0044] An abnormal event judgment module is used to adopt a human posture estimation model based on an attention mechanism to obtain human posture information in a video stream, calculate abnormal behavior data, and design an abnormal event judgment function based on the abnormal behavior data to determine whether an abnormal event has occurred;

[0045] The abnormal cause reasoning module is used to build a campus security knowledge graph based on campus multi-system data, and a graph neural network model based on the campus security knowledge graph. When an abnormal event occurs, the graph neural network model is used to learn the embedded representation of the abnormal event entity in the abnormal event and analyze the reasoning cause of the abnormal event;

[0046] An occurrence position calculation module is used to obtain a sensor combination based on the abnormal audio signal, calculate the signal similarity between each sensor pair in the sensor combination, and obtain the candidate sound source position with the highest matching score based on the signal similarity calculation, and use it as the occurrence position of the abnormal event;

[0047] The security information receiving module is used to send the location of the abnormal event and the reasoning cause of the abnormal event to the campus security personnel.

[0048] One aspect of the present application provides a readable storage medium, which stores a computer program, and the computer program is suitable for loading by a processor to execute the steps in a campus security management and control method integrating multiple systems.

[0049] Compared with the prior art, the campus security control method and system integrating multiple systems proposed in this application have the following advantages:

[0050] This application adopts a multimodal perception method of cameras and sensor arrays, which can capture more comprehensive abnormal event information, especially by triggering abnormal event location detection through analysis of audio signals, which can make up for the shortcomings of single video analysis and improve the sensitivity of early warning.

[0051] Traditional video analysis methods do not have high enough accuracy in estimating human posture, and it is difficult to accurately judge abnormal changes in human posture. However, this application uses a human posture estimation model enhanced by an attention mechanism, which can more finely characterize the motion characteristics of different parts of the human body, and design a more scientific abnormal event judgment standard through an abnormal event judgment function.

[0052] This application realizes the semantic fusion of multi-system data on campus through knowledge graph technology, and trains a graph neural network model on this basis to perform event cause reasoning. It is a new paradigm for campus data fusion and utilization.

[0053] This application innovatively proposes a sound source localization method based on sensor signal similarity, which can solve the problem that video analysis is easily disturbed by factors such as occlusion and lighting, and is particularly suitable for situations with limited vision in campus environments.

[0054] This application integrates data from multiple campus systems, monitors abnormal events in real time, and locates abnormal events, forming a complete solution from data perception, feature extraction, event identification to positioning and alarm, providing new ideas and methods for improving the level of campus safety management. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 A method flow chart of the campus security management and control method integrating multiple systems provided in this application;

[0056] Figure 2A flow chart of the reasoning cause analysis method for abnormal events provided in this application;

[0057] Figure 3 A flow chart of a method for estimating the location of an abnormal event provided in this application;

[0058] Figure 4 Functional module diagram of the campus security management and control system integrating multiple systems provided for this application. DETAILED DESCRIPTION

[0059] In order to better understand the present application, various aspects of the present application will be described in more detail with reference to the accompanying drawings. It should be understood that these detailed descriptions are only descriptions of exemplary embodiments of the present application and do not limit the scope of the present application in any way. Throughout the specification, the same reference numerals refer to the same elements. The expression "and / or" includes any and all combinations of one or more of the associated listed items.

[0060] In the accompanying drawings, the size, dimensions and shapes of the elements have been slightly adjusted for ease of illustration. The drawings are for illustration only and are not strictly drawn to scale. As used herein, the terms "substantially", "approximately" and similar terms are used as terms of approximation, not as terms of degree, and are intended to illustrate the inherent deviations in measurements or calculations that will be recognized by those of ordinary skill in the art. In addition, in this application, the order in which the steps are described does not necessarily represent the order in which these processes occur in actual operation, unless otherwise specified or can be derived from the context.

[0061] It should also be understood that expressions such as "including", "comprising", "having", "containing" and / or "comprising" are open rather than closed expressions in this specification, which indicate the presence of the stated features, elements and / or components, but do not exclude the presence of one or more other features, elements, components and / or combinations thereof. In addition, when expressions such as "at least one of..." appear after a list of listed features, they modify the entire list of features rather than just the individual elements in the list. In addition, when describing embodiments of the present application, "may" is used to mean "one or more embodiments of the present application". And, the term "exemplary" is intended to refer to an example or illustration.

[0062] Unless otherwise defined, all words (including engineering terms and scientific and technological terms) used in this article have the same meaning as those commonly understood by ordinary technicians in the field to which this application belongs. It should also be understood that unless there is a clear explanation in this application, words defined in commonly used dictionaries should be interpreted as having the same meaning as their meaning in the context of the relevant technology, and should not be interpreted in an idealized or overly formal sense.

[0063] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0064] Example 1

[0065] like Figure 1 As shown, the campus security management method integrating multiple systems provided by this application includes:

[0066] Step S100: deploying high-definition cameras and sensor arrays in the campus area to collect video streams and audio signals. When an abnormal audio signal is captured, the video streams before and after the abnormality occurs are obtained;

[0067] The specific method of arranging high-definition cameras and sensor arrays in the campus area to collect video streams and audio signals, and when an abnormal audio signal is captured, obtaining the video streams before and after the abnormality occurs is as follows: deploying high-definition cameras and sensor arrays in the campus area, arranging the sensor arrays in a uniformly distributed topological structure near the cameras, and the sensor spacing of the sensor arrays satisfy: , where c is the speed of sound, is the maximum frequency of the audio signal; when the sensor array is abnormal When an abnormal audio signal is captured, obtain the time when the abnormality occurs Duration Video stream;

[0068] Said The value of is set by those skilled in the art based on experience and refers to the duration of an abnormal event, which is usually a short time window;

[0069] Step S200: using a human posture estimation model based on an attention mechanism to obtain human posture information in a video stream, calculating abnormal behavior data, and designing an abnormal event judgment function for judging whether an abnormal event occurs based on the abnormal behavior data;

[0070] The specific method for obtaining the human posture estimation model based on the attention mechanism is:

[0071] Step S210: Use the human body posture estimation model based on the attention mechanism to estimate the human body posture, extract the spatial coordinates of the key points of the human body, and introduce the attention mechanism in the time dimension to weightedly aggregate the human body posture features of different frames. The attention weight of the tth frame is ;

[0072] The calculation formula of the attention weight of the t-th frame is: , ,in, is the attention score of the t-th frame, , are the weight matrix and bias term of the attention mechanism in the time dimension, is the hidden state of the tth frame, T1 represents the number of frames in the input sequence, represents the attention score of the i1th frame;

[0073] Step S220: In the spatial dimension, an attention mechanism is introduced to extract feature vectors of different parts of the human body. The attention weight of part j1 is ;

[0074] The calculation formula of the attention weight of the part j1 is: , ,in, is the attention score of part j1, , are the weight matrix and bias term of the attention mechanism in the spatial dimension, represents the feature vector of part j1, represents the attention score of part k1, and J1 represents the total number of all parts;

[0075] Step S230: The human posture estimation model based on the attention mechanism uses the Attention-LSTM network as the initial network, including a CNN backbone network, an attention mechanism and an LSTM timing module, obtains continuous frames of human body images based on the video stream as input data, uses the human body key points of each frame of the human body image as output data, and obtains human body posture information estimated by the Attention-LSTM network according to the spatial position of the human body key points, the relative position and distance between the human body key points, and the motion trajectory of the human body key points. The motion trajectory of the human body key points is obtained by connecting the coordinates of the same human body key points in human body images of different frames;

[0076] The training method of the human posture estimation model is:

[0077] The Attention-LSTM network is used as the initial network. The end of the Attention-LSTM network is connected to the fully connected layer. The hidden state output by the LSTM timing module is mapped to the two-dimensional coordinates of the key points of the human body. The output of the entire model is the coordinate sequence of the key points of the human body corresponding to each frame of the human body image.

[0078] Collect a human posture data set, the data set includes a human body image and corresponding human body key point coordinate annotations, take the human body image as input data, take the corresponding human body key point coordinate annotations as output data, constitute a training sample, use the training sample to train the Attention-LSTM network, use the mean square error of the human body key point coordinates as the loss function for training the human posture estimation model, and take minimizing the value of the loss function as the training goal. When the loss function reaches convergence, the training is completed, and a trained human posture estimation model is obtained;

[0079] Among them, the CNN backbone network is used to extract high-level feature representations of human body images. A pre-trained deep convolutional neural network is used, the last fully connected layer is removed, the convolutional layer and the pooling layer are retained, a single-frame human body image is used as input, and a feature map is output;

[0080] The attention mechanism is used to adaptively focus on the key parts of the human body posture. The attention weight is calculated according to the content of the feature map, and the feature map is weighted combined. The feature map output by the CNN backbone network is used as input, and the weighted combined attention feature map is used as output, and its size is the same as the feature map.

[0081] The LSTM timing module is used to model the evolution and dependency of human posture in the time dimension. It uses a long short-term memory network, takes the attention feature map sequence output by the attention mechanism as input, and takes the hidden state of each time step as output;

[0082] The specific method of obtaining the human body posture information in the video stream, calculating the abnormal behavior data, and designing an abnormal event judgment function for judging whether an abnormal event occurs based on the abnormal behavior data is:

[0083] Step S240: the abnormal behavior data includes the static time of the human body, the height difference of the center of gravity, the descending speed and acceleration of the center of gravity, the change of the angle of the human body falling and the angular velocity, the average key point density, and the average movement amplitude of all key points of the human body; the abnormal events include abnormal fainting events, abnormal gathering events, and abnormal limb movements, and the abnormal event judgment function includes abnormal fainting event judgment function, abnormal gathering event judgment function, and abnormal limb movement judgment function;

[0084] Step S250: Setting the static threshold , calculate the inter-frame displacement of human key points, and for the kth human key point, calculate its inter-frame displacement between the tth frame and the t+1th frame When the inter-frame displacement of a human key point is less than the static threshold, the human key point is considered to be in a static state. Keep still within the frame for a period of , and the static time exceeds the preset time threshold , then the person is considered to be in a stationary state, where is the frame interval time;

[0085] The calculation formula of the inter-frame displacement is: ,in, , They represent the horizontal and vertical coordinates of the kth human key point in the t+1th frame, respectively. , Respectively represent the horizontal and vertical coordinates of the kth human key point in the tth frame;

[0086] Said and the static threshold are set by those skilled in the art based on experience;

[0087] Step S260: Calculate the coordinates of the center of gravity of the human body according to the coordinates of the key points of the human body , calculate the height difference between the center of gravity of the human body in the tth frame and the t-1th frame , calculate the center of gravity descent speed and acceleration , calculate the change in the angle of the human body falling to the ground and angular velocity ;

[0088] The calculation formula of the coordinates of the center of gravity of the human body is: , where K is the total number of key points of the human body;

[0089] The calculation formula of the height difference of the center of gravity of the human body between the t-th frame and the t-1-th frame is: ,in, , are the ordinates of the center of gravity of the human body in the tth frame and the t+1th frame respectively;

[0090] The calculation formulas for the center of gravity descent speed and acceleration are: , ,in, is the falling speed of the center of gravity at frame t+1;

[0091] The specific method of changing the angle and angular velocity of the human body when falling to the ground is as follows: obtaining the direction vector of the upper body of the human body according to the coordinates of the key points of the human body , calculate the direction vector of the upper body In the tth frame and the Changes in the angle of the body falling between frames and the angular velocity of the upper body ;

[0092] The calculation formula for the change in the angle of the human body falling to the ground is: ,in, is the direction vector of the upper body in frame t+1, is the mold length;

[0093] The calculation formula of the angular velocity of the upper body is: ;

[0094] Furthermore, the direction vector of the upper body Select the vector connecting the midpoint of the shoulder and the midpoint of the hip;

[0095] Step S270: Divide the video stream into Grid, calculate the density of key points of the human body in each grid , get the average key point density of the entire picture , set the density threshold , define the abnormal aggregation event judgment function as: when the time threshold When the internal average key point density is continuously greater than the density threshold, it is determined that an abnormal aggregation event has occurred;

[0096] The calculation formula for the density of the key points of the human body in each grid is: ,in, For Grid The number of key points of the human body, For Grid area;

[0097] The calculation formula of the average key point density of the entire picture is: ;

[0098] Step S280: Calculate the kth human key point in the continuous Average motion within a frame , calculate the average movement amplitude of all human key points , set the amplitude threshold ,The abnormal limb movement judgment function is defined as: when the average movement amplitude of all human key points is greater than the amplitude threshold, it is determined that an abnormal limb movement has occurred;

[0099] The kth human key point is in the continuous The calculation formula for the average motion amplitude within a frame is: ,in, , Respectively represent the human key point k of the t-th frame and the t+1-th frame, Represents the Euclidean distance between two points;

[0100] The average movement amplitude of all human key points The calculation formula is: , where K represents the total number of all human key points;

[0101] Step S290: Setting the center of gravity height change threshold, center of gravity descent speed threshold and acceleration threshold, falling angle change threshold and angular velocity threshold, setting condition 1 as follows: The center of gravity height difference within the frame is continuously less than the negative center of gravity height change threshold. Condition 2 is: The center of gravity descent speed within the frame is continuously greater than the center of gravity descent speed threshold and the acceleration is continuously greater than the acceleration threshold. Condition three is: The falling angle change in the frame is continuously greater than the falling angle change threshold and the angular velocity is continuously greater than the angular velocity threshold. Condition 4 is: All human key points are continuously The frame remains still and the stillness duration exceeds the preset time threshold ; Design an abnormal fainting event judgment function including the above conditions one, two, three, and four. When any three of the four conditions are met, it is determined that an abnormal fainting event has occurred;

[0102] The values ​​of the center of gravity height change threshold, center of gravity drop speed threshold and acceleration threshold, falling angle change threshold and angular velocity threshold are set by those skilled in the art based on experience;

[0103] The above steps can significantly improve the accuracy of abnormal event detection and reduce false positives and false negatives through the optimization design of posture estimation and abnormal event judgment functions. At the same time, multi-dimensional abnormal behavior data also provides richer information for subsequent abnormal event analysis.

[0104] Step S300: construct a campus security knowledge graph based on campus multi-system data, and construct a graph neural network model based on the campus security knowledge graph. When an abnormal event occurs, use the graph neural network model to learn the embedded representation of the abnormal event entity in the abnormal event and analyze the reasoning cause of the abnormal event;

[0105] The specific method of constructing a campus security knowledge graph based on campus multi-system data and constructing a graph neural network model based on the campus security knowledge graph is:

[0106] Step S310: define the top-level concepts of the campus security knowledge graph to form an ontology architecture;

[0107] The top-level concepts include but are not limited to: people, events, and places;

[0108] Step S320: extract different entities, entity attributes, and relationships between entities from the multi-system database of the campus according to the top-level concepts, use the entities as nodes of the graph and the relationships as edges of the graph, and construct a campus security knowledge graph;

[0109] Exemplarily, knowledge elements such as entities, attributes, and relationships are extracted from the business data of multiple systems on campus. For example, personnel entities and their attributes are extracted from the student information system, event and place entities and their attributes are extracted from the campus management system, and the medical history relationships of personnel are extracted from the electronic medical record system. The knowledge elements formed include: personnel entities: "Zhang San", "Li Si", etc., with attributes including name, gender, identity, etc.; event entities: "fainting", "falling", "illness", etc., with attributes including time, location, severity, etc.; place entities: "teaching building A", "student dormitory B", etc., with attributes including name, location, function, etc.; relationships: "Zhang San is studying in the computer department", "Zhang San suffers from hypoglycemia", etc.;

[0110] Step S330: Build a graph neural network model based on the campus security knowledge graph. The graph neural network model includes a graph convolution layer and a graph attention layer. The graph convolution layer is used to perform weighted summation on the embedded representations of the node's neighbors. The weight of the weighted summation is determined by the edge weights between the nodes. Decide;

[0111] The formula for the embedding representation is: ,in, represents the embedding representation of node i at layer l, is the neighbor node set of node i, j is the neighbor node, is the previous layer embedding representation of neighbor node j, is the weight matrix of the lth layer, is the bias term of the lth layer, is the weight of the edge between node i and its neighbor node j, is the activation function;

[0112] The weight of the edge between node i and neighbor node j is used to balance the importance of different nodes, which can usually be simply set to ;

[0113] Step S340: Introduce the attention mechanism based on the graph convolution layer to obtain the graph attention layer, and calculate the attention weights of different neighbor nodes j to node i , based on the attention weight, we get the embedding representation of node i ;

[0114] The calculation formula of the attention weight is: ,in, is the LeakyReLU activation function, is the parameter vector of the attention mechanism, T is the transposed symbol, is the parameter matrix of the attention mechanism, , , are the embedding vectors of node i, neighbor node j, and neighbor node m respectively, and || is the concatenation operation of the embedding vectors;

[0115] The calculation formula of the embedding representation of the node i is: ,in, is the embedding representation of node i based on the attention weight;

[0116] When actually obtaining node embedding representations, we usually choose to use the graph convolution formula or the graph attention formula based on the task requirements and data characteristics. The graph convolution formula has a lower computational complexity and is suitable for large-scale graph data, while the graph attention formula can better capture the relationship between nodes, but has a higher computational complexity.

[0117] Step S350: train the graph neural network model in an unsupervised manner, construct a triplet (head node, relationship, tail node) based on the relationship between nodes in the campus security knowledge graph, use the triplet as a positive sample, construct a negative sample through negative sampling, use the positive sample and the negative sample as training data, use an unsupervised training method, use the nodes in the campus security knowledge graph as input, and use the embedded representation of the node as output;

[0118] Negative sampling refers to randomly replacing the head node or tail node of the positive sample;

[0119] Step S360: Taking maximizing the likelihood probability of positive samples and minimizing the likelihood probability of negative samples as the training objectives, the loss function of the training graph neural network model is obtained based on the likelihood probability of positive samples and the likelihood probability of negative samples. , train the graph neural network model and minimize the loss function. When the loss function converges, the trained graph neural network model is obtained;

[0120] The calculation formula of the loss function is: , where S is the positive sample set, are the head node i and tail node j in the positive sample, is the head node in the negative sample and tail node , Q is the number of negative samples, is the negative sampling probability distribution, For the expected operation;

[0121] In the loss function is the likelihood term of the positive sample, is the likelihood probability term of negative samples;

[0122] The number of negative samples refers to the number of negative samples corresponding to each positive sample;

[0123] When an abnormal event occurs, the specific method of using the graph neural network model to learn the embedded representation of the abnormal event entity in the abnormal event and analyze the reasoning cause of the abnormal event is:

[0124] Step S370: when an abnormal event occurs, extract the abnormal event entity of the abnormal event and input it into the graph neural network model, and output the embedded representation of the abnormal event entity;

[0125] Step S380: In the campus security knowledge graph, with the node where the abnormal event entity is located as the center, search for other entities with the smallest inner product with the abnormal event entity in the embedding space of the campus security knowledge graph, select the first K* entities with the smallest inner product as candidate causes, and calculate their semantic relevance probability based on the embedding vectors of the abnormal event entity and the candidate cause. , integrating semantically relevant probabilities and corresponding candidate causes into the inferred causes of abnormal events;

[0126] The calculation formula of the semantic relevance probability is: ,in, is the embedding vector of the abnormal event entity, is the embedding vector of the candidate cause, Cq is the set of candidate causes, q is the candidate cause, is the embedding vector of candidate reason q;

[0127] Figure 2 A flow chart of the reasoning cause analysis method for abnormal events provided in this application;

[0128] For example, suppose that a "Zhang San fainting incident" occurred on campus. Student Zhang San suddenly fainted and lost consciousness during the break at 3 pm. Zhang San had a history of hypoglycemia and did not eat on time at noon that day. According to the description of the incident, all entities related to the "Zhang San abnormal fainting incident" are obtained in the knowledge graph: the person entity "Zhang San", the place entity "playground", and the event entity "fainting". Using the pre-trained graph neural network model, the abnormal event entity of the "Zhang San abnormal fainting incident" is input into the model, and the embedded representation of the event entity is obtained according to the relevant related attributes. With the embedded representation as the center, other entities related to its semantics are searched in the embedding space of the knowledge graph, such as: the person entity "Zhang San" and the event entity "history of hypoglycemia", the event entity "fainting" and the event entity "not eating on time", the behavior entity "violent exercise" and the place entity "playground". Identify "history of hypoglycemia" and "failure to eat on time" in the semantically related entities as candidate causes, denoted as cause1 and cause2 respectively, and represent "fainting" as event. Calculate the semantic relevance probability of them causing "fainting" P(cause1|event) = 0.7, P(cause2|event) = 0.3, and output the final inference cause of "fainting": "history of hypoglycemia" is the most likely cause of "fainting" with a probability of 70%; "failure to eat on time" is also a potential cause, but the possibility is relatively low, with a probability of 30%.

[0129] The above steps can analyze the causes of abnormal events more comprehensively and deeply by integrating multi-dimensional semantic information such as personnel, environment, and activities on campus, so that event alarms can be upgraded from simple event notifications to situational intelligent warnings, providing more targeted reference information for campus security personnel's decision-making.

[0130] Step S400: obtaining a sensor combination based on the abnormal audio signal, calculating the signal similarity between each sensor pair in the sensor combination, and obtaining a candidate sound source position with the highest matching score according to the signal similarity calculation, and taking it as the occurrence position of the abnormal event;

[0131] The specific method of obtaining a sensor combination based on the abnormal audio signal, calculating the signal similarity between each sensor pair in the sensor combination, and obtaining the candidate sound source position with the highest matching score according to the signal similarity calculation, and using it as the occurrence position of the abnormal event is as follows:

[0132] Step S410: Obtain the short-term energy of each sensor when an abnormal event occurs and signal-to-noise ratio , select sensors whose short-time energy is greater than the energy threshold and whose signal-to-noise ratio is greater than the signal-to-noise ratio threshold as the sensor combination;

[0133] The calculation formula of the short-time energy is: ,in, is the short-time energy of the ath sensor, The time when the abnormality occurs. is the duration of the abnormality. is the abnormal audio signal of the ath sensor;

[0134] The calculation formula of the signal-to-noise ratio is: ,in, is the signal-to-noise ratio of the ath sensor, is the noise signal before and after the abnormal event;

[0135] The energy threshold and signal-to-noise ratio threshold are set by those skilled in the art according to actual needs;

[0136] Step S420: For the sensor pair in the sensor combination , the signal similarity between sensor pairs is calculated based on their generalized cross-correlation function at the time of abnormal event ;

[0137] The calculation formula of the generalized cross-correlation function is: ,in, represents the propagation delay of the audio signal from sensor ai to sensor aj, , Respectively represent the signals of sensors ai and aj in the frequency domain, represents the complex conjugate of the audio signal from sensor aj, is a complex number, where r is the imaginary unit and w is the frequency;

[0138] Step S430: Calculate the matching score of the candidate sound source position P by accumulating the signal similarity of each pair of sensors ;

[0139] The calculation formula for the matching score of the candidate sound source position P is: , where M is the sensor combination, is the propagation delay of the audio signal from sensor ai to sensor aj at a given candidate sound source position P;

[0140] Step S440: taking the candidate sound source position P with the largest matching score as the occurrence position of the abnormal event;

[0141] like Figure 3 As shown, it is a flow chart of the method for estimating the occurrence location of an abnormal event provided by the present application;

[0142] Compared with single video positioning, the above steps integrate the spatial distribution information of the audio signal, which can determine the location of abnormal events more quickly and accurately, which is of great significance for campus security personnel to arrive at the scene in time and implement rescue efficiently.

[0143] Step S500: sending the location of the abnormal event and the reasoning cause of the abnormal event to campus security personnel;

[0144] The specific method of sending the location of the abnormal event and the inferred cause of the abnormal event to the campus security personnel is: organize the location of the abnormal event and the inferred cause of the abnormal event into alarm information, send it to the campus security personnel through the campus communication channel, and the campus security personnel respond in time according to the received information.

[0145] Example 2

[0146] like Figure 4 As shown, the campus security management and control system integrating multiple systems provided by this application includes:

[0147] The video and audio acquisition module is used to arrange high-definition cameras and sensor arrays in the campus area to collect video streams and audio signals. When an abnormal audio signal is captured, the video streams before and after the abnormality occurs are obtained;

[0148] An abnormal event judgment module is used to adopt a human posture estimation model based on an attention mechanism to obtain human posture information in a video stream, calculate abnormal behavior data, and design an abnormal event judgment function based on the abnormal behavior data to determine whether an abnormal event has occurred;

[0149] The abnormal cause reasoning module is used to build a campus security knowledge graph based on campus multi-system data, and a graph neural network model based on the campus security knowledge graph. When an abnormal event occurs, the graph neural network model is used to learn the embedded representation of the abnormal event entity in the abnormal event and analyze the reasoning cause of the abnormal event;

[0150] An occurrence position calculation module is used to obtain a sensor combination based on the abnormal audio signal, calculate the signal similarity between each sensor pair in the sensor combination, and obtain the candidate sound source position with the highest matching score based on the signal similarity calculation, and use it as the occurrence position of the abnormal event;

[0151] The security information receiving module is used to send the location of the abnormal event and the reasoning cause of the abnormal event to the campus security personnel.

[0152] Example 3

[0153] According to one embodiment of the present application, a computer-readable storage medium is also provided. Computer-readable instructions are stored on the computer-readable storage medium. When the computer-readable instructions are executed by the processor, the campus safety management and control method integrating multiple systems according to the embodiment of the present application can be executed. The storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory (cache). Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0154] In addition, according to an embodiment of the present application, the process described above may be implemented as a computer software program. For example, the present application provides a non-temporary machine-readable storage medium, which stores machine-readable instructions, and the machine-readable instructions can be executed by a processor to execute instructions corresponding to the method steps provided in the present application, for example: arranging high-definition cameras and sensor arrays in the campus area, collecting video streams and audio signals, and when an abnormal audio signal is captured, obtaining the video streams before and after the abnormality occurs; using a human posture estimation model based on an attention mechanism to obtain human posture information in the video stream, calculate abnormal behavior data, and design an abnormal event judgment function based on the abnormal behavior data to determine whether an abnormal event has occurred; constructing a campus security knowledge graph based on campus multi-system data, and constructing a graph neural network model based on the campus security knowledge graph. When an abnormal event occurs, the graph neural network model is used to learn the embedded representation of the abnormal event entity in the abnormal event, and analyze the reasoning cause of the abnormal event; obtaining a sensor combination based on the abnormal audio signal, calculating the signal similarity between each sensor pair in the sensor combination, and obtaining the candidate sound source position with the highest matching score based on the signal similarity calculation, and using it as the occurrence location of the abnormal event; sending the occurrence location of the abnormal event and the reasoning cause of the abnormal event to campus security personnel. When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present application are performed.

[0155] The methods, systems, and storage media of the present application may be implemented in many ways. For example, the methods, systems, and storage media of the present application may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-mentioned sequence of steps for the method is for illustration only, and the method steps of the present application are not limited to the sequence specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present application may also be implemented as programs recorded in a recording medium, which include machine-readable instructions for implementing the methods according to the present application. Therefore, the present application also covers recording media storing programs for executing the methods according to the present application.

[0156] In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive redundancy.

[0157] The specific implementation modes as described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation mode of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A campus security management and control method integrating multiple systems, characterized in that: include: High-definition cameras and sensor arrays are deployed in the campus area to collect video streams and audio signals. When an abnormal audio signal is captured, the video streams before and after the abnormality occurs are obtained; A human posture estimation model based on the attention mechanism is used to obtain human posture information in the video stream, calculate abnormal behavior data, and design an abnormal event judgment function based on the abnormal behavior data to determine whether an abnormal event has occurred; Build a campus security knowledge graph based on campus multi-system data, and build a graph neural network model based on the campus security knowledge graph. When an abnormal event occurs, use the graph neural network model to learn the embedded representation of the abnormal event entity in the abnormal event and analyze the reasoning cause of the abnormal event; Based on the abnormal audio signal, a sensor combination is obtained, the signal similarity between each sensor pair in the sensor combination is calculated, and the candidate sound source position with the highest matching score is calculated according to the signal similarity, which is used as the occurrence position of the abnormal event; The location of the abnormal event and the reasoning cause of the abnormal event are sent to campus security personnel.

2. The campus security management and control method integrating multiple systems as claimed in claim 1, characterized in that: The specific method of arranging high-definition cameras and sensor arrays in the campus area to collect video streams and audio signals, and when an abnormal audio signal is captured, obtaining the video streams before and after the abnormality occurs is as follows: deploying high-definition cameras and sensor arrays in the campus area, arranging the sensor arrays in a uniformly distributed topological structure near the cameras, and the sensor spacing of the sensor arrays satisfy: , where c is the speed of sound, is the maximum frequency of the audio signal; when the sensor array is abnormal When an abnormal audio signal is captured, obtain the time when the abnormality occurs Duration video stream.

3. The campus security management and control method integrating multiple systems as claimed in claim 2 is characterized in that: The specific method for obtaining the human posture estimation model based on the attention mechanism is: The human posture estimation model based on the attention mechanism is used to estimate the human posture and extract the spatial coordinates of the key points of the human body. In the time dimension, the attention mechanism is introduced to weightedly aggregate the human posture features of different frames. The attention weight of the tth frame is ; In the spatial dimension, the attention mechanism is introduced to extract the feature vectors of different parts of the human body. The attention weight of part j1 is ; The human posture estimation model based on the attention mechanism uses the Attention-LSTM network as the initial network, including the CNN backbone network, the attention mechanism and the LSTM timing module, obtains continuous frames of human body images based on the video stream as input data, uses the human body key points of each frame of the human body image as output data, and obtains the human body posture information estimated by the Attention-LSTM network according to the spatial position of the human body key points, the relative position and distance between the human body key points, and the motion trajectory of the human body key points. The motion trajectory of the human body key points is obtained by connecting the coordinates of the same human body key points in human body images of different frames; The training method of the human posture estimation model is: The Attention-LSTM network is used as the initial network. The end of the Attention-LSTM network is connected to the fully connected layer. The hidden state output by the LSTM timing module is mapped to the two-dimensional coordinates of the key points of the human body. The output of the entire model is the coordinate sequence of the key points of the human body corresponding to each frame of the human body image. A human posture dataset is collected, where the dataset includes human body images and corresponding human body key point coordinate annotations. The human body images are used as input data, and the corresponding human body key point coordinate annotations are used as output data to form training samples. The Attention-LSTM network is trained using the training samples, and the mean square error of the human body key point coordinates is used as the loss function for training the human posture estimation model. The value of minimizing the loss function is used as the training goal. When the loss function converges, the training is completed, and a trained human posture estimation model is obtained.

4. The campus security management and control method integrating multiple systems as claimed in claim 3 is characterized in that: The specific method of obtaining the human body posture information in the video stream, calculating the abnormal behavior data, and designing an abnormal event judgment function for judging whether an abnormal event occurs based on the abnormal behavior data is: The abnormal behavior data include the static time of the human body, the height difference of the center of gravity, the descending speed and acceleration of the center of gravity, the change of the angle of the human body falling and the angular velocity, the average density of key points, and the average movement amplitude of all key points of the human body; the abnormal events include abnormal fainting events, abnormal gathering events, and abnormal limb movements, and the abnormal event judgment function includes abnormal fainting event judgment function, abnormal gathering event judgment function, and abnormal limb movement judgment function; Setting the Inactivity Threshold , calculate the inter-frame displacement of human key points, and for the kth human key point, calculate its inter-frame displacement between the tth frame and the t+1th frame When the inter-frame displacement of a human key point is less than the static threshold, the human key point is considered to be in a static state. Keep still within the frame for a period of , and the static time exceeds the preset time threshold , then the person is considered to be in a stationary state, where is the frame interval time; Calculate the coordinates of the center of gravity of the human body based on the coordinates of the key points of the human body , calculate the height difference between the center of gravity of the human body in the tth frame and the t-1th frame , calculate the center of gravity descent speed and acceleration , calculate the change in the angle of the human body falling to the ground and angular velocity ; Divide the video stream into Grid, calculate the density of key points of the human body in each grid , get the average key point density of the entire picture , set the density threshold , define the abnormal aggregation event judgment function as: when the time threshold When the internal average key point density is continuously greater than the density threshold, it is determined that an abnormal aggregation event has occurred; Calculate the kth human key point in the continuous Average motion within a frame , calculate the average movement amplitude of all human key points , set the amplitude threshold ,The abnormal limb movement judgment function is defined as: when the average movement amplitude of all human key points is greater than the amplitude threshold, it is determined that an abnormal limb movement has occurred; Set the center of gravity height change threshold, center of gravity descent speed threshold and acceleration threshold, falling angle change threshold and angular velocity threshold. Set condition 1 as follows: The center of gravity height difference within the frame is continuously less than the negative center of gravity height change threshold. Condition 2 is: The center of gravity descent speed within the frame is continuously greater than the center of gravity descent speed threshold and the acceleration is continuously greater than the acceleration threshold. Condition three is: The falling angle change in the frame is continuously greater than the falling angle change threshold and the angular velocity is continuously greater than the angular velocity threshold. Condition 4 is: All human key points are continuously The frame remains still and the stillness duration exceeds the preset time threshold ; Design an abnormal fainting event judgment function including the above conditions one, two, three, and four. When any three of the four conditions are met, it is determined that an abnormal fainting event has occurred.

5. The campus security management and control method integrating multiple systems as claimed in claim 4 is characterized in that: The specific method of constructing a campus security knowledge graph based on campus multi-system data and constructing a graph neural network model based on the campus security knowledge graph is: Define the top-level concepts of the campus security knowledge graph and form an ontology architecture; Extract different entities, entity attributes, and relationships between entities from the multi-system database of the campus based on the top-level concepts, use entities as graph nodes and relationships as graph edges to build a campus security knowledge graph; A graph neural network model is constructed based on the campus security knowledge graph. The graph neural network model includes a graph convolution layer and a graph attention layer. The graph convolution layer is used to perform weighted summation on the embedded representations of the neighbors of the node. The weight of the weighted summation is determined by the edge weights between the nodes. Decide; The attention mechanism is introduced on the basis of the graph convolution layer to obtain the graph attention layer, which calculates the attention weights of different neighbor nodes j on node i , based on the attention weight, we get the embedding representation of node i ; The graph neural network model is trained in an unsupervised manner. The relationship between nodes in the campus security knowledge graph constitutes a triple (head node, relationship, tail node). The triple is used as a positive sample, and a negative sample is constructed through negative sampling. The positive sample and the negative sample are used as training data. An unsupervised training method is adopted, and the nodes in the campus security knowledge graph are used as input, and the embedded representation of the node is used as output. The training objectives are to maximize the likelihood of positive samples and minimize the likelihood of negative samples. The loss function of the training graph neural network model is obtained based on the likelihood of positive samples and the likelihood of negative samples. , train the graph neural network model and minimize the loss function. When the loss function converges, the trained graph neural network model is obtained.

6. The campus security management and control method integrating multiple systems as claimed in claim 5, characterized in that: When an abnormal event occurs, the specific method of using the graph neural network model to learn the embedded representation of the abnormal event entity in the abnormal event and analyze the reasoning cause of the abnormal event is: When an abnormal event occurs, the abnormal event entity of the abnormal event is extracted and input into the graph neural network model, and the embedded representation of the abnormal event entity is output; In the campus security knowledge graph, with the node where the abnormal event entity is located as the center, search for other entities with the smallest inner product with the abnormal event entity in the embedding space of the campus security knowledge graph, select the top K* entities with the smallest inner product as candidate causes, and calculate their semantic relevance probabilities based on the embedding vectors of the abnormal event entity and the candidate causes. , the semantically relevant probabilities and the corresponding candidate causes are integrated into the inferred causes of abnormal events.

7. The campus security management and control method integrating multiple systems as claimed in claim 6, characterized in that: The specific method of obtaining a sensor combination based on the abnormal audio signal, calculating the signal similarity between each sensor pair in the sensor combination, and obtaining the candidate sound source position with the highest matching score according to the signal similarity calculation, and using it as the occurrence position of the abnormal event is as follows: Obtain the short-term energy of each sensor when an abnormal event occurs and signal-to-noise ratio , select sensors whose short-time energy is greater than the energy threshold and whose signal-to-noise ratio is greater than the signal-to-noise ratio threshold as the sensor combination; For sensor pairs in a sensor combination , the signal similarity between sensor pairs is calculated based on their generalized cross-correlation function at the time of abnormal event ; The matching score of the candidate sound source position P is calculated by accumulating the signal similarity of each pair of sensors ; The candidate sound source position P with the largest matching score is taken as the occurrence position of the abnormal event.

8. A campus safety management and control system integrating multiple systems, which is used to implement the campus safety management and control method integrating multiple systems as described in any one of claims 1 to 7, characterized in that: include: The video and audio acquisition module is used to arrange high-definition cameras and sensor arrays in the campus area to collect video streams and audio signals. When an abnormal audio signal is captured, the video streams before and after the abnormality occurs are obtained; An abnormal event judgment module is used to adopt a human posture estimation model based on an attention mechanism to obtain human posture information in a video stream, calculate abnormal behavior data, and design an abnormal event judgment function based on the abnormal behavior data to determine whether an abnormal event has occurred; The abnormal cause reasoning module is used to build a campus security knowledge graph based on campus multi-system data, and a graph neural network model based on the campus security knowledge graph. When an abnormal event occurs, the graph neural network model is used to learn the embedded representation of the abnormal event entity in the abnormal event and analyze the reasoning cause of the abnormal event; An occurrence position calculation module is used to obtain a sensor combination based on the abnormal audio signal, calculate the signal similarity between each sensor pair in the sensor combination, and obtain the candidate sound source position with the highest matching score based on the signal similarity calculation, and use it as the occurrence position of the abnormal event; The security information receiving module is used to send the location of the abnormal event and the reasoning cause of the abnormal event to the campus security personnel.

9. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which is suitable for being loaded by a processor to execute the steps in the campus security management and control method integrating multiple systems as described in any one of claims 1-7.

Images