Campus security abnormal event detection method, electronic device and computer program product

Abstract

The invention provides a campus security abnormal event detection method, electronic equipment and a computer program product. Video data in a classroom is acquired by using video monitoring equipment, and audio data in the classroom is acquired by using audio monitoring equipment. An edge analysis device is used for carrying out real-time analysis on audio data and judging whether an audio abnormal event occurs or not, and if the audio abnormal event is detected at a target moment, a control center device can obtain video data in a set time period before and after the moment, carry out detailed event analysis on the video data and judge whether the abnormal event really occurs or not. According to the embodiment, double evidences of audio and video are combined, and the accuracy of abnormal event detection is improved. Moreover, according to the embodiment of the invention, the automatic detection of the campus security abnormal event is realized, the requirement of manual monitoring is greatly reduced, the situation of potential safety hazards caused by omission is reduced, and the labor cost is reduced.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and more specifically, to a method for detecting abnormal campus safety events, an electronic device, and a computer program product. Background Art

[0002] At present, school classrooms are generally equipped with advanced surveillance camera systems. These cameras not only have high-definition imaging capabilities, but also often incorporate advanced technologies such as night vision, wide angle and motion capture to ensure that all situations in the classroom can be recorded in all directions and around the clock. The images captured by the surveillance cameras are transmitted to the school's monitoring management center in real time via a high-speed network, forming a centralized monitoring network.

[0003] In the monitoring management center, managers usually sit in front of a large screen that displays surveillance images from various classrooms. They carefully observe these images with their naked eyes, trying to capture any signs that may indicate abnormal events, such as student misbehavior, safety hazards, or sudden accidents. However, this traditional monitoring method has several significant flaws.

[0004] First, it completely relies on the naked eye observation and subjective judgment of managers. Human attention is limited. Staring at the monitoring screen for a long time can easily lead to visual fatigue, which may cause some key information to be missed. Especially in the case of simultaneous monitoring of multiple screens, the risk of such omissions will be further increased. Secondly, the cost of manual monitoring is quite high. With the continuous upgrading and expansion of the monitoring system, the number of personnel required will also increase accordingly, which will bring a heavy economic burden to the school. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a campus safety abnormal event detection method, electronic equipment and computer program product to solve the problem that the existing campus abnormal event monitoring method relies on manual labor, resulting in high labor costs and easy omissions that lead to safety hazards.

[0006] The embodiment of the present application provides a method for detecting abnormal campus security events, which is applied to a monitoring and analysis system, wherein the monitoring and analysis system includes video monitoring equipment, audio monitoring equipment, and edge analysis equipment in a classroom, and a control center device in a control center;

[0007] Methods include:

[0008] Use video surveillance equipment to obtain video data in the classroom, and use audio surveillance equipment to obtain audio data in the classroom;

[0009] Use edge analysis equipment to perform real-time audio event analysis on audio data to determine whether abnormal audio events have occurred;

[0010] If an abnormal audio event occurs at the target time, the control center equipment is used to obtain video data of a set time period before and after the target time and perform video event analysis to determine whether an abnormal event has occurred.

[0011] In the above technical solution, video surveillance equipment is used to obtain video data in the classroom, and audio surveillance equipment is used to obtain audio data in the classroom. The edge analysis device is used to perform real-time analysis on the audio data to determine whether an audio abnormality event has occurred. The edge analysis device can process the audio data in real time and identify possible abnormal events, such as sudden noise, screams, etc. This real-time analysis can quickly respond to potential safety issues. If an audio abnormality event is detected at the target moment, the control center device will obtain the video data of the set time period before and after the moment, and perform detailed event analysis on these video data to determine whether an abnormal event has actually occurred. Through the video analysis within the set time period, it can be further confirmed whether the audio abnormality event does correspond to a certain safety hazard or improper behavior. Therefore, this embodiment combines the dual evidence of audio and video to improve the accuracy of abnormal event detection. In addition, this embodiment realizes the automatic detection of abnormal campus safety events, greatly reduces the need for manual monitoring, reduces the situation of safety hazards caused by omissions, and reduces labor costs.

[0012] In some optional implementations, performing real-time audio event analysis on audio data to determine whether an audio abnormality event occurs includes:

[0013] The audio data is analyzed using an audio analysis model, which is used to extract features from the audio data and perform binary classification; wherein the features in the audio data include at least one of Mel-frequency cepstral coefficients, zero crossing rate, amplitude, spectral roll-off point, spectral flux, spectral entropy, chromaticity features, and pitch.

[0014] In the above technical solution, the audio analysis model is used to extract specific features in the audio data, and based on these features, a binary classification is performed (i.e., whether it is a normal event or an abnormal event). The following is a detailed explanation of these features and their application in audio analysis:

[0015] Mel-Frequency Cepstral Coefficients (MFCC): MFCC is a widely used feature in audio processing that captures the nonlinear frequency perception of the human ear. In abnormal audio event detection, MFCC can help identify different sound patterns, such as screams, quarrels, etc.

[0016] Zero crossing rate: Zero crossing rate refers to the number of times an audio signal crosses the zero point per unit time. It can reflect the frequency component of the audio signal and is very helpful for detecting sudden sound changes (such as explosions).

[0017] Amplitude: Amplitude indicates the strength or loudness of an audio signal. Abnormal audio events are often accompanied by significant changes in amplitude, such as a sudden loud shout or the sound of an object falling.

[0018] Spectral roll-off point: The spectral roll-off point refers to the frequency corresponding to when the power spectrum density of the audio signal drops from the maximum value to a certain threshold. It can reflect the spectral characteristics of the audio signal and is very helpful for distinguishing different types of sounds.

[0019] Spectral flux: Spectral flux is used to measure the rate at which the frequency spectrum of an audio signal changes over time. In abnormal audio event detection, spectral flux can capture the changing trend of sound, such as a gradually increasing scream or a sudden stop in conversation.

[0020] Spectral entropy: Spectral entropy is a measure of the uncertainty of the audio signal spectrum. Abnormal audio events are often accompanied by significant changes in the spectrum, resulting in an increase in spectral entropy.

[0021] Chroma features: Although chroma features are often used in the field of image processing, they may also be useful in some audio analysis (especially after converting the audio signal into a spectrogram). It may be used to capture certain color or texture characteristics of the audio signal, but its specific application in abnormal audio event detection may require further exploration.

[0022] Pitch: Pitch refers to the fundamental frequency or main frequency component of an audio signal. Pitch is an important feature in human voice detection or certain types of sound recognition.

[0023] After extracting the above features, the audio analysis model will use these features for binary classification. Binary classification models include support vector machines, decision trees, random forests, or deep learning models.

[0024] In some optional implementations, obtaining video data of a set time period before and after a target moment and performing video event analysis to determine whether an abnormal event occurs includes:

[0025] Identify whether dangerous objects appear in image frames in video data;

[0026] If dangerous objects are detected, it is determined that an emergency abnormal event has occurred in the classroom.

[0027] In the above technical solution, the method of identifying dangerous objects adopts the target detection algorithm. The target detection model used to identify dangerous objects is a convolutional neural network (CNN). The model needs to be trained in advance to learn the characteristics and appearance of dangerous objects, including knives, flammable items, toxic substances, etc. Once the dangerous objects are identified, the system will determine whether an emergency abnormal event has occurred in the classroom according to the preset rules or thresholds. This determination process may involve comprehensive consideration of multiple factors, such as the type, quantity, location of the dangerous objects and the contextual information of the surrounding environment. Once the dangerous objects are identified and determined to be emergency abnormal events, the system will immediately send an alarm to the control center. The alarm information may include the specific time and location of the event and the types of items that may be involved. After the control center is reviewed by the management personnel, according to the emergency plan formulated by the school, the control center will take corresponding actions, including immediately notifying campus security personnel to go to the scene, evacuating students, and blocking relevant areas.

[0028] In some optional implementations, after identifying whether a dangerous object appears in an image frame in the video data, the method further includes:

[0029] If no dangerous goods are identified, the distribution of people and facial expressions are identified for the image frames in the video data;

[0030] According to the distribution of personnel and facial expressions, the probability of various abnormal events is obtained;

[0031] Determine whether the probability of various abnormal events is greater than the set probability;

[0032] If the probability of an abnormal event of the target category occurring is greater than the set probability, it is determined that an abnormal event of the target category has occurred.

[0033] Among them, the target detection algorithm is used to identify the distribution of people, and the facial expression recognition algorithm is used to identify facial expressions. According to the distribution of people and facial expressions, the probability of abnormal events is obtained by using a graph neural network. The graph neural network inputs the images of the distribution of people and facial expressions, and the graph neural network outputs the probability of each type of abnormal event.

[0034] In the above technical solution, when no dangerous items are identified, the system will turn to analyze the distribution of people and facial expressions in the image frame. Personnel distribution recognition uses target detection algorithms (such as YOLO, Faster R-CNN, etc.) to identify and locate each person in the image, and count their location, number, and possible aggregation. Facial expression recognition uses facial expression recognition algorithms (such as convolutional neural network CNN combined with a model trained by expression recognition data set) to capture and analyze facial expressions in the image, such as happiness, sadness, anger, surprise, etc. Based on the recognition results of personnel distribution and facial expressions, the system will use a graph neural network (Graph Neural Network, GNN) to evaluate the probability of various abnormal events. The input of the graph neural network is the distribution of people in the image frame (which can be represented as a graph structure of nodes and edges, where nodes represent people and edges represent the relationship between people) and the recognition results of facial expressions (which can be used as additional features of nodes). The output of the graph neural network is the probability distribution of various types of abnormal events, including theft, conflicts between students, emotional fluctuations of students, chaos in the classroom, etc. For example, when a theft occurs in a classroom, there are usually few people in the classroom and the people involved look panicked; when a conflict occurs between students in the classroom, the distribution of people in the classroom often presents a crowd of onlookers surrounding the people involved, and the people involved look angry, etc. Therefore, graph neural networks can process complex graph structure data and effectively integrate information on personnel distribution and facial expressions, thereby improving the accuracy and robustness of abnormal event assessment.

[0035] In some optional implementations, after determining that an abnormal event of the target category has occurred, the method further includes:

[0036] Determine whether there are classrooms with more than a set number of people gathering near the target side windows of the teaching building;

[0037] If more than the set number of classrooms have people gathered near the windows on the target side of the teaching building, the location of the abnormal event is determined to be outside the target side of the teaching building.

[0038] In the above technical solution, the system first determines that an abnormal event of the target category has occurred according to the previous steps. Subsequently, the system will determine whether there are more than a set number of classrooms where people gather near the windows on the target side of the teaching building. This set number can be adjusted according to actual conditions to reflect the degree of influence of abnormal events that may occur outside the teaching building on the behavior of students in the classroom. If such a gathering occurs in more than a set number of classrooms, the system determines that the location of the abnormal event is outside the target side of the teaching building, and notifies the relevant management personnel, who can go to the outside of the target side of the teaching building for confirmation. This embodiment can not only determine whether an abnormal event has occurred, but also determine the actual location where the abnormal event occurred, so that the management personnel can go directly to the actual location for processing.

[0039] In some optional implementations, after determining whether more than a set number of classrooms have a situation where people are gathered near the target side windows of the teaching building, the method further includes:

[0040] If the personnel distribution is not concentrated near the target side windows of the teaching building, or if there are less than or equal to the set number of classrooms where the personnel distribution is concentrated near the target side windows of the teaching building, then the location of the abnormal event is determined to be in the corresponding classroom.

[0041] In the above technical solution, if the personnel distribution is not gathered near the target side window of the teaching building (that is, there is no abnormal gathering phenomenon near the window), or although there is gathering near the window, the number of classrooms involved is less than or equal to the set threshold, in this case, the system determines that the abnormal event occurred in the corresponding classroom, and can notify the management personnel to go to the corresponding classroom for processing. This embodiment can not only determine whether an abnormal event occurs, but also determine the actual location where the abnormal event occurs, so that the management personnel can directly go to the actual location for processing.

[0042] In some optional implementations, after determining that the location where the abnormal event occurred is in the corresponding classroom, the method further includes:

[0043] Perform face recognition on image frames in video data to identify people involved in abnormal events;

[0044] Conduct event correlation for abnormal events based on the people involved.

[0045] In the above technical solution, the system first determines that the abnormal event occurred in the corresponding classroom according to the previous steps. Then, the system performs face recognition on the image frames in the video data. Face recognition technology uses deep learning algorithms to determine the identity of the person involved by comparing the facial features in the image frame with the facial information in the database. The people involved may include the initiator, participant or witness of the abnormal event. Based on the determined persons involved, the system will associate the abnormal event. Event association refers to associating the persons involved with the abnormal event and recording their roles and behaviors in the event. This will help the school or relevant departments to conduct in-depth investigation and handling of abnormal events later.

[0046] In some optional implementations, event association is performed on abnormal events based on the persons involved, including:

[0047] Determine the tracing time period based on the type of abnormal event;

[0048] Query the historical abnormal events related to the persons involved in the traceable time period before the abnormal event occurs, and associate the abnormal event with the historical abnormal events.

[0049] In the above technical solution, the system first determines a reasonable tracing time period according to the type of abnormal event (such as violent conflict, bullying, vandalism, theft, etc.), and associates the historical abnormal events related to the persons involved before the event and within the tracing time period. This time period is usually set according to the nature and scope of the abnormal event. The length of the tracing time period may vary due to different types of abnormal events. For example, if an outsider is mixed into the school to commit theft, then the outsider will usually commit theft in multiple classrooms in a short period of time. Then the tracing time period set for the abnormal event of theft can be one day or 12 hours; if the type of abnormal event is bullying, there may be long-term continuous bullying, then the corresponding tracing time period can be one year or two years. By increasing the measures for associating abnormal events, this implementation method provides a more comprehensive and in-depth method for handling abnormal events. This method not only helps schools or relevant departments to more accurately assess the nature and potential risks of abnormal events, but also provides strong support for formulating more effective response measures.

[0050] An electronic device provided in an embodiment of the present application includes: a processor, a network interface, a memory, and a computer program; the computer program is loaded into the memory; and the computer program is executed by the processor to perform any of the above methods.

[0051] A computer program product provided in an embodiment of the present application includes a computer program / instruction, which implements the steps of any of the above methods when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0053] Figure 1 A flowchart of a method for detecting abnormal campus safety events provided in one embodiment of the present application;

[0054] Figure 2 A flow chart of a method for detecting abnormal campus security events provided in another embodiment of the present application. DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0056] Please refer to Figure 1 , Figure 1 A flowchart of a method for detecting abnormal campus security events provided in an embodiment of the present application, the method is applied to a monitoring and analysis system, the monitoring and analysis system includes video monitoring equipment, audio monitoring equipment and edge analysis equipment in a classroom, and a control center device in a control center;

[0057] A campus safety abnormal event detection method of this embodiment specifically includes:

[0058] Step S1, using video monitoring equipment to obtain video data in the classroom, and using audio monitoring equipment to obtain audio data in the classroom;

[0059] Step S2: Use edge analysis equipment to perform real-time audio event analysis on audio data to determine whether an audio abnormality event occurs;

[0060] Step S3: If an abnormal audio event occurs at the target time, the control center equipment is used to obtain video data of a set time period before and after the target time and perform video event analysis to determine whether an abnormal event occurs.

[0061] In the embodiment of the present application, video data in the classroom is obtained by video surveillance equipment, and audio data in the classroom is obtained by audio surveillance equipment. The edge analysis device is used to analyze the audio data in real time to determine whether an audio abnormality event has occurred. The edge analysis device can process the audio data in real time and identify possible abnormal events, such as sudden noise, screams, etc. This real-time analysis can quickly respond to potential safety issues. If an audio abnormality event is detected at the target time, the control center device will obtain the video data of the set time period before and after the moment, and perform detailed event analysis on these video data to determine whether an abnormal event has actually occurred. Through the video analysis within the set time period, it can be further confirmed whether the audio abnormality event does correspond to a certain safety hazard or improper behavior. Therefore, this embodiment combines the dual evidence of audio and video to improve the accuracy of abnormal event detection. In addition, this embodiment realizes the automatic detection of abnormal events in campus safety, greatly reduces the need for manual monitoring, reduces the situation of safety hazards caused by omissions, and reduces labor costs.

[0062] Please refer to Figure 2 , Figure 2 A flow chart of a method for detecting abnormal campus security events provided in another embodiment of the present application.

[0063] In some optional implementations, performing real-time audio event analysis on audio data to determine whether an audio abnormality event occurs includes:

[0064] The audio data is analyzed using an audio analysis model, which is used to extract features from the audio data and perform binary classification; wherein the features in the audio data include at least one of Mel-frequency cepstral coefficients, zero crossing rate, amplitude, spectral roll-off point, spectral flux, spectral entropy, chromaticity features, and pitch.

[0065] In the embodiment of the present application, the audio analysis model is used to extract specific features in the audio data, and a binary classification is performed based on these features (i.e., determining whether it is a normal event or an abnormal event). The following is a detailed explanation of these features and their application in audio analysis:

[0066] Mel-Frequency Cepstral Coefficients (MFCC): MFCC is a widely used feature in audio processing that captures the nonlinear frequency perception of the human ear. In abnormal audio event detection, MFCC can help identify different sound patterns, such as screams, quarrels, etc.

[0067] Zero crossing rate: Zero crossing rate refers to the number of times an audio signal crosses the zero point per unit time. It can reflect the frequency component of the audio signal and is very helpful for detecting sudden sound changes (such as explosions).

[0068] Amplitude: Amplitude indicates the strength or loudness of an audio signal. Abnormal audio events are often accompanied by significant changes in amplitude, such as a sudden loud shout or the sound of an object falling.

[0069] Spectral roll-off point: The spectral roll-off point refers to the frequency corresponding to when the power spectrum density of the audio signal drops from the maximum value to a certain threshold. It can reflect the spectral characteristics of the audio signal and is very helpful for distinguishing different types of sounds.

[0070] Spectral flux: Spectral flux is used to measure the rate at which the frequency spectrum of an audio signal changes over time. In abnormal audio event detection, spectral flux can capture the changing trend of sound, such as a gradually increasing scream or a sudden stop in conversation.

[0071] Spectral entropy: Spectral entropy is a measure of the uncertainty of the audio signal spectrum. Abnormal audio events are often accompanied by significant changes in the spectrum, resulting in an increase in spectral entropy.

[0072] Chroma features: Although chroma features are often used in the field of image processing, they may also be useful in some audio analysis (especially after converting the audio signal into a spectrogram). It may be used to capture certain color or texture characteristics of the audio signal, but its specific application in abnormal audio event detection may require further exploration.

[0073] Pitch: Pitch refers to the fundamental frequency or main frequency component of an audio signal. Pitch is an important feature in human voice detection or certain types of sound recognition.

[0074] After extracting the above features, the audio analysis model will use these features for binary classification. Binary classification models include support vector machines, decision trees, random forests, or deep learning models.

[0075] In some optional implementations, obtaining video data of a set time period before and after a target moment and performing video event analysis to determine whether an abnormal event occurs includes:

[0076] Identify whether dangerous objects appear in image frames in video data;

[0077] If dangerous objects are detected, it is determined that an emergency abnormal event has occurred in the classroom.

[0078] In the embodiment of the present application, the method of identifying dangerous objects adopts a target detection algorithm, and the target detection model used to identify dangerous objects is a convolutional neural network (CNN). The model needs to be trained in advance to learn the characteristics and appearance of dangerous objects, including knives, flammable items, toxic substances, etc. Once the dangerous objects are identified, the system will determine whether an emergency abnormal event has occurred in the classroom according to the preset rules or thresholds. This determination process may involve a comprehensive consideration of multiple factors, such as the type, quantity, location of the dangerous objects and the contextual information of the surrounding environment. Once the dangerous objects are identified and determined to be emergency abnormal events, the system will immediately send an alarm to the control center. The alarm information may include the specific time and location of the event and the types of items that may be involved. After the control center is reviewed by the management personnel, according to the emergency plan formulated by the school, the control center will take corresponding actions, including immediately notifying campus security personnel to go to the scene, evacuating students, and blocking relevant areas.

[0079] In some optional implementations, after identifying whether a dangerous object appears in an image frame in the video data, the method further includes:

[0080] If no dangerous goods are identified, the distribution of people and facial expressions are identified for the image frames in the video data;

[0081] According to the distribution of personnel and facial expressions, the probability of various abnormal events is obtained;

[0082] Determine whether the probability of various abnormal events is greater than the set probability;

[0083] If the probability of an abnormal event of the target category occurring is greater than the set probability, it is determined that an abnormal event of the target category has occurred.

[0084] Among them, the target detection algorithm is used to identify the distribution of people, and the facial expression recognition algorithm is used to identify facial expressions. According to the distribution of people and facial expressions, the probability of abnormal events is obtained by using a graph neural network. The graph neural network inputs the images of the distribution of people and facial expressions, and the graph neural network outputs the probability of each type of abnormal event.

[0085] In the embodiment of the present application, when no dangerous goods are identified, the system will turn to analyze the distribution of people and facial expressions in the image frame. Personnel distribution recognition uses target detection algorithms (such as YOLO, Faster R-CNN, etc.) to identify and locate each person in the image, and count their positions, quantities, and possible aggregation situations. Facial expression recognition uses facial expression recognition algorithms (such as convolutional neural networks CNN combined with models trained by expression recognition data sets) to capture and analyze facial expressions in images, such as happiness, sadness, anger, surprise, etc. Based on the recognition results of personnel distribution and facial expressions, the system will use a graph neural network (Graph Neural Network, GNN) to evaluate the probability of various abnormal events. The graph neural network inputs the distribution of people in the image frame (which can be represented as a graph structure of nodes and edges, where nodes represent people and edges represent the relationship between people) and the recognition results of facial expressions (which can be used as additional features of nodes), and the output of the graph neural network is the probability distribution of various types of abnormal events, which include theft, conflicts between students, emotional fluctuations of students, chaos in the classroom, etc. For example, when a theft occurs in a classroom, there are usually few people in the classroom and the people involved look panicked; when a conflict occurs between students in the classroom, the distribution of people in the classroom often presents a crowd of onlookers surrounding the people involved, and the people involved look angry, etc. Therefore, graph neural networks can process complex graph structure data and effectively integrate information on personnel distribution and facial expressions, thereby improving the accuracy and robustness of abnormal event assessment.

[0086] In some optional implementations, after determining that an abnormal event of the target category has occurred, the method further includes:

[0087] Determine whether there are classrooms with more than a set number of people gathering near the target side windows of the teaching building;

[0088] If more than the set number of classrooms have people gathered near the windows on the target side of the teaching building, the location of the abnormal event is determined to be outside the target side of the teaching building.

[0089] In an embodiment of the present application, the system first determines that an abnormal event of the target category has occurred according to the previous steps. Subsequently, the system will determine whether there are more than a set number of classrooms where people gather near the windows on the target side of the teaching building. This set number can be adjusted according to actual conditions to reflect the degree of influence of abnormal events that may occur outside the teaching building on the behavior of students in the classroom. If such a gathering occurs in more than a set number of classrooms, the system determines that the location of the abnormal event is outside the target side of the teaching building, and notifies the relevant management personnel, who can go to the outside of the target side of the teaching building for confirmation. This embodiment can not only determine whether an abnormal event has occurred, but also determine the actual location where the abnormal event occurred, so that the management personnel can go directly to the actual location for processing.

[0090] Specifically, the judgment of whether there is a gathering of people near the target side window can be: dividing the area close to the target side window in the image, and judging whether the number of people in the area is greater than a set percentage of the total number of people in the image based on the distribution of people, such as 40%. If so, there is a gathering of people; or judging whether the number of people in the area reaches a critical value. The setting of this critical value is related to the number of people that the area can accommodate, such as 20 people. If so, there is a gathering of people.

[0091] In some optional implementations, after determining whether more than a set number of classrooms have a situation where people are gathered near the target side windows of the teaching building, the method further includes:

[0092] If the personnel distribution is not concentrated near the target side windows of the teaching building, or if there are less than or equal to the set number of classrooms where the personnel distribution is concentrated near the target side windows of the teaching building, then the location of the abnormal event is determined to be in the corresponding classroom.

[0093] In the embodiment of the present application, if the personnel distribution is not gathered near the target side window of the teaching building (i.e., there is no abnormal gathering near the window), or if there is gathering near the window but the number of classrooms involved is less than or equal to the set threshold, in this case, the system determines that the abnormal event occurred in the corresponding classroom, and can notify the management personnel to go to the corresponding classroom for processing. This embodiment can not only determine whether an abnormal event occurs, but also determine the actual location where the abnormal event occurs, so that the management personnel can directly go to the actual location for processing.

[0094] In some optional implementations, after determining that the location where the abnormal event occurred is in the corresponding classroom, the method further includes:

[0095] Perform face recognition on image frames in video data to identify people involved in abnormal events;

[0096] Conduct event correlation for abnormal events based on the people involved.

[0097] In an embodiment of the present application, the system first determines that the abnormal event occurred in the corresponding classroom according to the previous steps. Then, the system performs face recognition on the image frames in the video data. Face recognition technology uses a deep learning algorithm to determine the identity of the person involved by comparing the facial features in the image frame with the facial information in the database. The people involved may include the initiator, participant or witness of the abnormal event. Based on the determined persons involved, the system will associate the abnormal event with the event. Event association refers to associating the persons involved with the abnormal event and recording their roles and behaviors in the event. This helps the school or relevant departments to conduct in-depth investigation and handling of abnormal events later.

[0098] In some optional implementations, event association is performed on abnormal events based on the persons involved, including:

[0099] Determine the tracing time period based on the type of abnormal event;

[0100] Query the historical abnormal events related to the persons involved in the traceable time period before the abnormal event occurs, and associate the abnormal event with the historical abnormal events.

[0101] In the embodiment of the present application, the system first determines a reasonable tracing time period according to the type of abnormal event (such as violent conflict, bullying, vandalism, theft, etc.), and associates the historical abnormal events related to the persons involved before the event and within the tracing time period. This time period is usually set according to the nature and scope of the abnormal event. The length of the tracing time period may vary due to different types of abnormal events. For example, if an outsider is mixed into the school to commit theft, then the outsider will usually commit theft in multiple classrooms in a short period of time, then the tracing time period set for the abnormal event of theft can be one day or 12 hours; if the type of abnormal event is bullying, there may be long-term continuous bullying, then the corresponding tracing time period can be one year or two years. By increasing the measures associated with abnormal events, this embodiment provides a more comprehensive and in-depth method for handling abnormal events. This method not only helps schools or relevant departments to more accurately assess the nature and potential risks of abnormal events, but also provides strong support for formulating more effective countermeasures.

[0102] An electronic device provided in an embodiment of the present application includes: a processor, a network interface, a memory, and a computer program; the computer program is loaded into the memory; and the computer program is executed by the processor to perform any of the above methods.

[0103] A computer program product provided in an embodiment of the present application includes a computer program / instruction, which implements the steps of any of the above methods when executed by a processor.

[0104] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0105] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0106] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0107] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0108] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A campus safety abnormal event detection method, characterized in that: The method is applied to a monitoring and analysis system, which includes video monitoring equipment, audio monitoring equipment and edge analysis equipment in a classroom, and a control center equipment in a control center; The method comprises: Using the video monitoring device to obtain video data in the classroom, and using the audio monitoring device to obtain audio data in the classroom; Using the edge analysis device, real-time audio event analysis is performed on the audio data to determine whether an audio abnormality event occurs; If an abnormal audio event occurs at the target time, the control center equipment is used to obtain video data of a set time period before and after the target time and perform video event analysis to determine whether an abnormal event occurs.

2. The method according to claim 1, characterized in that The real-time audio event analysis of the audio data to determine whether an audio abnormality event occurs includes: The audio data is analyzed using an audio analysis model, wherein the audio analysis model is used to extract features in the audio data and perform binary classification; wherein the features in the audio data include at least one of Mel-frequency cepstral coefficients, zero crossing rate, amplitude, spectral roll-off point, spectral flux, spectral entropy, chromaticity features, and pitch.

3. The method according to claim 2, characterized in that The step of acquiring video data of a set time period before and after the target moment and performing video event analysis to determine whether an abnormal event occurs includes: Identify whether dangerous objects appear in image frames in video data; If dangerous objects are detected, it is determined that an emergency abnormal event has occurred in the classroom.

4. The method according to claim 3, characterized in that After identifying whether dangerous objects appear in the image frames in the video data, the method further includes: If no dangerous goods are identified, the distribution of people and facial expressions are identified for the image frames in the video data; According to the distribution of personnel and facial expressions, the probability of various abnormal events is obtained; Determine whether the probability of various abnormal events is greater than the set probability; If the probability of an abnormal event of the target category occurring is greater than the set probability, it is determined that an abnormal event of the target category has occurred.

5. The method according to claim 4, characterized in that After determining that an abnormal event of the target category has occurred, the method further includes: Determine whether there are classrooms with more than a set number of people gathering near the target side windows of the teaching building; If more than the set number of classrooms have people gathered near the windows on the target side of the teaching building, it is determined that the location where the abnormal event occurred is outside the target side of the teaching building.

6. The method according to claim 5, characterized in that After determining whether the number of classrooms greater than the set number of classrooms has a situation where the personnel are gathered near the target side windows of the teaching building, the method further includes: If the personnel distribution is not concentrated near the target side windows of the teaching building, or if there are less than or equal to the set number of classrooms where personnel distribution is concentrated near the target side windows of the teaching building, it is determined that the location where the abnormal event occurred is in the corresponding classroom.

7. The method according to claim 6, characterized in that After determining that the location where the abnormal event occurred is in the corresponding classroom, the method further includes: Performing face recognition on image frames in the video data to determine persons involved in the abnormal event; The abnormal events are associated with each other according to the persons involved.

8. The method according to claim 7, characterized in that The event association of the abnormal event according to the persons involved includes: Determine the tracing time period according to the type of the abnormal event; Query the historical abnormal events related to the person involved in the traceability period before the abnormal event occurs, and associate the abnormal event with the historical abnormal events.

9. An electronic device, characterized in that: include: A processor, a network interface, a memory and a computer program; the computer program is loaded into the memory; When the computer program is executed by the processor, the method according to any one of claims 1 to 8 is performed.

10. A computer program product, characterized in that The method comprises a computer program / instruction, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.

Images