Campus abnormal behavior detection method and device and electronic equipment

Abstract

The invention provides a campus abnormal behavior detection method and device and electronic equipment, and belongs to the technical field of artificial intelligence and biological signal processing, and the method comprises the steps: obtaining the facial expression features, body motion features, voice emotion features and physiological signal data of a target person in a monitoring region in a campus scene; fusing the facial expression features, the voice emotion features, the body action features and the physiological signal data to obtain fused data; performing emotion recognition and behavior prediction on the fused data based on a preset time sequence model to obtain an analysis result; and predicting whether an abnormal behavior occurs based on the analysis result, and performing early warning when the prediction result is that the abnormal behavior occurs. The technical problem that potential safety hazards still exist in actual detection results due to the fact that application of emotion and physiological signals in abnormal behavior detection is not fully considered in the prior art is effectively solved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and biological signal processing technology, and in particular to a method, device and electronic equipment for detecting abnormal behavior on campus. Background Art

[0002] Currently, campus security management mainly relies on video surveillance and behavioral analysis technologies. Common methods include capturing the behavioral patterns of students or faculty members through cameras and identifying possible abnormal or violent behaviors.

[0003] Traditional monitoring systems primarily focus on an individual's outward behavioral characteristics, such as body movements and aggressive behavior, with little consideration of emotions and physiological states. In reality, emotional changes (such as anger, anxiety, and fear) and physiological signals (such as heart rate and skin temperature) can, in certain situations, be early signs of violent behavior or mental health issues. However, methods for detecting abnormal campus behavior based on these emotional and physiological signals have yet to receive sufficient attention and application, making it difficult for existing technologies to comprehensively and accurately provide early warnings of potential safety hazards on campus. Summary of the Invention

[0004] In view of this, it is necessary to provide a campus abnormal behavior detection method, device and electronic equipment to solve the technical problem in the existing technology that the actual detection results still have security risks due to insufficient consideration of the application of emotions and physiological signals in abnormal behavior detection.

[0005] In order to solve the above technical problems, the present invention provides a method for detecting abnormal behavior on campus, comprising: Obtain facial expression features, body movement features, voice emotion features, and physiological signal data of the target person in the monitoring area in a campus scene; Fusing facial expression features, voice emotion features, body movement features, and physiological signal data to obtain fused data; Performing emotion recognition and behavior prediction on the fused data based on a preset time series model to obtain analysis results; Based on the analysis results, it is predicted whether abnormal behavior occurs and an early warning is issued when the predicted result indicates that abnormal behavior occurs.

[0006] In one possible implementation, acquiring facial expression features, body movement features, voice emotion features, and physiological signal data of a target person in a monitored area in a campus scenario includes: Acquire continuous image data captured by a camera; Extracting the facial expression features and body movement features from the continuous image data; Obtaining voice information recorded by a microphone array; Extracting the speech emotion feature from the speech information; Acquire thermal imaging video data recorded by an infrared camera; The physiological signal data is extracted from the thermal imaging video data, wherein the physiological signal data includes: heart rate, respiratory rate and skin temperature.

[0007] In a possible implementation, extracting the facial expression features and body movement features from the continuous image data includes: Extracting the facial expression features from the continuous image data based on a convolutional neural network; The body motion features are extracted from the continuous image data based on a posture estimation algorithm.

[0008] In a possible implementation, extracting the speech emotion feature from the speech information includes: Performing filtering processing on the voice information; The speech emotion feature is obtained by extracting the speech information after filtering based on the Mel-frequency cepstral coefficient.

[0009] In a possible implementation, extracting the physiological signal data from the thermal imaging video data includes: The physiological signal data is extracted from the thermal imaging video data based on a pulse signal extraction algorithm.

[0010] In one possible implementation, the analysis result includes: a combination of emotional state and behavioral state; The predicting whether abnormal behavior occurs based on the analysis result and issuing an early warning when the predicted result indicates that abnormal behavior occurs includes: Based on a preset scoring function and a combination of the emotional state and the behavioral state, it is determined whether abnormal behavior occurs and an early warning is issued when the predicted result indicates that abnormal behavior occurs.

[0011] In a possible implementation, the warning includes: low-risk warning, medium-risk warning, and high-risk warning; The emotional states include: anger, anxiety, fear, sadness and normal emotions; The behavioral states include: violence, fighting, and normal actions; The determining whether abnormal behavior occurs based on a preset scoring function and a combination of the emotional state and the behavioral state and issuing an early warning when the predicted result indicates abnormal behavior occurs includes: Obtaining a score value of the combination of the emotional state and the behavioral state based on the scoring function; Different levels of warnings are generated based on the score values, wherein the warning levels include: high risk warning, medium risk warning and low risk warning.

[0012] In a possible implementation, after predicting whether abnormal behavior occurs based on the analysis result and issuing an early warning when the predicted result indicates that abnormal behavior occurs, the following steps are further included: According to the level of the warning, it is sent to the client of the relevant personnel through different preset paths for manual intervention; Saving the facial expression features, body movement features, voice emotion features, and physiological signal data, and performing data encryption and rights management to obtain historical data; Query and review the historical data of any target person, and generate a mental health assessment of the target person based on the historical data.

[0013] The present invention also provides a campus abnormal behavior detection device, comprising: A data processing module is used to obtain the target person's facial expression features, body movement features, voice emotion features and physiological signal data; The data fusion module is used to fuse facial expression features, voice emotion features, body movement features and physiological signal data to obtain fused data; The emotion and behavior analysis module is used to perform emotion recognition and behavior prediction on the fused data based on a preset time series model to obtain analysis results; The anomaly detection and warning module is used to predict whether abnormal behavior occurs based on the analysis results and issue an early warning when the predicted result is abnormal behavior.

[0014] The present invention further provides an electronic device, comprising: Memory, used to store programs; A processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps of the campus abnormal behavior detection method described in any one of the above method items.

[0015] The beneficial effects of the present invention are as follows: the present invention provides a method for detecting abnormal behavior on campus, which, in addition to general body movement features, also obtains facial expression features, voice emotion features and physiological signal data of the target person in the monitoring area, enriches the information volume of the original data, and fuses the facial expression features, voice emotion features and physiological signal data to obtain fused data, and performs emotion recognition on the fused data and behavior prediction on the body movement features based on the time series model to obtain analysis results, and finally predicts whether abnormal behavior occurs based on the analysis results and issues an early warning when the predicted result is that abnormal behavior occurs. Since the acquired original data includes different modes , making the detection angle more comprehensive, so the detection results obtained based on these multimodal data are also more accurate, and emotional features and physiological signals are more forward-looking than single behavioral features. Therefore, the fusion data that integrates emotional features, physiological signals and behavioral features has a high sensitivity to potential safety accidents after being processed by the time series model, and can effectively complete the detection before abnormal behavior occurs, thereby eliminating safety accidents caused by abnormal behavior before they occur, effectively solving the technical problem in the existing technology that the actual detection results still have safety hazards due to insufficient consideration of the application of emotions and physiological signals in abnormal behavior detection. BRIEF DESCRIPTION OF THE DRAWINGS In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 A schematic diagram of an embodiment of the method for detecting abnormal behavior on campus provided by the present invention; Figure 2 For the present invention Figure 1 A schematic flow chart of an embodiment of S101; Figure 3 For the present invention Figure 1 A schematic flow chart of an embodiment of S104; Figure 4 A schematic structural diagram of an embodiment of the campus abnormal behavior detection device provided by the present invention; Figure 5 This is a schematic structural diagram of an embodiment of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0017] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0018] In the description of the embodiments of the present invention, unless otherwise specified, "plurality" means two or more. "And / or" describes the association relationship between associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone.

[0019] The terms "first," "second," and so on, used in the embodiments of the present invention are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, technical features designated as "first" or "second" may explicitly or implicitly include at least one such feature.

[0020] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0021] The present invention provides a method, device and electronic equipment for detecting abnormal behavior on campus, which are described below respectively.

[0022] Figure 1 A flow chart of an embodiment of the abnormal behavior detection method on campus provided by the present invention is shown as follows: Figure 1 As shown, the abnormal behavior detection method on campus includes: Step S101: Acquire facial expression features, body movement features, voice emotion features, and physiological signal data of a target person in a monitoring area in a campus scene; It should be noted that the present invention is specifically applied to campus scenarios, and the data used are all campus data. The reasons are: first, campus sample data is relatively easy to collect and train; second, when conducting behavioral detection, discrete data is meaningless in recognition, and it is necessary to collect continuous and time-series data from active human bodies in places of continuous activities for analysis to be meaningful. The data collected in a centralized environment such as a campus has better time series and continuity, and therefore performs better.

[0023] Step S102: fusing facial expression features, voice emotion features, body movement features, and physiological signal data to obtain fused data; It is understandable that the above-mentioned facial expression features, voice emotion features, body movement features and physiological signal data are all data of different modalities, and therefore these data need to be normalized and fused before they can be used for abnormal behavior detection.

[0024] Specifically, there are features from different modalities: facial expression features F face 、Body movement characteristics F body , voice emotion features F audio and physiological characteristics F HR 、F BR and F ST (t).

[0025] The final feature vector F can be obtained by the following fusion method fusion : 1. Feature concatenation: Concatenate the feature vectors from different modalities to obtain a high-dimensional feature vector: F fusion (t) = [F face (t), F body (t), F audio (t), F HR (t), F BR (t), F ST (t)](1) In formula (1), F fusion (t) is the eigenvector after concatenation of all modal data at time t.

[0026] It should be noted that, in this embodiment, all features are represented as time series data, which facilitates the study of the correlation between different data at different time points and the prediction of abnormal behavior through this correlation. 2. Multimodal weighted fusion: Perform weighted summation on data from different modalities to obtain a weighted feature vector: (2) In formula (2), w 1, w 2, w 3, w 4, w 5, w 6， The weight of each modality is obtained through learning, reflecting the importance of different modalities in abnormal behavior prediction.

[0027] Step S103: performing emotion recognition on the fused data and behavior prediction on the body movement features based on a preset time series model to obtain analysis results; It should be noted that since most existing technologies detect and judge the accident scene through a single behavioral feature after the safety accident occurs, the real-time detection effect is poor and the sensitivity to potential safety accidents is low. The present implementation focuses on obtaining forward-looking feature data before the safety accident occurs. By fusing these data, the temporal correlation between emotions and behaviors is extracted, and the occurrence of abnormal behavior is predicted based on these correlations. It has a high sensitivity to safety hazards and can effectively prevent safety accidents caused by abnormal behavior before they occur.

[0028] Specifically, for time series data (especially emotional and behavioral feature data), a time series model is constructed to capture the relationship between different time points. The long short-term memory network (LSTM) time series model can capture the temporal dependencies of emotions and behaviors: The eigenvector at a given time t: F fusion (t) Use an LSTM network to predict the emotional state E at time t t and behavioral state B t ,Right now: (3) Based on the prediction results of emotions and behaviors, combined with historical data and time series models, the system can generate real-time early warning signals.

[0029] When high-risk behavior is detected, the system will send early warning information to relevant personnel (such as security personnel and mental health counselors). In addition, before using the time series model, it is necessary to collect a large amount of labeled data (such as historical behavioral data, emotional data, and physiological signal data) to train the model.

[0030] Cross entropy loss function is often used to optimize the accuracy of emotion classification, behavior classification, and abnormal behavior prediction: L total = L emotion + L behavior + L risk (4) In formula (4), L emotion , L behavior , L risk They are the loss functions for emotion classification, behavior classification, and risk prediction, using cross entropy loss: (5) During the training process, the weights of the model are updated through back propagation and optimization algorithms (Adam optimizer), making the final prediction results more accurate. The output of LSTM can be a classification result, indicating the category of the current emotion or behavior. Step S104: predict whether abnormal behavior occurs based on the analysis result and issue an early warning if the predicted result indicates abnormal behavior occurs.

[0031] Compared with the existing technology, the present invention provides a method for detecting abnormal behavior on campus. In addition to general body movement features, it also obtains facial expression features, voice emotion features and physiological signal data of the target person in the monitoring area, enriching the information content of the original data, and fuses the facial expression features, voice emotion features and physiological signal data to obtain fused data. Based on the time series model, emotion recognition is performed on the fused data and behavior prediction is performed on the body movement features to obtain analysis results. Finally, based on the analysis results, it is predicted whether abnormal behavior occurs and an early warning is issued when the predicted result is that abnormal behavior occurs. Since the obtained original data includes different modalities, the detection angle is more comprehensive, and the detection results obtained based on these multimodal data also have higher accuracy. In addition, the emotion features are forward-looking. Abnormal behavior detection that integrates emotion features can more effectively prevent security incidents before they occur, thereby effectively solving the technical problem in the existing technology that the actual detection results still have security risks due to insufficient consideration of the application of emotions and physiological signals in abnormal behavior detection.

[0032] like Figure 2 In a possible implementation, step S101 includes: Step S201: Acquire continuous image data captured by a camera; Step S202: extracting facial expression features and body movement features from the continuous image data; Furthermore, step S202 includes: Extract facial expression features from continuous image data based on convolutional neural networks; Body motion features are extracted from continuous image data based on posture estimation algorithm.

[0033] Specifically, given a video stream V = {V1, V2, …, VT} where T is the number of video frames and Vt represents the image data of the tth frame. Video data is mainly used for facial expression analysis and behavior recognition. The relevant feature extraction process can be carried out through the following steps: 1. Facial expression extraction: For each frame Vt, facial expression recognition model (based on convolutional neural network CNN model) is used to extract facial features: F face(Vt) = CNN(Vt)(6) In formula (6), F face (Vt) is the facial expression feature of the t-th frame, usually a vector representing facial expression (anger, anxiety, fear, happiness, etc.). 2. Behavior recognition: You can use the pose estimation algorithm OpenPose to perform action recognition on the video, and identify human movements and behaviors by detecting key points: F body (Vt) = Pose(Vt)(7) In formula (7), F body (Vt) is the body motion feature of the t-th frame, which can be the key point position of each part of the human body.

[0034] Step S203: Acquire voice information recorded by the microphone array; Step S204: extracting speech emotion features from the speech information; Furthermore, step S204 includes: Filtering the voice information; The speech emotion features are extracted from the filtered speech information based on the Mel frequency cepstral coefficients.

[0035] Specifically, the speech information A = {A1, A2, …, AT} can be collected by a multi-directional microphone array, including speech signals (including tone, speaking rate, etc.), and emotion-related features are obtained through Mel Frequency Cepstral Coefficients (MFCC): F audio (At) = MFCC(At)(8) In formula (8), F audio (At) is the speech emotion feature at the tth moment, which is extracted by performing short-time Fourier transform and other techniques on the audio signal. It is usually a high-dimensional feature vector that represents the emotional information of the speech.

[0036] Step S205: Acquire thermal imaging video data recorded by the infrared camera; Step S206: extracting the physiological signal data from the thermal imaging video data, wherein the physiological signal data includes: heart rate, respiratory rate and skin temperature.

[0037] Furthermore, step S206 includes: Physiological signal data is extracted from thermal imaging video data based on the pulse signal extraction algorithm.

[0038] Specifically, the thermal imaging video data S = {S1, S2, …, ST} can be collected by thermal imaging or near-infrared technology, including physiological indicators such as heart rate, respiratory rate, and skin temperature. 1. Heart rate extraction: By analyzing the tiny pulse changes in the video through thermal imaging technology, heart rate data can be obtained: F HR (St) = HR Detection (St)(9) In formula (9), F HR (St) is the heart rate characteristic at the tth moment, which represents the heart rate of the human body.

[0039] 2. Respiratory rate extraction: Extract respiratory rate using the same technique: F BR (St) = BR Detection (St)(10) In formula (10), F BR (St) is the respiratory frequency characteristic at time t.

[0040] 3. Skin temperature extraction: Extract skin temperature using the same technology: F ST (St) = ST Detection (St)(11) In one possible implementation, the analysis result includes: a combination of emotional state and behavioral state; Step S104 specifically includes: judging whether abnormal behavior occurs based on a preset scoring function and a combination of emotional state and behavioral state, and issuing an early warning when the predicted result indicates that abnormal behavior occurs.

[0041] Furthermore, if Figure 3 Step S104 specifically includes: judging whether abnormal behavior has occurred based on a preset scoring function and a combination of emotional state and behavioral state, and issuing an early warning when the predicted result indicates abnormal behavior has occurred. The steps are as follows: Step S301: obtaining a score value of a combination of an emotional state and a behavioral state based on a scoring function; Step S302: Generate warnings of different levels based on the scoring values.

[0042] Among them, the warning levels include: high-risk warning, medium-risk warning and low-risk warning.

[0043] Specifically, when the emotional state E t and behavioral state B t When the combination of emotions reaches a certain threshold (for example, the combination of high-risk emotions such as anger or anxiety and violent behavior), an early warning mechanism for abnormal behavior detection can be triggered.

[0044] For example, the scoring function is as follows: Risk(t) =f(E t , Bt ) (11) In formula (11), Risk(t) is the risk score of abnormal behavior, which indicates the possibility of abnormal behavior occurring at time t. This risk score can be determined by a threshold function: (12) If Risk(t) = 1, the system triggers an alarm, believing that abnormal behavior may occur.

[0045] It is understandable that when the emotional state of a target person is anger or anxiety, and the behavior of the target person himself or others around him is violence or fighting, it is judged that abnormal behavior may occur in the monitoring area and a high-risk warning is generated.

[0046] In a possible implementation, after step S104, the following steps are further included: Step S105: Send the warning to the client of the relevant personnel according to different preset paths according to the warning level for manual intervention; For example, if a student at a university displays fear or anxiety on their face, accompanied by anger and increased heart rate among surrounding students, a high-risk alert is generated and automatically sent to the school's mental health counselors and security personnel. Counselors can intervene based on the student's emotional state, while security personnel can monitor the situation on-site.

[0047] By monitoring emotions over time, the system can analyze students' mood trends. If the system detects a student exhibiting persistent depression, anxiety, or other symptoms, accompanied by changes in physiological signals (such as a decreased heart rate or rapid breathing), it automatically generates a health warning report and sends it to the school's mental health department, providing timely counseling and intervention.

[0048] Step S106: saving facial expression features, body movement features, voice emotion features, and physiological signal data, and performing data encryption and rights management to obtain historical data; Specifically, to protect personal privacy, this embodiment may also adopt the following measures: 1. Data anonymization: The system will anonymize information related to the identities of students and faculty to avoid leaking personal identity information; 2. Localized processing: Some data analysis can be performed on the camera or edge device, reducing privacy risks during data transmission; 3. Regulatory compliance: The system design complies with relevant laws and regulations on data privacy protection, such as GDPR, COPPA, etc.

[0049] Step S107: query and review the historical data of any target person, and generate a mental health assessment of the target person based on the historical data.

[0050] For example, in the application scenario of long-term emotional monitoring, if the target person's emotional state is detected to be depressed or anxious, the historical data of the target person can be traced back, and a mental health assessment can be generated through analysis of the historical data to provide timely psychological counseling.

[0051] like Figure 4 The present invention further provides a campus abnormal behavior detection device 40, comprising: Data processing module 410, for acquiring facial expression features, body movement features, voice emotion features, and physiological signal data of the target person; The data fusion module 420 is used to fuse facial expression features, voice emotion features, body movement features and physiological signal data to obtain fused data; The emotion and behavior analysis module 430 is used to perform emotion recognition and behavior prediction on the fused data based on a preset time series model to obtain analysis results; The anomaly detection and warning module 440 is used to predict whether abnormal behavior occurs based on the analysis results and issue an early warning when the predicted result is that abnormal behavior occurs.

[0052] like Figure 5 The present invention further provides an electronic device 50, comprising: Memory 510, used for storing programs; The processor 520 is coupled to the memory 510 and is used to execute the program stored in the memory 510 to implement the steps of the campus abnormal behavior detection method described in any one of the above method items.

[0053] Those skilled in the art will appreciate that all or part of the process flow of the above-described method embodiment can be implemented by instructing related hardware (such as a processor, controller, etc.) through a computer program, and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a magnetic disk, an optical disk, a read-only memory, or a random access memory.

[0054] The above is a detailed introduction to the campus abnormal behavior detection method, device and electronic equipment provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for detecting abnormal behavior on campus, characterized in that: include: Obtain facial expression features, body movement features, voice emotion features, and physiological signal data of the target person in the monitoring area in the campus scene; The facial expression features, voice emotion features, body movement features and physiological signal data are integrated to obtain fused data; Based on a preset time series model, emotion recognition and behavior prediction are performed on the fused data to obtain analysis results; Based on the analysis result, it is predicted whether abnormal behavior occurs and an early warning is issued when the predicted result is that abnormal behavior occurs.

2. The campus abnormal behavior detection method according to claim 1, characterized in that: The acquisition of facial expression features, body movement features, voice emotion features, and physiological signal data of the target person in the monitoring area in the campus scene includes: Acquire continuous image data captured by a camera; Extracting the facial expression features and body movement features from the continuous image data; Acquire voice information recorded by a microphone array; Extracting the speech emotion feature from the speech information; Acquire thermal imaging video data recorded by infrared cameras; The physiological signal data is obtained from the thermal imaging video data, wherein the physiological signal data includes: heart rate, respiratory rate and skin temperature.

3. The campus abnormal behavior detection method according to claim 2, characterized in that: The extracting of the facial expression features and body movement features from the continuous image data comprises: Extracting the facial expression features from the continuous image data based on a convolutional neural network; The body motion features are extracted from the continuous image data based on a posture estimation algorithm.

4. The campus abnormal behavior detection method according to claim 2, characterized in that: The step of extracting the speech emotion feature from the speech information comprises: Performing filtering processing on the voice information; The speech emotion feature is extracted from the speech information after filtering based on the Mel frequency cepstral coefficient.

5. The campus abnormal behavior detection method according to claim 2, characterized in that: The step of extracting the physiological signal data from the thermal imaging video data comprises: The physiological signal data is extracted from the thermal imaging video data based on a pulse signal extraction algorithm.

6. The campus abnormal behavior detection method according to claim 1, characterized in that: The analysis results include: a combination of emotional state and behavioral state; The predicting whether abnormal behavior occurs based on the analysis result and issuing an early warning when the predicted result is that abnormal behavior occurs, includes: Based on a preset scoring function and a combination of the emotional state and the behavioral state, it is determined whether abnormal behavior occurs and an early warning is issued when the predicted result is that abnormal behavior occurs.

7. The campus abnormal behavior detection method according to claim 6, characterized in that: The determining whether abnormal behavior occurs based on a preset scoring function and a combination of the emotional state and the behavioral state and issuing an early warning when the predicted result is abnormal behavior, includes: Acquire a score value of the combination of the emotional state and the behavioral state based on the score function; Different levels of warnings are generated based on the score values, wherein the warning levels include: high risk warning, medium risk warning and low risk warning.

8. The campus abnormal behavior detection method according to claim 7, characterized in that: After predicting whether abnormal behavior occurs based on the analysis result and issuing an early warning when the predicted result is that abnormal behavior occurs, the following steps are also included: According to the level of the warning, the warning is sent to the client of the relevant personnel according to different preset paths for manual intervention; The facial expression features, body movement features, voice emotion features and physiological signal data are saved, and data encryption and permission management are performed to obtain historical data; Query and trace back the historical data of any target person, and generate a mental health assessment of the target person based on the historical data.

9. A campus abnormal behavior detection device, characterized in that: include: A data processing module is used to obtain facial expression features, body movement features, voice emotion features and physiological signal data of the target person; A data fusion module is used to fuse facial expression features, voice emotion features, body movement features and physiological signal data to obtain fused data; The emotion and behavior analysis module is used to perform emotion recognition and behavior prediction on the fusion data based on a preset time series model to obtain analysis results; The anomaly detection and warning module is used to predict whether abnormal behavior occurs based on the analysis results and issue an early warning when the predicted result is abnormal behavior.

10. An electronic device, characterized in that: include: Memory, used to store programs; A processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the campus abnormal behavior detection method described in any one of claims 1 to 8.

Images