Teenager bullying early warning identification method fusing multi-modal data

Abstract

The invention discloses a teenager bullying early warning identification method fusing multi-modal data, and relates to the technical field of campus safety early warning, and the method comprises the steps: S1, data collection: deploying audio and video collection equipment in a campus, wearing a wristband on a student, and carrying out data collection; s2, data preprocessing and cleaning: performing preprocessing operation on the collected original data by using a data preprocessing algorithm; S3, deep learning model application: inputting the preprocessed data into a trained deep learning model, s4, abnormal behavior detection and early warning triggering: when the deep learning model recognizes that the monitored student is in an abnormal behavior mode, the system immediately triggers an early warning mechanism, and S5, continuous monitoring and data analysis: providing data support for subsequent prevention and intervention measures; through a real-time monitoring and real-time early warning mechanism, the occurrence frequency of bullying events in the campus can be remarkably reduced, the self-protection consciousness and ability of students are enhanced, and the overall sense of security of the students is further improved.

Description

Technical Field

[0001] The present invention relates to the technical field of campus safety early warning, and in particular to a method for early warning and identification of youth bullying by integrating multimodal data. Background Art

[0002] Campus safety mainly includes two aspects: physical and psychological. Campus bullying refers to intentional, repetitive harmful behaviors between students or between students and teachers in the school environment. These behaviors can be physical, verbal or psychological. The purpose of bullying is to threaten, control or harm others, and usually involves an unequal power relationship. Victims of bullying may experience psychological problems such as anxiety, depression, and low self-esteem. Victims may also be unable to concentrate due to anxiety or fear, which affects their academic performance. At the same time, long-term bullying experience may cause victims to distrust others and limit their social skills. Long-term psychological pressure caused by bullying may also cause students to have physical health problems, such as sleep disorders or gastrointestinal discomfort. Therefore, preventing and promptly dealing with campus bullying is of great significance to students' physical and psychological safety.

[0003] According to a campus safety management method based on face recognition proposed by the invention patent with patent publication number CN 108830761 A, a campus safety database is established, which includes a baseline public security high-risk personnel database, a school personnel database, and a dangerous student database; face recognition equipment in classrooms, playgrounds, dormitory floors, or the periphery of the school recognizes the faces of people entering the shooting range and sends the recognition results to the server; based on the number of recognized faces, the density of people is determined to judge the size of the density of people; if the density of people is not greater than 2, the method of preventing infringement by outsiders is used to process the recognition results; if the density of people is greater than or equal to 2, the method of preventing campus bullying is used to process the recognition results. The present invention introduces face recognition technology to perform identity authentication by analyzing and comparing facial visual feature information, and issue an early warning when high-risk personnel appear; by collecting facial expressions and analyzing people's emotions, campus safety accidents are prevented and coordinated;

[0004] This early warning method mainly collects people's facial information to perform early warning judgment operations, but relying solely on people's facial information has certain limitations and cannot accurately determine whether bullying exists. Summary of the invention

[0005] The purpose of the present invention is to solve the deficiencies in real life, so a method for early warning and identification of youth bullying by integrating multimodal data is proposed to solve the above problems.

[0006] The specific technical solutions adopted are as follows:

[0007] Identification methods include the following:

[0008] S1 Data Collection: Deploy audio and video collection equipment on campus to monitor students' social activities and voice conversations, and wear bracelets on students to monitor their activity trajectories and heart rates. By monitoring students' physiological indicators, students' psychological states can be judged, achieving the effect of real-time monitoring of students' physiological and psychological states;

[0009] S2 Data preprocessing and cleaning: Use data preprocessing algorithms to perform denoising, standardization, and normalization on the collected raw data to ensure data quality and consistency, and provide an accurate basis for subsequent analysis;

[0010] S3 Deep Learning Model Application: Input the preprocessed data into the trained deep learning model to identify and analyze the student's behavior pattern, thereby determining whether the student's behavior pattern is abnormal;

[0011] S4 Abnormal behavior detection and warning triggering: When the deep learning model identifies that the monitored student is in an abnormal behavior pattern, the system immediately triggers the warning mechanism and publishes the warning information to the client in the hands of school administrators, teachers and parents through a pre-set communication method. After receiving the warning information, the client user intervenes and intervenes in the abnormal situation;

[0012] S5 Continuous monitoring and data analysis: After discovering and handling abnormal situations, audio and video acquisition equipment and bracelets continuously monitor students' subsequent behavioral data, and use data analysis technology to predict students' future behavior patterns and development trends, providing data support for subsequent prevention and intervention measures.

[0013] Preferably, in step S1, in the data collection operation, the data collected by the equipment deployed on campus include the students' walking tracks, behavioral movements and voice content. A volume sensor is used to monitor the students' voice volume on campus as a basis for determining whether it is normal communication. The bracelet worn by the students monitors the students' heart rate variability and skin conductivity through optical heart rate sensors and galvanic skin response sensors to reflect the balance of the wearer's sympathetic and parasympathetic nerves, which in turn serves as a basis for evaluating the students' stress and emotional state.

[0014] Preferably, in step S1, during the data collection operation, the bracelet identifies the user's emotional state and analyzes changes in the autonomic nervous system through the big data AI emotional stress comprehensive analysis model and crisis warning model, and provides early warning measures for wearers who have emotional fluctuations. At the same time, it monitors the students' sleep quality, provides improvement plans for students with abnormal sleep status, records students' emotional changes and living habits data during the wearing period, and assists parents and teachers in managing students' mental health status.

[0015] Preferably, in step S2, in the data preprocessing and cleaning operation, the collected data is denoised by means of mean filtering, the neighborhood around each data point is averaged to smooth the data and reduce noise, and the denoised result is compared with the original data to observe whether the noise is removed, and then the denoised result is analyzed to extract useful information for subsequent evaluation, modeling and decision-making.

[0016] Preferably, in step S2, in the data preprocessing and cleaning operation, before standardizing the data, first ensure that there are no missing values ​​in the data, and then convert the data into a distribution with a mean of 0 and a standard deviation of 1 to eliminate the dimension of the data and provide stable conditions for the training of the deep learning model.

[0017] Preferably, in step S3, in the deep learning model application operation, when training the deep learning model, a convolutional neural network is used as the model architecture, and the hierarchy and parameters of the model are defined using a deep learning framework. The deep learning model first reads data from a database, and then performs feature extraction, normalization and scaling operations on the data, and divides the preprocessed data into a training set, a validation set and a test set to evaluate the performance of the model and ensure the generalization ability of the model in practical applications. For image and video data, they are rotated, cropped and translated to expand the data set.

[0018] Preferably, in step S3, in the deep learning model application operation, MATLAB is used as the programming language and environment, and the ImageDatastore class is used to create a data storage object, which is used to automatically read all image files in a specified directory, and combine ImageDatastore with a preprocessing function to adjust the image size and crop the image when loading it to meet the input requirements of the model.

[0019] Preferably, in step S4, in the abnormal behavior detection and warning triggering operation, the warning mechanism used includes the basic equipment layer, data layer, service layer, application layer and front-end layer.

[0020] Preferably, in step S5, in the abnormal behavior detection and early warning triggering operation, the basic equipment layer includes a network system, multimedia equipment, a cloud platform and a storage device, the data layer includes a relational database, data backup, data recovery and data call, and the service layer is used to store and analyze behavior data and user data, and provide log records, use qualitative and quantitative analysis methods to evaluate the degree of bullying, and predict development trends.

[0021] Preferably, in step S5, in the continuous monitoring and data analysis operation, a decision plan is generated using the knowledge graph and the LLM large language model according to the specific extent of the bullying phenomenon.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] 1. The present invention can monitor students' behavior and physiological and psychological state in real time by deploying monitoring equipment and monitoring student activity information in the campus environment, and wearing bracelet equipment for monitoring physiological indicators on students' hands. The monitored data is identified and analyzed through a deep learning model to accurately determine whether the student is in a state of bullying. When bullying is detected, an early warning mechanism will be triggered. The system can automatically transmit the bullying information to parents and teachers for timely prevention and processing. Through real-time monitoring and instant early warning mechanisms, this method can significantly reduce the frequency of bullying incidents on campus, create a safe and harmonious learning environment for students, enhance students' self-protection awareness and ability, and thus improve students' overall sense of security.

[0024] 2. The present invention provides solutions by using knowledge graphs and LLM large language models, using artificial intelligence and big data technologies, and can provide personalized counseling and support plans for each student. For students who are bullied or commit bullying behaviors, the system can recommend appropriate personalized psychological counseling resources and intervention strategies based on their behavioral characteristics and psychological state. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A schematic diagram of a method for early warning and identifying youth bullying by integrating multimodal data in the present invention;

[0026] Figure 2 A front-end layer composition block diagram of a method for early warning and identification of youth bullying that integrates multimodal data in the present invention;

[0027] Figure 3 A block diagram of the service layer of a method for early warning and identification of youth bullying that integrates multimodal data in the present invention;

[0028] Figure 4 A block diagram of the cloud computing layer of a method for early warning and identifying youth bullying that integrates multimodal data in the present invention;

[0029] Figure 5 A data layer composition block diagram of a method for early warning and identifying youth bullying that integrates multimodal data in the present invention;

[0030] Figure 6 This is a block diagram of the device layer composition of a method for early warning and identification of youth bullying that integrates multimodal data in the present invention. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0032] Embodiment 1:

[0033] like Figure 1-Figure 6 As shown, a method for early warning and identification of adolescent bullying by integrating multimodal data, the identification method includes the following:

[0034] S1 Data Collection: Deploy audio and video collection equipment on campus to monitor students' social activities and voice conversations, and wear bracelets on students to monitor their activity trajectories and heart rates. By monitoring students' physiological indicators, students' psychological states can be judged, achieving the effect of real-time monitoring of students' physiological and psychological states;

[0035] S2 Data preprocessing and cleaning: Use data preprocessing algorithms to perform denoising, standardization, and normalization on the collected raw data to ensure data quality and consistency, and provide an accurate basis for subsequent analysis;

[0036] S3 Deep Learning Model Application: Input the preprocessed data into the trained deep learning model to identify and analyze the student's behavior pattern, thereby determining whether the student's behavior pattern is abnormal;

[0037] S4 Abnormal behavior detection and warning triggering: When the deep learning model identifies that the monitored student is in an abnormal behavior pattern, the system immediately triggers the warning mechanism and publishes the warning information to the client in the hands of school administrators, teachers and parents through a pre-set communication method. After receiving the warning information, the client user intervenes and intervenes in the abnormal situation;

[0038] S5 Continuous monitoring and data analysis: After discovering and handling abnormal situations, audio and video acquisition equipment and bracelets continuously monitor students' subsequent behavioral data, and use data analysis technology to predict students' future behavior patterns and development trends, providing data support for subsequent prevention and intervention measures.

[0039] Specifically, in step S1, during the data collection operation, the data collected by the equipment deployed on campus include students' walking tracks, behavioral movements and voice content. A volume sensor is used to monitor the students' voice volume on campus as a basis for determining whether it is normal communication. The bracelet worn by the students monitors the students' heart rate variability and skin conductivity through optical heart rate sensors and galvanic skin response sensors, reflecting the balance of the wearer's sympathetic and parasympathetic nerves, which in turn serves as a basis for assessing the students' stress and emotional state.

[0040] Specifically, in step S1, during the data collection operation, the bracelet uses the big data AI emotional stress comprehensive analysis model and crisis warning model to identify the user's emotional state, and analyze the changes in the autonomic nervous system, to provide early warning measures for wearers who have emotional fluctuations. At the same time, it monitors the students' sleep quality, provides improvement plans for students with abnormal sleep status, records the students' emotional changes and living habits data during the wearing period, and assists parents and teachers in managing the students' mental health status.

[0041] From the above, it can be seen that this method is applied to the early warning and processing of campus bullying. By deploying monitoring information and monitoring equipment for student activity in the campus environment, and wearing bracelet devices for monitoring physiological indicators on students' hands, students' behavior and physiological and psychological state can be monitored in real time. The monitored data is identified and analyzed through a deep learning model to accurately determine whether the student is in a state of being bullied. When bullying is detected, an early warning mechanism will be triggered. The system can automatically transmit bullying information to parents and teachers for timely prevention and processing. This method can significantly reduce the frequency of bullying incidents on campus through real-time monitoring and instant early warning mechanisms, create a safe and harmonious learning environment for students, enhance students' self-protection awareness and ability, and thereby enhance students' overall sense of security.

[0042] Embodiment 2:

[0043] like Figure 1-Figure 6 As shown in the figure, in the data preprocessing and cleaning operations, the collected data is denoised by mean filtering, and the neighborhood around each data point is averaged to smooth the data and reduce noise. The denoised results are compared with the original data to see whether the noise is removed, and then the denoised results are analyzed to extract useful information for subsequent evaluation, modeling, and decision-making.

[0044] Specifically, in step S2, in the data preprocessing and cleaning operation, before standardizing the data, first ensure that there are no missing values ​​in the data, and then convert the data into a distribution with a mean of 0 and a standard deviation of 1 to eliminate the dimension of the data and provide stable conditions for the training of the deep learning model.

[0045] Specifically, in step S3, in the deep learning model application operation, when training the deep learning model, a convolutional neural network is used as the model architecture, and the deep learning framework is used to define the hierarchy and parameters of the model. The deep learning model first reads data from the database, and then performs feature extraction, normalization and scaling operations on the data, and divides the preprocessed data into a training set, a validation set and a test set to evaluate the performance of the model and ensure the generalization ability of the model in practical applications. For image and video data, they are rotated, cropped and translated to expand the data set.

[0046] Specifically, in step S3, in the deep learning model application operation, MATLAB is used as the programming language and environment, and the ImageDatastore class is used to create a data storage object, which is used to automatically read all image files in the specified directory, combine ImageDatastore with the preprocessing function, and adjust the image size and crop it when loading the image to meet the input requirements of the model.

[0047] From the above, it can be seen that the present invention can improve the quality of data by removing irrelevant, useless or interfering parts of the data through denoising, standardization and normalization operations on the collected raw data, which helps to reduce interference factors in model training, enable the model to better learn useful information, thereby improving prediction accuracy. Noise removal can reduce the overfitting of the model to the training data, thereby improving the reliability of the data and ensuring that the analysis results are more accurate and standardized, so that the data have the same scale, thereby avoiding the problem of numerical instability caused by differences in feature scales. Data normalization can reduce the impact of extreme values ​​on model training and make the data distribution smoother. These preprocessing steps can significantly improve the performance and training effect of the model.

[0048] Embodiment three:

[0049] like Figure 1-Figure 6 As shown, in the abnormal behavior detection and warning triggering operation, the warning mechanism used includes the basic equipment layer, data layer, service layer, application layer and front-end layer.

[0050] Specifically, in step S5, in the abnormal behavior detection and early warning triggering operation, the basic equipment layer includes network systems, multimedia devices, cloud platforms and storage devices, the data layer includes relational databases, data backup, data recovery and data calls, and the service layer is used to store and analyze behavioral data and user data, and provide log records, use qualitative and quantitative analysis methods to evaluate the degree of bullying, and predict development trends.

[0051] Specifically, in step S5, during the continuous monitoring and data analysis operations, a decision plan is generated using the knowledge graph and the LLM large language model according to the specific extent of the bullying phenomenon.

[0052] From the above, it can be seen that the present invention provides solutions by using knowledge graphs and LLM large language models, utilizing artificial intelligence and big data technologies, and can provide personalized counseling and support plans for each student. For students who are bullied or commit bullying behaviors, the system can recommend appropriate personalized psychological counseling resources and intervention strategies based on their behavioral characteristics and psychological state.

[0053] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. The replacement may be a replacement of a part of the structure, device, method step, or a complete technical solution. Any equivalent replacement or change according to the technical solution and the inventive concept of the present invention shall be covered within the protection scope of the present invention.

Claims

1. A method for early warning and identification of youth bullying by integrating multimodal data, characterized in that: The following steps are involved: S1 Data Collection: Deploy audio and video acquisition equipment on campus to monitor students' social activities and voice conversations, and wear bracelets on students to monitor their activity trajectories and heart rates. By monitoring students' physiological indicators, the psychological state of students can be judged, and the effect of real-time monitoring of students' physiological and psychological states can be achieved. Use cameras to capture students' movements on campus, and use recording equipment to collect students' conversation data on campus, and upload the collected action and conversation data to the front-end layer; S2 Data preprocessing and cleaning: Use data preprocessing algorithms to perform denoising, standardization, and normalization on the collected raw data to ensure data quality and consistency, and provide an accurate basis for subsequent analysis; S3 deep learning model application: input the pre-processed data into the trained deep learning model to identify and analyze the student's behavior pattern, so as to determine whether the student's behavior pattern is abnormal. When the smart bracelet detects the presence of pre-set insulting words in the student's voice recording information, the student's recent psychological test level is low, the student's heartbeat changes suddenly in a certain period of time, the student's location changes abnormally, and there are hidden phenomena, the bracelet will retrieve information from the monitoring platform and send it to the teachers and parents on the communication platform simultaneously, repeatedly confirm the student's safety situation, and rate the student's degree of danger; S4 Abnormal behavior detection and warning triggering: When the deep learning model identifies that the monitored student is in an abnormal behavior pattern, the system immediately triggers the warning mechanism and publishes the warning information to the client in the hands of school administrators, teachers and parents through a pre-set communication method. After receiving the warning information, the client user intervenes and intervenes in the abnormal situation; S5 Continuous monitoring and data analysis: After discovering abnormal situations and handling them, audio and video acquisition equipment and bracelets continuously monitor students' subsequent behavioral data, and use data analysis technology to predict students' future behavior patterns and development trends, providing data support for subsequent prevention and intervention measures. Specifically, linear regression and nonlinear regression analysis methods are used to predict continuous variables, and logistic regression, decision tree, random forest, and support vector machine algorithms are used to classify and analyze data.

2. The method for early warning and identification of youth bullying by integrating multimodal data as claimed in claim 1, characterized in that: In step S1, during the data collection operation, the data collected by the equipment deployed on campus include students' walking tracks, behavioral movements and voice content. A volume sensor is used to monitor the students' voice volume on campus as a basis for determining whether it is normal communication. The bracelet worn by the students monitors the students' heart rate variability and skin conductivity through optical heart rate sensors and galvanic skin response sensors, reflecting the balance of the wearer's sympathetic and parasympathetic nerves, which in turn serves as a basis for evaluating the students' stress and emotional state.

3. The method for early warning and identification of youth bullying by integrating multimodal data as claimed in claim 1, characterized in that: In step S1, in the data collection operation, the bracelet identifies the user's emotional state through the big data AI emotional stress comprehensive analysis model and the crisis warning model, and analyzes the changes in the autonomic nervous system, provides warning measures for wearers who have emotional fluctuations, and monitors the students' sleep quality at the same time. Provide improvement plans for students with abnormal sleep status, record students' emotional changes and living habits during wearing, and assist parents and teachers in managing students' mental health. The big data AI emotional stress comprehensive analysis model first performs cleaning, denoising, standardization and normalization preprocessing operations on the uploaded student behavior data, and then uses statistical methods or machine learning techniques to select the most relevant features, and uses the training data set to train the model, adjust the model parameters to optimize performance, and use the verification data set to evaluate the performance of the model, perform hyperparameter tuning, and use data visualization tools to display the model results and the dynamic changes of emotions and pressure. The implementation process of the crisis warning model is to preprocess the data first, then perform crisis identification and risk assessment, and then generate a warning signal, and transmit the warning signal to the client of parents and teachers.

4. The method for early warning and identification of youth bullying by integrating multimodal data as claimed in claim 1, characterized in that: In step S2, in the data preprocessing and cleaning operation, the collected data is denoised by means of mean filtering, and the neighborhood around each data point is averaged to smooth the data and reduce noise. The denoised result is compared with the original data to see whether the noise is removed, and then the denoised result is analyzed to extract useful information for subsequent evaluation, modeling and decision-making.

5. The method for early warning and identification of youth bullying by integrating multimodal data as claimed in claim 1, characterized in that: In step S2, in the data preprocessing and cleaning operation, before standardizing the data, first ensure that there are no missing values ​​in the data, and then convert the data into a distribution with a mean of 0 and a standard deviation of 1 to eliminate the dimension of the data and provide stable conditions for the training of the deep learning model.

6. The method for early warning and identification of youth bullying by integrating multimodal data as claimed in claim 1, characterized in that: In step S3, in the deep learning model application operation, when training the deep learning model, a convolutional neural network is used as the model architecture, and the deep learning framework is used to define the hierarchy and parameters of the model. The deep learning model first reads data from the database, and then performs feature extraction, normalization and scaling operations on the data, and divides the preprocessed data into a training set, a validation set and a test set to evaluate the performance of the model and ensure the generalization ability of the model in practical applications. For image and video data, they are rotated, cropped and translated to expand the data set.

7. The method for early warning and identification of youth bullying by integrating multimodal data as claimed in claim 1, characterized in that: In step S3, in the deep learning model application operation, MATLAB is used as the programming language and environment, and the ImageDatastore class is used to create a data storage object, which is used to automatically read all image files in the specified directory, combine ImageDatastore with the preprocessing function, and adjust the image size and crop it when loading the image to meet the input requirements of the model.

8. The method for early warning and identification of youth bullying by integrating multimodal data as claimed in claim 1, characterized in that: In step S4, in the abnormal behavior detection and warning triggering operation, the warning mechanism used includes the basic equipment layer, data layer, service layer, application layer and front-end layer.

9. The method for early warning and identification of youth bullying by integrating multimodal data as claimed in claim 8, characterized in that: In step S5, in the abnormal behavior detection and early warning triggering operation, the basic equipment layer includes network systems, multimedia devices, cloud platforms and storage devices, the data layer includes relational databases, data backup, data recovery and data calls, and the service layer is used to store and analyze behavioral data and user data, and provide log records, use qualitative and quantitative analysis methods to evaluate the degree of bullying, and predict development trends.

10. The method for early warning and identification of youth bullying by integrating multimodal data according to claim 1, characterized in that: In step S5, in the continuous monitoring and data analysis operation, a decision plan is generated using the knowledge graph and the LLM large language model according to the specific extent of the bullying phenomenon.

Images