A voice warning system and method for preventing campus bullying

Abstract

The present invention discloses a voice warning system and method for preventing campus bullying, and relates to the fields of information technology and artificial intelligence application technology. The system consists of several functional modules, including: a data acquisition module: used to collect audio data and point cloud data in campus toilets; a data processing module: used to calculate the decibel value of the audio data, and judge whether the decibel value is greater than a preset decibel threshold; based on the point cloud data, obtain a three-dimensional point cloud model and a cubicle point cloud slice; a bullying analysis module: used to perform bullying analysis based on the audio data after noise reduction processing, the three-dimensional point cloud model and the cubicle point cloud slice respectively; the bullying analysis includes: performing semantic recognition and segmentation based on audio data to obtain audio semantics; performing human posture estimation, behavior classification and recognition and abnormal behavior detection based on the three-dimensional point cloud model to obtain behavior monitoring results; performing area calculation based on the cubicle point cloud slice to obtain the point cloud slice area.

Description

Technical Field

[0001] The present invention relates to the field of information technology and artificial intelligence application technology, and specifically to a voice warning system and method for preventing campus bullying. Background Art

[0002] School bullying is a global social problem that can have a negative impact on students' mental health, academic performance, and even long-term social adaptability. Traditional countermeasures often rely on post-event reporting and intervention, but this approach is difficult to prevent the occurrence and development of bullying behavior in a timely manner. Therefore, it is particularly urgent to develop a technical solution that can monitor and warn of bullying behavior in real time. In recent years, with the rapid development of artificial intelligence (AI), the Internet of Things (IoT), and big data analysis technology, it has become possible to build an intelligent voice warning system, which provides new ideas and technical support for school bullying prevention.

[0003] At present, the campus security management system mainly focuses on physical protection such as video surveillance and access control, while there is a relative lack of preventive measures for verbal violence and psychological harm. Although some schools have begun to try to introduce smart cameras for behavioral analysis, it is still difficult to effectively identify the more concealed verbal bullying. For places where it is inconvenient to install cameras, campus toilets have become places where bullying activities frequently occur. Existing technical means are difficult to ensure privacy while effectively stopping bullying. In addition, traditional manual patrols or student reporting methods have problems of lag and subjectivity, and cannot achieve all-weather, all-round real-time monitoring. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the shortcomings of the prior art, the present invention provides a voice warning system and method for preventing campus bullying, so as to at least solve the problem that campus toilets are places where it is inconvenient to install cameras, making them places where bullying activities frequently occur, and the existing technical means are difficult to ensure privacy and effectively stop bullying behavior.

[0006] (II) Technical solution

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0008] A voice warning system for preventing campus bullying, comprising:

[0009] Data collection module: used to collect audio data and point cloud data in campus toilets;

[0010] Data processing module: used to calculate the decibel value of the audio data and determine whether the decibel value is greater than a preset decibel threshold; based on the point cloud data, obtain a three-dimensional point cloud model and a compartment point cloud slice;

[0011] Bullying analysis module: used to perform bullying analysis based on the audio data after noise reduction, the three-dimensional point cloud model and the compartment point cloud slices respectively; the bullying analysis includes: performing semantic recognition and segmentation based on the audio data to obtain audio semantics; performing human posture estimation, behavior classification and recognition and abnormal behavior detection based on the three-dimensional point cloud model to obtain behavior monitoring results; performing area calculation based on the compartment point cloud slices to obtain the point cloud slice area;

[0012] Evaluation and mobilization module: used to perform evaluations based on the audio semantics, the behavior monitoring results and the point cloud slice area, respectively, to obtain a first score, a second score and a third score, to calculate a bullying score based on the first score, the second score and the third score, to compare and judge the bullying score with a preset bullying warning threshold T, and to execute a preset mobilization in response to the comparison and judgment result.

[0013] Furthermore, the collection of audio data and point cloud data on campus includes:

[0014] The sound is captured in the form of an analog electrical signal, which is then sent to an analog-to-digital converter and converted into a discrete digital signal. The amplitude value of the received digital signal at a discrete time point is output as audio data;

[0015] Point cloud data is obtained by capturing laser point clouds reflected in three-dimensional space.

[0016] Further, the calculating the decibel value of the audio data and determining whether the decibel value is greater than a preset decibel threshold includes:

[0017] Obtaining the amplitude value of each time period of the digital signal corresponds to a specific sound pressure level, and measuring the decibel value of the sound in the campus toilet, and obtaining the preset decibel threshold Cf of each time period according to the decibel value of each time period;

[0018] The audio data processed by noise reduction is compared with the preset decibel threshold Cf to determine:

[0019] When the decibel value of the audio data is greater than the preset decibel threshold Cf, it is a warning value, and the laser point cloud scanning is started in association;

[0020] When the decibel value of the audio data is less than the preset decibel threshold Cf, it is a normal value and there is no need to be vigilant. The audio data in the campus toilets should be continuously monitored.

[0021] Furthermore, before obtaining the three-dimensional point cloud model and the compartment point cloud slices based on the point cloud data, preprocessing is required, and the preprocessing includes:

[0022] Spatial point cloud preprocessing:

[0023] By eliminating abnormal points caused by scanning equipment errors or environmental factors, the high-density point cloud data is downsampled, and multiple scanning positions are aligned to the same coordinate system through the registration algorithm to construct a three-dimensional point cloud model;

[0024] Compartment point cloud preprocessing:

[0025] Perform plane detection on the acquired compartment point cloud and slice the compartment point cloud;

[0026] From a single laser scanner, the 3D point cloud data is taken as input, and the parameter threshold distance g is set respectively. th , the number of iterations N and the minimum number of plane points M;

[0027] Randomly select three non-collinear points p1, p 2, p3 is used as the initial sample to define the hypothetical plane. 2, p3, calculate the plane equation:

[0028] a*x + b*y + c*z + f = 0

[0029] In the formula, a, b and c are the projection lengths of the normal vector on the x-axis, y-axis and z-axis respectively; f is a constant term; by traversing each point in all point cloud data, the distance from the point to the hypothetical plane is calculated; traversing each point p in all point cloud data i , calculate the distance g from the point to the hypothetical plane i , the formula is:

[0030] ;

[0031] In the formula, x i ,y i and z i are the midpoint p of the source point cloud respectively. i The x, y and z coordinate values ​​in the three-dimensional coordinate system; n is the number of point pairs; m is the ordinal number of the maximum number of vertices.

[0032] Furthermore, the step of obtaining the compartment point cloud slices includes:

[0033] The line between the midpoint of the compartment and the laser scanner is used as the midline to slice the acquired compartment point cloud data. The geometric center point C of the compartment is obtained by identifying the compartment border and calculating its geometric center. The optimal plane is obtained through the maximum number of iterations N, and the optimal plane is used as a slice of the compartment point cloud.

[0034] Furthermore, the performing semantic recognition and segmentation on the audio data after the noise reduction process to obtain audio semantics includes:

[0035] Semantic recognition and segmentation of audio data:

[0036] Acquire speech samples and annotations containing target keywords and phrases through open source websites; extract useful acoustic features from raw audio signals using Mel-frequency cepstral coefficients, build a neural network architecture using convolutional neural networks, train the model using annotated data from open source websites, minimize the prediction error rate by optimizing the loss function, detect keyword activation boundaries, and determine the exact time points when the keywords start and end.

[0037] Furthermore, based on the three-dimensional point cloud model, human posture estimation, behavior classification and recognition, and abnormal behavior detection are performed to obtain behavior monitoring results, including:

[0038] Perform human posture estimation, motion feature extraction and behavior classification and recognition on the three-dimensional point cloud model;

[0039] The RANSAC algorithm is used to detect and separate the ground or other flat surfaces, and a deep learning-based method is used for segmentation. The deep learning model is used to directly predict the active joints of the human body from the point cloud, and the human skeleton structure is formed according to the detected active joints.

[0040] Based on the human skeleton structure, characteristics of different behavior types are obtained.

[0041] Further, based on the audio semantics, the behavior monitoring results and the point cloud slice area, evaluations are performed respectively to obtain a first score, a second score and a third score respectively, and a bullying score is calculated based on the first score, the second score and the third score, including:

[0042] Obtaining a first score based on the audio semantic evaluation:

[0043] By detecting the same keyword in multiple consecutive frames and assigning values, a deep learning model is used to identify specific keywords or phrases. The pre-trained sentiment analysis model is used to classify the sentiment of each voice clip, determine the speaker's emotional state, and obtain the first score. audio , the formula is:

[0044] ;

[0045] In the formula, w1 and w2 are the weights of keyword recognition and sentiment analysis respectively, Score keywords Score for keyword recognition, Score emotion Scoring for sentiment analysis;

[0046] Obtaining a second score based on the behavior monitoring result evaluation:

[0047] Through the deep learning model, the key parts of the human body are estimated based on the point cloud data to form a virtual skeleton. Combined with the time series information, the position changes of each key point are continuously tracked to establish a continuous posture trajectory. The classifier is trained using the labeled point cloud data set to automatically identify various behavior patterns. At the same time, the bullying behavior is mapped to a virtual point cloud. The scores of each dimension are added according to the set weights to obtain the second score. pointcloud , the formula is as follows:

[0048] ;

[0049] Where w3, w4 and w5 are the weights of posture estimation, motion features and behavior recognition respectively. pose ,Score motion , Score behavior They are posture estimation accuracy, action feature saliency and behavior recognition confidence respectively;

[0050] Obtain a third score based on the point cloud slice area evaluation:

[0051] Take the line between the geometric center C and the laser scanner as the midline, calculate the proportion of the point cloud slice area in the compartment frame, and assign corresponding weights to the area density of different evaluation dimensions according to the importance of the application scenario; add the scores of each dimension according to the set weight to get the third score Score room , the formula is:

[0052] ;

[0053] Where w6, w7, w8 are the weights of density assessment; Score occupancy Score for individual quantity; movement Score for location tracking; Score density Score the distance level;

[0054] The bullying score is calculated based on the first score, the second score and the third score:

[0055] According to historical data and experimental results, the bullying warning threshold T is set to distinguish normal behavior from potential bullying behavior, and the bullying score Score is calculated through the first score, the second score and the third score. bullying , the formula is:

[0056] ;

[0057] Where, L is the maximum scan frame; is the current scanning frame; is the bullying coefficient; Q is the maximum number of point clouds in the current scanning frame.

[0058] Furthermore, the bullying score is compared with a preset bullying warning threshold T, and in response to the comparison result, a preset adjustment is performed, including:

[0059] When Score bullying >T, if bullying is detected, immediately notify relevant personnel or the teacher on duty to take action and record the incident for subsequent investigation;

[0060] When Score bullying When ≤T, it is judged that there is no bullying behavior and continue to be vigilant.

[0061] A voice warning method for preventing campus bullying includes the following steps:

[0062] Step 1: Collect audio data and point cloud data in the campus toilet; calculate the decibel value of the audio data and determine whether the decibel value is greater than a preset decibel threshold; obtain a three-dimensional point cloud model and a cubicle point cloud slice based on the point cloud data;

[0063] Step 2: Perform bullying analysis based on the audio data after noise reduction, the three-dimensional point cloud model and the compartment point cloud slices respectively; the bullying analysis includes: performing semantic recognition and segmentation based on the audio data to obtain audio semantics; performing human posture estimation, behavior classification and recognition and abnormal behavior detection based on the three-dimensional point cloud model to obtain behavior monitoring results; performing area calculation based on the compartment point cloud slices to obtain the point cloud slice area;

[0064] Step three: Based on the audio semantics, the behavior monitoring results and the point cloud slice area, evaluations are performed respectively to obtain a first score, a second score and a third score respectively; a bullying score is calculated based on the first score, the second score and the third score; the bullying score is compared with a preset bullying warning threshold T; and in response to the comparison result, a preset action is executed.

[0065] (III) Beneficial effects

[0066] The present invention provides a voice warning system and method for preventing campus bullying, which has the following beneficial effects:

[0067] (1) This solution uses noise recognition technology to monitor the decibel level of noise in campus toilets. When a certain threshold is reached, the system triggers a voice alarm to disperse the crowd and initiates laser point cloud scanning to determine whether there is bullying on campus. If there is potential bullying, the system can immediately notify the teacher on duty of the corresponding gender for on-site confirmation. Combined with noise recognition and comprehensive analysis of point cloud data, the system can effectively reduce false alarms and ensure the accuracy of alarm information, thereby improving management efficiency.

[0068] (2) This solution adjusts the sensitivity setting of the system according to the actual situation, optimizes its performance and reduces the occurrence of false alarms. The system uses laser scanning technology similar to multi-camera to provide high-density point cloud data and generate a three-dimensional human motion model, avoiding the blurred point cloud clusters formed by superposition. It fully considers privacy protection and ensures that personal privacy is not violated when collecting and processing data.

[0069] (3) This scheme uses the line between the geometric center C and the laser scanner as the midline, calculates the proportion of the point cloud slice area in the cubicle frame, and assigns corresponding weights to the area density of different evaluation dimensions according to the importance of the application scenario; the scores of each dimension are added according to the set weight to obtain the final third score, thereby fuzzily identifying bullying behavior in the cubicle area of ​​the campus toilet. To a certain extent, bullying in the cubicle of the campus toilet can be effectively identified and prevented. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 It is a schematic diagram of the system flow of the present invention;

[0071] Figure 2 It is a schematic diagram of the overall method of the present invention. DETAILED DESCRIPTION

[0072] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Example

[0073] See also Figure 1 This embodiment provides a voice warning system for preventing campus bullying, and the warning system includes:

[0074] Data acquisition module: used to collect audio data and point cloud data, and output the audio data and point cloud data;

[0075] The process of collecting audio data is as follows:

[0076] By installing several hidden microphones in the toilet area of ​​the campus, the sound in the toilet on campus is monitored in real time and captured in the form of analog electrical signals. The analog signals are then sent to the analog-to-digital converter and converted into discrete digital signals. The amplitude values ​​of the received digital signals at a series of discrete time points are represented and output; for the protection of students' privacy, after the audio clips are stored, they are only kept for the shortest necessary time for subsequent investigations, unless an emergency situation is confirmed;

[0077] The process of collecting point cloud data is as follows:

[0078] Use low-power, eye-safe laser scanners, and install them at the apex of the ceiling in each corner of the campus toilets to blend the exterior color with the surrounding wall environment, reduce the possibility of being noticed, causing anxiety to students, and the possibility of equipment damage;

[0079] By adjusting the resolution of the laser scanner, the laser point cloud reflected in the three-dimensional space can be indirectly captured, which can extend the service life of the laser scanner and the pressure of data processing. The laser point cloud captured with a certain resolution can be used for subsequent analysis, and the privacy of students will not be violated due to overly detailed laser point clouds. In addition, the system can automatically adjust the scanning parameters according to factors such as changes in ambient light and personnel flow to optimize the collection efficiency of point cloud data. For bullying that may occur in campus toilet cubicles, different from the multi-angle laser scanner point cloud imaging in the public areas of campus toilets, the laser scanner is used to scan the inside of the cubicle to capture the cubicle point cloud information and cubicle space information.

[0080] Data processing module: used to calculate the decibel value of the received audio data, determine whether it is greater than the decibel threshold, and perform noise reduction on the audio data; at the same time, process the acquired point cloud data and output the processed audio data and point cloud data; the point cloud data includes space point cloud and compartment point cloud;

[0081] The decibel value calculation process is:

[0082] The microphone installed in the toilet area of ​​the campus continuously monitors the ambient sound, and the amplitude value of the digital signal at each time point corresponds to a specific sound pressure level. The decibel value of the current sound in the campus toilet is calculated by the decibel meter as D;

[0083] Determine whether it is greater than the decibel threshold:

[0084] By obtaining the current decibel value of the sound in the campus toilet and comparing it with the pre-set decibel threshold Cf:

[0085] When the decibel value is greater than the decibel threshold Cf, it is a warning value, and the laser scanner is activated to perform a laser point cloud scan of the current space in the campus toilet;

[0086] When the decibel value is less than or equal to the decibel threshold Cf, it is a normal value and there is no need to be vigilant. The audio data in the campus toilets should be continuously monitored;

[0087] The process of denoising audio data is as follows:

[0088] Split the continuous audio stream into short time windows and perform fast Fourier transform on the audio signal in each time window to facilitate subsequent segment-by-segment processing; automatically collect a sample representing the background noise during class time or quiet periods, calculate its spectral characteristics as a reference, and convert it from the time domain to the frequency domain, which makes it easier to identify and separate different frequency components;

[0089] An appropriate overlap (e.g., 50%) is set between adjacent windows to ensure a smooth transition and avoid boundary effects. As time and the environment change, the noise model is continuously monitored and dynamically adjusted to ensure that the latest noise features are taken into account. The noise power spectrum density at each frequency point is estimated, and the estimated noise spectrum is subtracted from the spectrum of the current frame to leave a clean speech signal and prevent sound distortion caused by over-subtraction. The processed frequency domain signal is converted back to the time domain to reconstruct the denoised audio signal. At the same time, combined with voice activity detection technology, noise reduction processing is performed only when actual speech is detected, saving computing resources.

[0090] The processing process of spatial point cloud is:

[0091] Pre-process the point cloud data to eliminate abnormal points caused by scanning equipment errors or environmental factors, including isolated points, duplicate points and artifact points; downsample high-density point cloud data to retain only representative points, reduce the amount of calculation and improve processing speed; and align multiple scanning positions or different time points to the same coordinate system through registration algorithms to construct a three-dimensional point cloud model;

[0092] Isolated points: Far away from surrounding points, usually caused by poor reflection or false detection;

[0093] Duplicate points: multiple points at the same location, possibly due to multiple scans;

[0094] Artifact points: erroneous points caused by moving objects or other interference sources;

[0095] The main point cloud data (the point cloud data formed by students) are all located in a relatively dense area, while the noise points are sparse points far away from the main point cloud data area and are discarded;

[0096] Downsampling: Reduce the amount of point cloud data without affecting important features, thereby speeding up subsequent processing and reducing storage requirements; it may cause information loss in some areas and is suitable for scenarios with low accuracy requirements;

[0097] Registration algorithm: For images from multiple scanning positions or different time points, registration is to align them into the same coordinate system; by iterative nearest point algorithm, the correspondence between two sets of point clouds is repeatedly found, and the distance between them is minimized to optimize the transformation parameters, and the nearest neighbor point pairs are found, and then the optimal rigid transformation is calculated based on the nearest neighbor point pairs;

[0098] Example:

[0099] The goal is to extract each point p from the source point cloud P i Find the closest point q in the target point cloud Q j , forming a point pair (p i ,q j） ; Through the Euclidean distance metric, through a given point p i , click p i and q j The coordinates of i, y i ,z i） and (x j ,y j ,z j） , calculate its intersection with all points q in the target point cloud Q j The Euclidean distance of , and select the point corresponding to the minimum distance as the nearest neighbor point q j :

[0100] ;

[0101] In the formula, d(p i ,q j） For p i and q j The Euclidean distance between the two points in three-dimensional space is represented by the straight-line distance between them; i ,y i and z i are the midpoint p of the source point cloud respectively. i The x, y, and z coordinate values ​​in the three-dimensional coordinate system; x j ,y j and z j are the midpoint q of the source point cloud respectively. j x, y and z coordinate values ​​in a three-dimensional coordinate system;

[0102] When the nearest neighbor point pair is obtained, the source point cloud P is converted into the optimal rigid transformation of the target point cloud Q, including the rotation matrix R and the translation vector t; the sum of the squared distances between the source point cloud and the target point cloud is minimized, and the formula is:

[0103] ;

[0104] Where R is the rotation matrix; t is the translation vector; minR,t makes the objective function (i.e., the sum of the squared distances between point pairs) reach the minimum value; n is the number of point pairs; Rp i is the source point p i Rotate by the rotation matrix R;

[0105] The processing process of the compartment point cloud is:

[0106] Perform plane detection on the acquired partition point cloud to determine the area enclosed by the partition walls and door panels, and slice the partition point cloud;

[0107] By taking a single 3D point cloud data obtained from a laser scanner as input, the parameter threshold distance g is set respectively th , the number of iterations N and the minimum number of plane points M;

[0108] Threshold distance g th : The distance threshold used to determine whether a point belongs to a hypothetical plane;

[0109] Iteration number N: maximum number of iterations for fitting the plane;

[0110] Minimum number of plane points M: the minimum number of points required to form a valid plane;

[0111] Randomly select three non-collinear points p1, p 2, p3 is used as the initial sample to define the hypothetical plane. 2, p3, calculate the plane equation:

[0112] a*x + b*y + c*z + f = 0

[0113] In the formula, a, b and c are the projection lengths of the normal vector on the x-axis, y-axis and z-axis respectively; f is a constant term; by traversing each point in all point cloud data, the distance from the point to the hypothetical plane is calculated; traversing each point p in all point cloud data i , calculate the distance g from the point to the hypothetical plane i , the formula is:

[0114] ;

[0115] When i<g th , then the point is considered to belong to this hypothetical plane, and it is added to the internal point set, and the number of internal points n corresponding to the current hypothetical plane is recorded inliers , after each iteration, compare the number of newly obtained internal points n inliers and the number of internal points n of the previously recorded optimal plane best ;

[0116] When n inliers > n best , then update the optimal plane and its corresponding set of interior points, continue to randomly select three non-collinear points and compare the number of interior points, until the preset maximum number of iterations N is reached or enough interior points are found (for example, more than a certain proportion of the total number of points);

[0117] The process of compartment point cloud slicing is:

[0118] The line between the current compartment midpoint and the laser scanner is used as the midline to slice the acquired compartment point cloud data, and the plane covering the largest number of compartment point clouds is selected as the final slice;

[0119] First, determine the geometric center point C of the compartment, which can be obtained by identifying the border of the compartment and calculating its geometric center, through the maximum number of iterations N or finding enough internal points, and obtaining the best plane, using the best plane as a slice of the interval point cloud; the compartment is a polygon, whose vertices are Pu(xu,yu,zu), where u=1,2,...,m, then the geometric center C, the formula is:

[0120] ;

[0121] Define the center line. The straight line L connecting the laser scanner position S (Sx, Sy, Sz) and the geometric center C (Cx, Cy, Cz) is defined as the center line. The direction vector of this straight line is , the formula is:

[0122] ;

[0123] The RANSAC algorithm is used to find the best plane. The model parameters are estimated by randomly selecting a set of points and evaluating whether these parameters are suitable for most data points. The angle formed by the selected plane pb and the midline L is as small as possible, that is, the two are close to perpendicular. The normal vector of pb can be adjusted Bring it closer The orthogonal complement space of pb is used as the slicing plane. The original point cloud is cut according to the plane to obtain a series of slices parallel to pb. Each slice represents the point cloud distribution at a specific height.

[0124] Bullying analysis module: Bullying analysis is performed on the processed audio data, spatial point cloud and compartment point cloud respectively; semantic recognition and segmentation are performed on the processed audio data; human posture estimation is performed on the constructed three-dimensional point cloud model, and the action feature extraction algorithm is used to classify and recognize behaviors and detect abnormal behaviors; the area of ​​the compartment point cloud slices is calculated, and the recognized semantics and behavior monitoring, as well as the area of ​​the point cloud slices are output;

[0125] The process of semantic segmentation and semantic recognition of audio data is as follows:

[0126] Through keyword detection speech recognition technology, predefined key phrases or words are detected from the continuous voice stream. It is trained to recognize specific words or phrases related to bullying behavior and trigger an alarm when these keywords are detected. Voice samples and their annotations containing target keywords and phrases (insulting language and threatening words, etc.) are obtained through open source websites; useful acoustic features are extracted from the original audio signal using Mel frequency cepstral coefficients, and a neural network architecture is built through convolutional neural networks. The model is trained using annotated data from open source websites, and the prediction error rate is minimized by optimizing the loss function. Special attention should be paid to preventing overfitting during the training process; keyword activation boundary detection is used to determine the exact time points when the keyword starts and ends, accurately capture keywords, and implement smoothing filters or other strategies to reduce the false alarm rate, such as requiring that the same keyword is detected in multiple consecutive frames to be considered valid;

[0127] The process of behavior classification and identification is:

[0128] Perform human posture estimation, motion feature extraction, and behavior classification and recognition on the three-dimensional point cloud model; use the RANSAC algorithm to detect and separate the ground or other flat surfaces to better focus on the standing human body and divide the point cloud into different parts, such as the human-shaped area and other background elements; deep learning-based methods can be used for segmentation; use deep learning models to directly predict the position of key parts of the human body from the point cloud, and connect the detected key points to form a human skeleton structure for subsequent motion analysis; select the features that best represent different behavior types from the above features, including temporal features, spatial features, local features, and global features;

[0129] Temporal features: capturing the motion trajectory that changes over time, such as the speed and acceleration of joint movements;

[0130] Spatial characteristics: Analyze the spatial relationship between different parts of the body, such as the angle between the arm and the torso, the distance between the legs, etc.;

[0131] Local features: focus on dynamic changes in a specific area, such as hand waving, foot movement, etc.

[0132] Global features: consider the posture and displacement of the entire body, such as turning, jumping, and other overall action patterns;

[0133] The area calculation process of compartment point cloud slicing is as follows:

[0134] By obtaining the optimal plane, point cloud slices can be discretized into a set of small rectangles or polygons. These graphics together constitute the approximate outline of the slice. By combining the data of multiple continuous slices, the trend of changes over time can be observed, such as the increase or decrease of the number of people in a certain area.

[0135] Evaluation and mobilization module: used to evaluate the acquired semantics, behavior monitoring, and point cloud slice area respectively, obtain the first score, second score, and third score respectively, calculate the bullying score by the sum of the score results, compare it with the preset bullying warning threshold T for judgment, and make corresponding adjustments based on the judgment results;

[0136] The scoring process for the first rating is:

[0137] By detecting the same keyword in multiple consecutive frames, assigning values, and dividing the keywords into different degrees, a deep learning model is used to identify specific keywords or phrases (vocabulary is usually related to bullying). For each recognized keyword, a probability value of its occurrence, i.e., a confidence score, is given. A higher confidence means that the word is more likely to be accurately detected. The pre-trained sentiment analysis model is used to classify the sentiment of each voice clip, determine the speaker's emotional state, and obtain the first score. audio , the formula is:

[0138] ;

[0139] In the formula, w1 and w2 are the weights of keyword recognition and sentiment analysis respectively, Score keywords Score for keyword recognition, Score emotion Scoring for sentiment analysis;

[0140] The second scoring process is:

[0141] Through the deep learning model, the active joints of the human body are estimated based on the point cloud data to form a virtual skeleton. Combined with the time series information, the position changes of each active joint are continuously tracked to establish a continuous posture trajectory; the posture change trend of the whole body, such as the displacement of the center of gravity and the rotation angle, is considered to capture the overall action pattern; a series of common normal behaviors (such as standing and walking) and abnormal behaviors (such as pushing and pulling) templates are pre-defined, and the current action is judged to which category by similarity calculation; the classifier (such as SVM, random forest) is trained with the labeled point cloud data set to enable it to automatically identify various behavior patterns, and at the same time, virtual point cloud mapping can be performed on bullying behaviors. By learning virtual point cloud mapping, the richness of samples is increased; according to the importance of the application scenario, corresponding weights are assigned to different evaluation dimensions, and the weights of each dimension are set according to the importance of the application scenario; basic information is provided to help judge the overall safety status, which is indirectly reflected through density assessment. In the point cloud environment, the proportion of the influence of different dimensions on the final score, and the rationality of the weight setting is re-evaluated regularly, and appropriate adjustments are made according to the actual situation; the scores of each dimension are added according to the set weights to obtain the final second score Score pointcloud , the formula is as follows:

[0142] ;

[0143] In the formula, w3, w4 and w5 are the weights of posture estimation, action features and behavior recognition respectively. pose ,Score motion , Score behavior They are posture estimation accuracy, action feature saliency and behavior recognition confidence respectively;

[0144] The third scoring process is:

[0145] Take the line between the geometric center C and the laser scanner as the midline, select the plane covering the largest number of compartment point clouds as the final slice, calculate the proportion of the slice point cloud area in the compartment frame, and assign corresponding weights to the area density of different evaluation dimensions according to the importance of the application scenario; add the scores of each dimension according to the set weights to get the final third score Score room , the formula is:

[0146] ;

[0147] Where w6, w7, w8 are the weights of density assessment; Score occupancy Score for individual quantity; movement Score for location tracking; Score density Score the distance level;

[0148] The bullying score and the comparison with the preset bullying warning threshold T are as follows:

[0149] According to historical data and experimental results, a reasonable bullying warning threshold T is set to distinguish normal behavior from potential bullying behavior. By obtaining the first score, the second score and the third score, the bullying score Score is calculated. bullying , the formula is:

[0150] ;

[0151] Where, L is the maximum scan frame; is the current scanning frame; is the bullying coefficient; Q is the maximum number of point clouds in the current scanning frame;

[0152] The judgment is made by comparing with the preset bullying warning threshold T. The judgment process is as follows:

[0153] When Score bullying >T, if bullying is detected, immediately notify relevant personnel or the teacher on duty to take action and record the incident for subsequent investigation;

[0154] When Score bullying When ≤T, it is judged that there is no bullying behavior and continue to be vigilant;

[0155] Through the integrated SMS gateway or WeChat interface, instant messages are sent to pre-set contacts (such as teachers on duty, security managers). Through the integrated SMS gateway or WeChat interface, instant messages are sent to pre-set contacts (such as teachers on duty, security managers). Detailed incident reports are saved to relevant persons in charge, including time, location, preliminary judgment and other information. After arriving at the scene, first observe the situation to determine whether there is actual bullying behavior, and comfort the students involved.

[0156] Embodiment 2:

[0157] See also Figure 2 Based on Example 1, this embodiment also provides a voice warning method for preventing campus bullying, including the following specific steps:

[0158] Step 1: Collect audio data and point cloud data in the campus toilet; calculate the decibel value of the audio data and determine whether the decibel value is greater than a preset decibel threshold; obtain a three-dimensional point cloud model and a cubicle point cloud slice based on the point cloud data;

[0159] Step 2: Perform bullying analysis based on the audio data after noise reduction, the three-dimensional point cloud model and the compartment point cloud slices respectively; the bullying analysis includes: performing semantic recognition and segmentation based on the audio data to obtain audio semantics; performing human posture estimation, behavior classification and recognition and abnormal behavior detection based on the three-dimensional point cloud model to obtain behavior monitoring results; performing area calculation based on the compartment point cloud slices to obtain the point cloud slice area;

[0160] Step three: Based on the audio semantics, the behavior monitoring results and the point cloud slice area, evaluations are performed respectively to obtain a first score, a second score and a third score respectively; a bullying score is calculated based on the first score, the second score and the third score; the bullying score is compared with a preset bullying warning threshold T; and in response to the comparison result, a preset action is executed.

[0161] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented using software, the above embodiments may be implemented in whole or in part in the form of a computer program product. A person of ordinary skill in the art may appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution.

[0162] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0163] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A voice warning system for preventing campus bullying, the system comprising: Data collection module: used to collect audio data and point cloud data in campus toilets; Data processing module: used to calculate the decibel value of the audio data and determine whether the decibel value is greater than a preset decibel threshold; based on the point cloud data, obtain a three-dimensional point cloud model and a compartment point cloud slice; Bullying analysis module: used to perform bullying analysis based on the audio data after noise reduction, the three-dimensional point cloud model and the compartment point cloud slices; The bullying analysis includes: performing semantic recognition and segmentation based on the audio data to obtain audio semantics; performing human posture estimation, behavior classification and recognition, and abnormal behavior detection based on the three-dimensional point cloud model to obtain behavior monitoring results; performing area calculation based on the compartment point cloud slices to obtain the point cloud slice area; Evaluation and mobilization module: used to perform evaluations based on the audio semantics, the behavior monitoring results and the point cloud slice area, respectively, to obtain a first score, a second score and a third score, to calculate a bullying score based on the first score, the second score and the third score, to compare and judge the bullying score with a preset bullying warning threshold T, and to execute a preset mobilization in response to the comparison and judgment result.

2. A voice warning system for preventing campus bullying according to claim 1, characterized in that: The collecting of audio data and point cloud data on campus includes: The sound is captured in the form of an analog electrical signal, which is then sent to an analog-to-digital converter and converted into a discrete digital signal. The amplitude value of the received digital signal at a discrete time point is output as audio data; Point cloud data is obtained by capturing laser point clouds reflected in three-dimensional space.

3. The voice warning system for preventing campus bullying according to claim 1 is characterized in that: The step of calculating the decibel value of the audio data and determining whether the decibel value is greater than a preset decibel threshold comprises: Obtaining the amplitude value of each time period of the digital signal corresponds to a specific sound pressure level, and measuring the decibel value of the sound in the campus toilet, and obtaining the preset decibel threshold Cf of each time period according to the decibel value of each time period; The audio data processed by noise reduction is compared with the preset decibel threshold Cf to determine: When the decibel value of the audio data is greater than the preset decibel threshold Cf, it is a warning value, and the laser point cloud scanning is started in association; When the decibel value of the audio data is less than the preset decibel threshold Cf, it is a normal value and there is no need to be vigilant. The audio data in the campus toilets should be continuously monitored.

4. The voice warning system for preventing campus bullying according to claim 1 is characterized in that: Before obtaining the three-dimensional point cloud model and the compartment point cloud slices based on the point cloud data, preprocessing is required, and the preprocessing includes: Spatial point cloud preprocessing: By eliminating abnormal points caused by scanning equipment errors or environmental factors, the high-density point cloud data is downsampled, and multiple scanning positions are aligned to the same coordinate system through the registration algorithm to construct a three-dimensional point cloud model; Compartment point cloud preprocessing: Perform plane detection on the acquired compartment point cloud and slice the compartment point cloud; From a single laser scanner, the 3D point cloud data is taken as input, and the parameter threshold distance g is set respectively. th , the number of iterations N and the minimum number of plane points M; Randomly select three non-collinear points p1, p 2, p3 is used as the initial sample to define the hypothetical plane. 2, p3, calculate the plane equation: a*x + b*y + c*z + f = 0; In the formula, a, b and c are the projection lengths of the normal vector on the x-axis, y-axis and z-axis respectively; f is a constant term; by traversing each point in all point cloud data, the distance from the point to the hypothetical plane is calculated; traversing each point p in all point cloud data i , calculate the distance g from the point to the hypothetical plane i , the formula is: ； In the formula, x i ,y i and z i are the midpoint p of the source point cloud respectively. i The x, y and z coordinate values ​​in the three-dimensional coordinate system; n is the number of point pairs; m is the ordinal number of the maximum number of vertices.

5. The voice warning system for preventing campus bullying according to claim 1 is characterized in that: The step of obtaining the compartment point cloud slice comprises: The line between the midpoint of the compartment and the laser scanner is used as the midline to slice the acquired compartment point cloud data. The geometric center point C of the compartment is obtained by identifying the compartment border and calculating its geometric center. The optimal plane is obtained through the maximum number of iterations N, and the optimal plane is used as a slice of the compartment point cloud.

6. The voice warning system for preventing campus bullying according to claim 1 is characterized by: The performing semantic recognition and segmentation on the audio data after the noise reduction process to obtain audio semantics includes: Semantic recognition and segmentation of audio data: Acquire speech samples and annotations containing target keywords and phrases through open source websites; extract useful acoustic features from raw audio signals using Mel-frequency cepstral coefficients, build a neural network architecture using convolutional neural networks, train the model using annotated data from open source websites, minimize the prediction error rate by optimizing the loss function, detect keyword activation boundaries, and determine the exact time points when the keywords start and end.

7. The voice warning system for preventing campus bullying according to claim 1 is characterized by: Based on the stereo point cloud model, human posture estimation, behavior classification and recognition, and abnormal behavior detection are performed to obtain behavior monitoring results, including: Perform human posture estimation, motion feature extraction and behavior classification and recognition on the three-dimensional point cloud model; The RANSAC algorithm is used to detect and separate the ground or other flat surfaces, and a deep learning-based method is used for segmentation. The deep learning model is used to directly predict the active joints of the human body from the point cloud, and the human skeleton structure is formed according to the detected active joints. Based on the human skeleton structure, characteristics of different behavior types are obtained.

8. The voice warning system for preventing campus bullying according to claim 7 is characterized by: Based on the audio semantics, the behavior monitoring result and the point cloud slice area, evaluation is performed respectively to obtain a first score, a second score and a third score respectively, and a bullying score is calculated based on the first score, the second score and the third score, including: Obtaining a first score based on the audio semantic evaluation: By detecting the same keyword in multiple consecutive frames and assigning values, a deep learning model is used to identify specific keywords or phrases. The pre-trained sentiment analysis model is used to classify the sentiment of each voice clip, determine the speaker's emotional state, and obtain the first score. audio , the formula is: ； In the formula, w1 and w2 are the weights of keyword recognition and sentiment analysis respectively, Score keywords Score for keyword recognition, Score emotion Scoring for sentiment analysis; Obtaining a second score based on the behavior monitoring result evaluation: Through the deep learning model, the key parts of the human body are estimated based on the point cloud data to form a virtual skeleton. Combined with the time series information, the position changes of each key point are continuously tracked to establish a continuous posture trajectory. The classifier is trained using the labeled point cloud data set to automatically identify various behavior patterns. At the same time, the bullying behavior is mapped to a virtual point cloud. The scores of each dimension are added according to the set weights to obtain the second score. pointcloud , the formula is as follows: ； Where w3, w4 and w5 are the weights of posture estimation, motion features and behavior recognition respectively. pose ,Score motion , Score behavior They are posture estimation accuracy, action feature saliency and behavior recognition confidence respectively; Obtain a third score based on the point cloud slice area evaluation: Take the line between the geometric center C and the laser scanner as the midline, calculate the proportion of the point cloud slice area in the compartment frame, and assign corresponding weights to the area density of different evaluation dimensions according to the importance of the application scenario; add the scores of each dimension according to the set weight to get the third score Score room , the formula is: ； Where w6, w7, w8 are the weights of density assessment; Score occupancy Score for individual quantity; movement Score for location tracking; Score density Score the distance level; The bullying score is calculated based on the first score, the second score and the third score: According to historical data and experimental results, the bullying warning threshold T is set to distinguish normal behavior from potential bullying behavior, and the bullying score Score is calculated through the first score, the second score and the third score. bullying , the formula is: ； Where, L is the maximum scan frame; is the current scanning frame; is the bullying coefficient; Q is the maximum number of point clouds in the current scanning frame.

9. The voice warning system for preventing campus bullying according to claim 8 is characterized by: The bullying score is compared with a preset bullying warning threshold T, and in response to the comparison result, a preset adjustment is performed, including: When Score bullying >T, if bullying is detected, immediately notify relevant personnel or the teacher on duty to take action and record the incident for subsequent investigation; When Score bullying When ≤T, it is judged that there is no bullying behavior and continue to be vigilant.

10. A voice warning method for preventing campus bullying, characterized by: The steps include: Step 1: Collect audio data and point cloud data in the campus toilet; calculate the decibel value of the audio data and determine whether the decibel value is greater than a preset decibel threshold; obtain a three-dimensional point cloud model and a cubicle point cloud slice based on the point cloud data; Step 2: performing bullying analysis based on the audio data after noise reduction, the three-dimensional point cloud model and the compartment point cloud slices respectively; The bullying analysis includes: performing semantic recognition and segmentation based on the audio data to obtain audio semantics; performing human posture estimation, behavior classification and recognition, and abnormal behavior detection based on the three-dimensional point cloud model to obtain behavior monitoring results; performing area calculation based on the compartment point cloud slices to obtain the point cloud slice area; Step three: Based on the audio semantics, the behavior monitoring results and the point cloud slice area, evaluations are performed respectively to obtain a first score, a second score and a third score respectively; a bullying score is calculated based on the first score, the second score and the third score; the bullying score is compared with a preset bullying warning threshold T; and in response to the comparison result, a preset action is executed.

Images