Risk early warning method and system for student behavior track and action analysis

Abstract

The invention relates to the technical field of behavior risk early warning, and provides a risk early warning method and system for student behavior track and action analysis, and the method comprises the following steps: respectively collecting video data and sensing data of student behaviors; preprocessing the video data and the sensing data to obtain student behavior multi-source data; performing target detection and target tracking on the student behavior multi-source data to generate a student movement track; based on the student movement track, behavior characteristics of the student are extracted, abnormal behavior recognition is carried out on the behavior characteristics, and an abnormal behavior recognition result is obtained; performing risk level evaluation on the abnormal behavior recognition result to obtain an abnormal behavior risk level; and according to the abnormal behavior risk level, triggering early warning information of a corresponding level, processing the early warning information, generating a final early warning result, and pushing the final early warning result to a manager. According to the invention, omnibearing monitoring, timely identification and rapid early warning of abnormal behaviors of students are realized, and the efficiency of campus safety management is improved.

Description

Technical Field

[0001] The present invention relates to the field of behavioral risk warning technology, and in particular to a risk warning method and system for analyzing student behavior trajectory and movements. Background Art

[0002] As campus safety management requirements continue to increase, real-time monitoring of student behavior and risk warnings are becoming increasingly important. Students may exhibit various abnormal behaviors during campus activities, such as climbing over fences, fainting, and staying for a long time. If these behaviors are not discovered and handled in a timely manner, they may lead to safety accidents.

[0003] Existing technologies generally use video surveillance to monitor campuses, adopt a fixed-route patrol mode, lack intelligent patrol planning and path optimization, resulting in low detection efficiency, inability to timely discover and deal with abnormal behavior, and cannot fully reflect the characteristics of various dimensions of student behavior. There is a lack of risk assessment of abnormal behavior in multiple dimensions, and it is impossible to accurately assess the degree of danger and urgency of the behavior. Summary of the invention

[0004] In view of this, the present invention proposes a risk warning method and system for student behavior trajectory and action analysis, which solves the problem that the existing technology lacks intelligent inspection planning and path optimization, resulting in low detection efficiency, inability to fully reflect the characteristics of various dimensions of student behavior, and inability to accurately assess the degree of danger and urgency of the behavior.

[0005] The technical solution of the present invention is implemented as follows: In the first aspect, the present invention provides a risk warning method for student behavior trajectory and action analysis, comprising the following steps:

[0006] Through camera arrays and multiple types of sensors, video data and sensor data of student behavior are collected respectively;

[0007] Preprocessing the video data and the sensor data to obtain multi-source data of student behavior;

[0008] Performing target detection and target tracking on the multi-source data of student behavior to generate student movement trajectories;

[0009] Based on the student's movement trajectory, extract the student's behavior characteristics, and based on a preset normal behavior pattern library, perform abnormal behavior recognition on the behavior characteristics to obtain an abnormal behavior recognition result;

[0010] Perform risk level assessment on abnormal behavior identification results to obtain abnormal behavior risk level;

[0011] According to the abnormal behavior risk level, the warning information of the corresponding level is triggered, the warning information is confirmed and optimized, and the final warning result is generated, and the final warning result is pushed to the management personnel.

[0012] On the basis of the above technical solution, preferably, the video data and sensor data of student behavior are collected respectively by using a camera array and multiple types of sensors, specifically including:

[0013] Deploy camera arrays in key areas of the campus, determine the installation position and angle of the camera based on the spatial distribution characteristics of the key areas of the campus, according to the field of view coverage requirements and occlusion conditions, optimize the spacing between adjacent cameras based on the spatial distance and field of view overlap, adjust the camera parameter settings based on the lighting conditions of the camera array's collection area, and collect video data of student behavior;

[0014] Multiple types of sensors are deployed in key areas of the campus, including infrared sensors, sound sensors and motion sensors. The corresponding types of sensor data are collected through the multiple types of sensors. Infrared sensors are installed at the entrances and corridors of buildings to detect the passage of people; sound sensors are installed in classrooms and activity areas to collect abnormal sound signals; motion sensors are installed around fences and dangerous areas to detect abnormal movements.

[0015] On the basis of the above technical solution, preferably, the preprocessing of the video data and the sensor data to obtain the multi-source data of student behavior specifically includes:

[0016] Performing data cleaning on the video data and the sensor data respectively, including removing noise data, filling missing values ​​and eliminating outliers;

[0017] Performing image enhancement and noise reduction processing on the video data to obtain enhanced noise-reduced video data; applying a sliding window filtering algorithm to the sensor data to remove burst noise to obtain denoised sensor data; using an interpolation algorithm to repair missing data portions of the enhanced noise-reduced video data and the denoised sensor data, and identifying and eliminating outliers based on a statistical method to obtain cleaned video data and cleaned sensor data;

[0018] Based on the timestamp information, the sampling time of the cleaned video data and the cleaned sensor data is uniformly converted into a standard time format, and the cleaned video data and the cleaned sensor data are resampled based on the minimum time interval to obtain resampled video data and resampled sensor data. The linear interpolation method is used to synchronize the time of the resampled video data and the resampled sensor data to obtain multi-source data of student behavior.

[0019] On the basis of the above technical solution, preferably, the target detection and target tracking are performed on the multi-source data of student behavior to generate the student movement trajectory, which specifically includes:

[0020] Extracting image features and signal features from the multi-source data of student behavior, calculating feature weights of different data sources, performing weighted fusion to obtain behavior fusion features, and inputting the behavior fusion features into a deep learning model to obtain target detection results;

[0021] The calculation formula of the weighted fusion is:

[0022]

[0023] Among them, W i is the fusion weight of the i-th data source, R i is the characteristic reliability index of the i-th data source, S i is the feature significance index of the i-th data source, R j is the characteristic reliability index of the jth data source, S j is the feature significance index of the jth data source, α i , β i , α j , β j are the feature reliability adjustment factor of the i-th data source, the feature significance adjustment factor of the i-th data source, the feature reliability adjustment factor of the j-th data source, and the feature significance adjustment factor of the j-th data source, respectively. N1 is the number of data sources;

[0024] The optimization formula for the confidence of target detection is:

[0025]

[0026] Among them, C is the optimized confidence, C0 is the original confidence, λ is the confidence adjustment coefficient, N1 is the number of data sources, and W i is the fusion weight of the i-th data source, D i is the detection score of the i-th data source;

[0027] Based on the target detection result, a target tracking algorithm with multi-feature fusion is used to track the detected target, and a target feature descriptor is constructed. The target feature descriptor includes the appearance feature and motion feature of the target. A Kalman filter algorithm is used to predict the target position, and data association is performed based on matching degree calculation to generate a continuous motion trajectory. The generated trajectory is evaluated for smoothness and optimized to generate a student motion trajectory with spatiotemporal information.

[0028] The calculation formula for the smoothness evaluation is:

[0029]

[0030] Among them, S is the trajectory smoothness, Δθ k,k+1 is the angle change between the kth trajectory point and the k+1th trajectory point, w k is the time weight factor of the kth trajectory point, and N2 is the number of trajectory points minus 1.

[0031] On the basis of the above technical solution, preferably, the extracting of the student's behavior characteristics based on the student's movement trajectory, and performing abnormal behavior recognition on the behavior characteristics based on a preset normal behavior pattern library to obtain abnormal behavior recognition results specifically include:

[0032] Based on the student's motion trajectory, extract the student's behavior characteristics, the behavior characteristics include spatiotemporal characteristics and posture characteristics, the spatiotemporal characteristics include motion speed, activity range, and dwell time, the posture characteristics include posture angle, limb movement, and motion amplitude, and construct a student behavior feature vector;

[0033] The calculation formula of the student behavior feature vector is:

[0034] F=[w t ×V t +w s ×V s +w p ×V p ];

[0035] Among them, F is the student behavior feature vector, V t 、V s 、V p are the time feature vector, space feature vector and posture feature vector respectively, w t 、w s 、w p are the weight coefficients of the temporal feature vector, spatial feature vector and posture feature vector, respectively, w t 、w s 、w p The sum is 1;

[0036] Calculate the feature similarity between the student behavior feature vector and each pattern in the normal behavior pattern library, comprehensively consider the feature similarity and behavior duration, calculate the abnormality score, determine the abnormal behavior type according to the abnormality score, and output the abnormal behavior recognition result, which includes the abnormal behavior type and the abnormality score;

[0037] The calculation formula of the feature similarity is:

[0038]

[0039] Among them, Sim(·) is the similarity function, F1 is a student behavior feature vector, F2 is the feature vector to be compared in the normal behavior pattern library, exp(·) is the exponential function, γ is the distance decay coefficient, σ is the scale parameter, T1 and T2 are the behavior durations corresponding to F1 and F2 respectively, T max is the maximum time difference threshold;

[0040] The calculation formula of the abnormality score is:

[0041]

[0042] Among them, score(F0) is the abnormality score corresponding to another student behavior feature vector F0, max(Sim) is the maximum feature similarity value, β0 is the time influence factor, D(F0) is the behavior duration corresponding to F0, and D0 is the benchmark duration.

[0043] On the basis of the above technical solution, preferably, the risk level assessment of the abnormal behavior recognition result is performed to obtain the abnormal behavior risk level, specifically including:

[0044] Based on historical data and expert experience, we quantitatively evaluate the risk level of different types of abnormal behaviors, analyze the impact of behavior duration, location sensitivity, environmental complexity, and population density on the risk level, and establish risk level classification standards;

[0045] The risk level classification standard is used to evaluate the risk level of abnormal behavior identification results, calculate the basic risk score of abnormal behavior, weight the risk based on the duration of abnormal behavior, and comprehensively consider the regulatory effect of scenario characteristics on risk to obtain a comprehensive risk score. Based on a preset level threshold, the comprehensive risk score is mapped to a corresponding abnormal behavior risk level, and the abnormal behavior risk levels include low risk, medium risk and high risk.

[0046] On the basis of the above technical solution, preferably, triggering the warning information of the corresponding level according to the risk level of the abnormal behavior, confirming and optimizing the warning information, generating the final warning result, and pushing the final warning result to the management personnel specifically includes:

[0047] Generate warning information including behavior type, location information, risk level and disposal suggestions according to the risk level of abnormal behavior, and conduct credibility assessment and priority sorting on the warning information;

[0048] Based on the risk level and priority of the warning information, determine the push objects and methods of the warning information, establish a hierarchical push strategy, and push the warning information to managers at the corresponding level.

[0049] In a second aspect, the present invention further provides a risk warning system for student behavior trajectory and action analysis, the system comprising:

[0050] A data acquisition module is used to collect video data and sensor data of student behavior through a camera array and multiple types of sensors;

[0051] A data processing module, used for preprocessing the video data and the sensor data to obtain multi-source data of student behavior;

[0052] A trajectory generation module is used to perform target detection and target tracking on the multi-source data of student behavior to generate a student movement trajectory;

[0053] An abnormality identification module is used to extract the student's behavior characteristics based on the student's movement trajectory, and to identify abnormal behaviors based on the behavior characteristics based on a preset normal behavior pattern library to obtain abnormal behavior identification results;

[0054] The risk assessment module is used to assess the risk level of abnormal behavior recognition results and obtain the abnormal behavior risk level;

[0055] The early warning push module is used to trigger the early warning information of the corresponding level according to the risk level of the abnormal behavior, confirm and optimize the early warning information, generate the final early warning result, and push the final early warning result to the management personnel.

[0056] In a third aspect, the present invention further provides an electronic device, comprising: at least one processor, at least one memory, a communication interface and a bus;

[0057] Among them, the processor, memory, and communication interface communicate with each other through the bus, the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement steps such as a risk warning method for student behavior trajectory and action analysis.

[0058] In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions enable a computer to implement steps such as a risk warning method for analyzing student behavior trajectories and movements.

[0059] The risk warning method and system for analyzing student behavior trajectory and action of the present invention has the following beneficial effects compared with the prior art:

[0060] (1) By deploying camera arrays and multi-type sensors to collect multi-source data, after data cleaning and timestamp alignment preprocessing, the system detects and tracks student behaviors, and identifies abnormal behaviors by combining spatiotemporal and posture features. Abnormal behaviors are then graded through risk level assessment, and corresponding warning information is finally generated and pushed to management personnel, thus achieving all-round monitoring, timely identification, and rapid warning of students' abnormal behaviors, and improving the efficiency and accuracy of campus safety management.

[0061] (2) By applying feature weighted fusion and deep learning models, combined with multi-feature fusion target tracking algorithm and Kalman filter prediction, the accuracy and confidence of target detection are improved, stable tracking and smooth optimization of students' motion trajectories are achieved, and false detections and missed detections are reduced;

[0062] (3) Through the anomaly recognition method of multi-dimensional feature extraction and similarity calculation, combined with the weighted fusion of spatiotemporal features and posture features, and the scoring mechanism considering the time factor, the accurate recognition and quantitative evaluation of students' abnormal behaviors are achieved, the false alarm rate is reduced, and the accuracy of abnormal behavior recognition is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0064] Figure 1 A flow chart of a risk warning method for analyzing student behavior trajectory and action according to the present invention;

[0065] Figure 2 This is a structural diagram of a risk warning system for student behavior trajectory and action analysis according to the present invention. DETAILED DESCRIPTION

[0066] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] See also Figure 1 The present invention provides a risk warning method for analyzing student behavior trajectory and action, comprising the following steps:

[0068] Through camera arrays and multiple types of sensors, video data and sensor data of student behavior are collected respectively;

[0069] Preprocessing the video data and the sensor data, including data cleaning and time stamp alignment, to obtain multi-source data of student behavior;

[0070] Performing target detection and target tracking on the multi-source data of student behavior to generate student movement trajectories;

[0071] Based on the student's motion trajectory, extract the student's behavior characteristics, the behavior characteristics include spatiotemporal characteristics and posture characteristics, and based on a preset normal behavior pattern library, perform abnormal behavior recognition on the behavior characteristics to obtain abnormal behavior recognition results;

[0072] Perform risk level assessment on abnormal behavior identification results to obtain abnormal behavior risk level;

[0073] According to the abnormal behavior risk level, the corresponding level of warning information is triggered, the warning information is confirmed and optimized, and the final warning result is generated, and the final warning result is pushed to the management personnel.

[0074] Specifically, this embodiment collects multi-source data by deploying camera arrays and multi-type sensors. After preprocessing of data cleaning and timestamp alignment, it performs target detection and tracking of student behaviors, and identifies abnormal behaviors in combination with spatiotemporal features and posture features. It then classifies abnormal behaviors through risk level assessment, and finally generates corresponding warning information and pushes it to management personnel, thereby achieving all-round monitoring, timely identification and rapid warning of students' abnormal behaviors, effectively improving the efficiency and accuracy of campus safety management.

[0075] The video data and sensor data of student behavior are collected through the camera array and multiple types of sensors, including:

[0076] Deploy camera arrays in key areas of the campus, determine the installation position and angle of the camera based on the spatial distribution characteristics of the key areas of the campus, according to the field of view coverage requirements and occlusion conditions, optimize the spacing between adjacent cameras based on the spatial distance and field of view overlap, adjust the camera parameter settings based on the lighting conditions of the camera array's collection area, and collect video data of student behavior;

[0077] Multiple types of sensors are deployed in key areas of the campus, including infrared sensors, sound sensors and motion sensors. The corresponding types of sensor data are collected through the multiple types of sensors. Infrared sensors are installed at the entrances and corridors of buildings to detect the passage of people; sound sensors are installed in classrooms and activity areas to collect abnormal sound signals; motion sensors are installed around fences and dangerous areas to detect abnormal movements.

[0078] In a specific embodiment, camera arrays and multi-type sensors are deployed in key areas of the campus, such as the entrance of the teaching building, the playground, and the corridor. The installation position and angle of the camera are optimized according to the spatial distribution characteristics of the campus to ensure maximum field of view coverage and reduce occlusion. The spacing between adjacent cameras is adjusted according to the spatial distance and field of view overlap to achieve seamless monitoring. The parameter settings of the camera are adjusted according to the lighting conditions of the collection area to ensure that clear video data can be collected under different lighting conditions.

[0079] Specifically, this embodiment ensures full coverage of key campus areas, reduces monitoring blind spots, and improves monitoring efficiency by reasonably deploying cameras and sensors. It combines video data and multi-type sensor data to provide more comprehensive behavior information, which helps to more accurately identify abnormal behaviors. The dynamic adjustment of camera parameters enables adaptation to different lighting conditions and ensures the stability of data collection. Through the collaborative work of multiple types of sensors, abnormal behaviors can be detected and identified more accurately, reducing the possibility of false positives and false negatives.

[0080] The preprocessing of the video data and the sensor data to obtain the multi-source data of student behavior specifically includes:

[0081] Performing data cleaning on the video data and the sensor data respectively, including removing noise data, filling missing values ​​and eliminating outliers;

[0082] Performing image enhancement and noise reduction processing on the video data to obtain enhanced noise-reduced video data, eliminating image quality problems caused by factors such as insufficient lighting and weather changes; applying a sliding window filtering algorithm to the sensor data to remove burst noise and obtain denoised sensor data; using an interpolation algorithm to repair missing data parts of the enhanced noise-reduced video data and the denoised sensor data, and identifying and eliminating outliers based on statistical methods to obtain cleaned video data and cleaned sensor data;

[0083] Based on the timestamp information, the sampling times of the cleaned video data and the cleaned sensor data are uniformly converted into a standard time format, and the cleaned video data and the cleaned sensor data are resampled based on the minimum time interval to obtain resampled video data and resampled sensor data, so that the sampling frequencies of the cleaned video data and the cleaned sensor data are consistent. The linear interpolation method is used to time synchronize the resampled video data and the resampled sensor data to achieve precise alignment of multi-source data and obtain multi-source data of student behavior.

[0084] In a specific embodiment, the collected video data and sensor data are preprocessed, specifically including:

[0085] Perform image enhancement and noise reduction on video data, use Gaussian filters to reduce noise and enhance image quality, especially in low light or weather conditions, apply sliding window filtering algorithm (window size can be set to 5 data points) to sensor data to remove burst noise, use linear interpolation to fill missing values, and identify and remove outliers through statistical methods (such as the 3σ principle).

[0086] The sampling time of the cleaned video data and sensor data is uniformly converted into a standard time format (such as UTC), and the cleaned video data and sensor data are resampled based on the minimum time interval (such as 100 milliseconds) to make the sampling frequency of the two consistent. The linear interpolation method is used to synchronize the time of the resampled video data and sensor data to achieve accurate alignment of multi-source data.

[0087] Specifically, this embodiment improves the clarity of video data and the accuracy of sensor data through image enhancement and noise reduction processing, reduces the impact of noise and outliers, ensures the time synchronization of multi-source data through timestamp alignment and resampling, enables information from different data sources to be analyzed in the same time frame, improves the accuracy of data fusion, and makes the data after preprocessing more complete and consistent, providing more reliable sample data for target detection, behavior feature extraction and anomaly recognition.

[0088] The performing target detection and target tracking on the multi-source data of student behavior to generate the student movement trajectory specifically includes:

[0089] Extracting image features and signal features from the multi-source data of student behavior, calculating feature weights of different data sources, performing weighted fusion, obtaining behavior fusion features, inputting the behavior fusion features into a deep learning model, and obtaining target detection results, wherein the confidence of target detection is optimized and calculated according to the feature fusion weights;

[0090] The calculation formula of the weighted fusion is:

[0091]

[0092] Among them, W i is the fusion weight of the i-th data source, R i is the characteristic reliability index of the i-th data source, S i is the feature significance index of the i-th data source, R j is the characteristic reliability index of the jth data source, S j is the feature significance index of the jth data source, α i , β i , α j , β j are the feature reliability adjustment factor of the i-th data source, the feature significance adjustment factor of the i-th data source, the feature reliability adjustment factor of the j-th data source, and the feature significance adjustment factor of the j-th data source, respectively. N1 is the number of data sources;

[0093] The optimization formula for the confidence of target detection is:

[0094]

[0095] Among them, C is the optimized confidence, C0 is the original confidence, λ is the confidence adjustment coefficient, N1 is the number of data sources, and W i is the fusion weight of the i-th data source, D i is the detection score of the i-th data source;

[0096] Based on the target detection result, a target tracking algorithm with multi-feature fusion is used to track the detected target, and a target feature descriptor is constructed. The target feature descriptor includes the appearance feature and motion feature of the target. A Kalman filter algorithm is used to predict the target position, and data association is performed based on matching degree calculation to generate a continuous motion trajectory. The generated trajectory is evaluated for smoothness and optimized to generate a student motion trajectory with spatiotemporal information.

[0097] The calculation formula for the smoothness evaluation is:

[0098]

[0099] Among them, S is the trajectory smoothness, Δθ k,k+1 is the angle change between the kth trajectory point and the k+1th trajectory point, w k is the time weight factor of the kth trajectory point, and N2 is the number of trajectory points minus 1.

[0100] In a specific embodiment, image features and signal features are extracted from multi-source data of student behavior. Image features may include color histograms, edge features, etc., and signal features may include frequency responses of sensors. Feature weights of different data sources are calculated, for example, image feature weights are set to 0.6, and signal feature weights are set to 0.4.

[0101] The extracted features are weighted and fused to obtain the behavioral fusion features. The fusion process can use the weighted average method, in which the weight of each feature is assigned according to its importance.

[0102] The behavior fusion features are input into a pre-trained deep learning model (such as a convolutional neural network) to obtain the target detection results.

[0103] Based on the target detection results, a target tracking algorithm with multi-feature fusion is used to track the detected target. A target feature descriptor is constructed, including the target's appearance features (such as color and shape) and motion features (such as speed and direction). The Kalman filter algorithm is used to predict the target position, and data association is performed based on the matching degree calculation to generate a continuous motion trajectory. The generated trajectory is evaluated for smoothness and optimized to ensure the accuracy and continuity of the trajectory.

[0104] Specifically, this embodiment improves the accuracy and confidence of target detection, reduces false detection and missed detection, and adopts the target tracking method of Kalman filter algorithm and multi-feature fusion to effectively predict the target position and perform data association, and the generated motion trajectory is smoother and more continuous. This embodiment can process multi-source data in real time in complex environments, maintain efficient target detection and tracking capabilities, and adapt to different lighting and environmental changes.

[0105] The extracting of the student's behavior characteristics based on the student's movement trajectory, and performing abnormal behavior recognition on the behavior characteristics based on a preset normal behavior pattern library to obtain abnormal behavior recognition results specifically include:

[0106] Based on the student's motion trajectory, extract the student's behavior characteristics, the behavior characteristics include spatiotemporal characteristics and posture characteristics, the spatiotemporal characteristics include motion speed, activity range, and dwell time, the posture characteristics include posture angle, limb movement, and motion amplitude, and construct a student behavior feature vector;

[0107] The calculation formula of the student behavior feature vector is:

[0108] F=[w t ×V t +w s ×V s +w p ×V p ];

[0109] Among them, F is the student behavior feature vector, V t 、V s 、V p are the time feature vector, space feature vector and posture feature vector respectively, w t 、w s 、w p are the weight coefficients of the temporal feature vector, spatial feature vector and posture feature vector, respectively, w t 、w s 、w p The sum is 1;

[0110] Calculate the instantaneous speed and acceleration of the student's motion trajectory to obtain the motion speed; count the spatial distribution of the activity area to obtain the activity range; count the length of stay to obtain the stay time; extract posture features based on skeleton key point detection, including posture angle, limb movement and movement amplitude; normalize the extracted spatiotemporal features and posture features to construct a multidimensional feature vector, thereby obtaining the student behavior feature vector;

[0111] Calculate the feature similarity between the student behavior feature vector and each pattern in the normal behavior pattern library, comprehensively consider the feature similarity and behavior duration, calculate the abnormality score, determine the abnormal behavior type according to the abnormality score, and output the abnormal behavior recognition result, which includes the abnormal behavior type and the abnormality score;

[0112] The calculation formula of the feature similarity is:

[0113]

[0114] Among them, Sim(·) is the similarity function, F1 is a student behavior feature vector, F2 is the feature vector to be compared in the normal behavior pattern library, exp(·) is the exponential function, γ is the distance decay coefficient, σ is the scale parameter, T1 and T2 are the behavior durations corresponding to F1 and F2 respectively, T max is the maximum time difference threshold;

[0115] The calculation formula of the abnormality score is:

[0116]

[0117] Among them, score(F0) is the abnormality score corresponding to another student behavior feature vector F0, max(Sim) is the maximum feature similarity value, β0 is the time influence factor, D(F0) is the behavior duration corresponding to F0, and D0 is the benchmark duration.

[0118] In a specific embodiment, extracting behavioral features and identifying anomalies includes:

[0119] 1) Behavioral feature extraction:

[0120] Extraction of spatiotemporal features includes: movement speed: calculating the displacement-time ratio between adjacent trajectory points; activity range: statistical spatial distribution range of trajectory points (e.g., the range in the classroom should not exceed 20 square meters); residence time: recording the length of stay at a specific location (e.g., staying in the toilet for more than 30 minutes is considered abnormal);

[0121] The extracted posture features include: posture angle: calculating the angles of various body parts through skeletal key point detection; limb movement: identifying abnormal movements (such as falling, fighting, etc.); movement amplitude: calculating the amplitude changes of limb movement.

[0122] 2) Feature vector construction:

[0123] It is constructed by the calculation formula of the student behavior feature vector, where w t 、w s 、w p Take 0.3, 0.4, and 0.3 respectively.

[0124] 3) Abnormal behavior identification:

[0125] The feature similarity is calculated by the feature similarity calculation formula, where the distance decay coefficient γ is 0.5, the scale parameter σ is 1.0, and the maximum time difference threshold T is max Take 300 seconds.

[0126] The abnormality degree score is calculated by the calculation formula of the abnormality degree score, wherein the time influence factor β0 is 0.2 and the reference duration D0 is 180 seconds.

[0127] Specifically, this embodiment achieves a comprehensive characterization of student behavior through multi-dimensional feature extraction, the weight distribution of feature vectors ensures balanced consideration of various features, and the normalization process ensures the comparability of different features and reduces the false alarm rate. The accuracy of recognition is improved by considering the scoring mechanism of time factors, and the robustness of abnormal behavior recognition is improved by using multi-dimensional feature fusion.

[0128] The risk level assessment of the abnormal behavior recognition result to obtain the abnormal behavior risk level specifically includes:

[0129] Based on historical data and expert experience, we quantitatively evaluate the risk level of different types of abnormal behaviors, analyze the impact of behavior duration, location sensitivity, environmental complexity, and population density on the risk level, and establish risk level classification standards;

[0130] According to the risk level classification standard, the abnormal behavior identification result is evaluated for risk level, the basic risk score of the abnormal behavior is calculated, the risk is weighted in combination with the duration of the abnormal behavior, and the regulatory effect of the scene characteristics on the risk is integrated to obtain a comprehensive risk score, and based on the preset level threshold, the comprehensive risk score is mapped to the corresponding abnormal behavior risk level, and the abnormal behavior risk level includes low risk, medium risk and high risk;

[0131] The calculation formula of the comprehensive risk score is:

[0132]

[0133] Among them, R is the comprehensive risk score, B is the basic risk score, T4 is the duration of abnormal behavior, T3 is the benchmark duration of abnormal behavior, S a is the scene influence factor, α1 is the basic weight coefficient, μ is the time influence coefficient, λ1 is the scene adjustment coefficient, L, E, M are location sensitivity, environment complexity, and personnel density, respectively, ω1, ω2, ω3 are the weight coefficients of location sensitivity, environment complexity, and personnel density, respectively.

[0134] In a specific embodiment, the specific steps of performing risk level assessment include:

[0135] 1) Establishment of risk level classification standards:

[0136] Based on historical data and expert experience, a quantitative assessment of different types of abnormal behaviors is conducted, and the weights of risk influencing factors are established: the location sensitivity weight (ω1) is 0.35, the environmental complexity weight (ω2) is 0.30, and the personnel density weight (ω3) is 0.35.

[0137] 2) Calculation of comprehensive risk score:

[0138] The calculation is performed using the formula for calculating the comprehensive risk score, wherein the basic weight coefficient α1 is 0.8, the time influence coefficient μ is 0.2, the scenario adjustment coefficient λ1 is 0.15, and the abnormal behavior duration benchmark time T3 is 300 seconds.

[0139] 3) Calculation of scene impact factors:

[0140] The calculation is performed using the formula for scene influence factors, where the location sensitivity L ranges from 0 to 1, is 0.3 for ordinary areas, is 0.6 for key areas, and is 0.9 for dangerous areas; the environment complexity E ranges from 0 to 1, is 0.2 for simple environments, is 0.5 for general environments, and is 0.8 for complex environments; the population density M ranges from 0 to 1, is 0.3 for sparse crowds, is 0.6 for general gathered crowds, and is 0.9 for dense crowds.

[0141] 4) Risk level classification:

[0142] The classification is based on the threshold of the comprehensive risk score R: when R < 0.4, it is classified as low risk; when 0.4 ≤ R < 0.7, it is classified as medium risk; when R ≥ 0.7, it is classified as high risk.

[0143] Specifically, this embodiment makes the risk level assessment more objective through a quantitative risk assessment model, improves the comprehensiveness of the assessment based on multi-dimensional risk factors, and ensures the reliability of the assessment results through parameter optimization of historical data. This embodiment can dynamically adjust the weight parameters according to different scenarios, introduce time impact factors to make the assessment more timely, and comprehensively consider scenario characteristics to improve the adaptability of the assessment.

[0144] Clear level divisions facilitate quick response by managers, quantitative evaluation results facilitate statistical analysis, and the evaluation process is automated, thus improving work efficiency.

[0145] According to the abnormal behavior risk level, triggering the corresponding level of warning information, confirming and optimizing the warning information, generating a final warning result, and pushing the final warning result to the management personnel, specifically includes:

[0146] Generate warning information including behavior type, location information, risk level and disposal suggestions according to the risk level of abnormal behavior, and conduct credibility assessment and priority sorting on the warning information;

[0147] Extract key feature information of abnormal behavior, including the time, location, duration and risk level of the behavior; establish a warning credibility assessment model based on historical warning data; generate targeted handling suggestions based on scenario information and historical handling experience; remove duplicate warning information and merge it to avoid duplicate warnings; prioritize warning information according to risk level and urgency;

[0148] Based on the risk level and priority of the warning information, determine the push targets and push methods of the warning information, establish a hierarchical push strategy, and push the warning information to managers at the corresponding level;

[0149] Establish a three-level early warning push mechanism, push low-risk warnings to general managers, medium-risk warnings to regional supervisors, and high-risk warnings to school safety managers at the same time; select the most suitable push targets based on the on-the-job status and location information of managers; adopt multi-channel push methods, including mobile APP push, SMS notifications and phone reminders; automatically upgrade and push warning information that has not been processed in time; record the processing process and results of warning information for subsequent optimization of warning strategies.

[0150] In a specific embodiment, the specific steps of generating and pushing the warning information include:

[0151] 1) Early warning information generation and evaluation:

[0152] The warning information includes: behavior type, such as climbing over a wall, fainting, and staying for a long time; location information, accurate to specific areas, such as a basketball court on the playground, a classroom on a certain floor of a certain unit in a certain building of Teaching Building A; risk level, such as low risk, medium risk, and high risk; handling suggestions, such as if a student fainted on the basketball court on the playground, immediately notify the playground management staff to go to the rescue.

[0153] Calculation of warning credibility score:

[0154] A weighted calculation is performed on the risk level, timeliness, and historical accuracy, with weights of 0.4, 0.3, and 0.3 respectively, to calculate the warning credibility of the corresponding warning information.

[0155] 2) Warning priority ranking:

[0156] The priority score is calculated by weighting the risk level and urgency, with weights of 0.6 and 0.4 respectively. For priority classification, warning information with a priority score of not less than 0.8 is marked as the highest priority; warning information with a priority score between 0.5 and 0.8 is marked as high priority; warning information with a priority score of not more than 0.5 is marked as normal priority.

[0157] 3) Hierarchical push strategy:

[0158] For low-risk warnings: the push targets are ordinary managers, the response time is 30 minutes, and the push method is APP push; for medium-risk warnings: the push targets are regional supervisors, the response time is 15 minutes, and the push method is APP push and SMS notification at the same time; for high-risk warnings: the push targets are school safety managers, the response time is 5 minutes, and the push method is APP push, SMS notification, and phone reminder at the same time.

[0159] 4) Early warning upgrade mechanism:

[0160] Timeout upgrade threshold: For low-risk warnings, if they are not processed for more than 45 minutes, they will be automatically upgraded to medium-risk; for medium-risk warnings, if they are not processed for more than 20 minutes, they will be automatically upgraded to high-risk. Re-push interval after upgrade: 5 minutes for the first time, 3 minutes for the second time, and 1 minute for the third time.

[0161] Specifically, this embodiment improves the accuracy of warning information through multi-dimensional information integration, reduces the false alarm rate through a credibility evaluation mechanism, and adopts hierarchical push to ensure the accuracy of information transmission.

[0162] This embodiment uses priority sorting to ensure that important warnings are handled in a timely manner, improves information delivery rate based on multi-channel push, and applies an automatic upgrade mechanism to prevent warnings from being ignored. Gradual push can reduce the workload of managers, provide disposal suggestions to improve processing efficiency, and leave warning records for subsequent optimization and analysis.

[0163] See also Figure 2 The present invention also provides a risk warning system for student behavior trajectory and action analysis, the system comprising:

[0164] A data acquisition module is used to collect video data and sensor data of student behavior through a camera array and multiple types of sensors;

[0165] A data processing module, used for preprocessing the video data and the sensor data to obtain multi-source data of student behavior;

[0166] A trajectory generation module is used to perform target detection and target tracking on the multi-source data of student behavior to generate a student movement trajectory;

[0167] An abnormality identification module is used to extract the student's behavior characteristics based on the student's movement trajectory, and to identify abnormal behaviors based on the behavior characteristics based on a preset normal behavior pattern library to obtain abnormal behavior identification results;

[0168] The risk assessment module is used to assess the risk level of abnormal behavior recognition results and obtain the abnormal behavior risk level;

[0169] The early warning push module is used to trigger the early warning information of the corresponding level according to the risk level of the abnormal behavior, confirm and optimize the early warning information, generate the final early warning result, and push the final early warning result to the management personnel.

[0170] Specifically, a risk warning system for student behavior trajectory and action analysis in this embodiment realizes the accurate collection, intelligent analysis and efficient warning of multi-source data, while ensuring the integrity of data processing and improving the efficiency of system operation, it also improves the level of intelligence of campus safety management.

[0171] The present invention also discloses an electronic device, comprising: at least one processor, at least one memory communication interface and a bus: wherein the processor, memory and communication interface communicate with each other through the bus; the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement a distributed photovoltaic power station drone inspection method.

[0172] The present invention also discloses a computer-readable storage medium, which stores computer instructions, and the computer instructions enable the computer to implement all or part of the steps of a distributed photovoltaic power station drone inspection method described in an embodiment of the present invention. The storage medium includes: a U disk, a mobile hard disk, a read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk, and other media that can store program codes.

[0173] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A risk warning method for student behavior trajectory and action analysis, characterized in that: The following steps are involved: Through camera arrays and multiple types of sensors, video data and sensor data of student behavior are collected respectively; Preprocessing the video data and the sensor data to obtain multi-source data of student behavior; Performing target detection and target tracking on the multi-source data of student behavior to generate student movement trajectories; Based on the student's movement trajectory, extract the student's behavior characteristics, and based on a preset normal behavior pattern library, perform abnormal behavior recognition on the behavior characteristics to obtain an abnormal behavior recognition result; Perform risk level assessment on abnormal behavior identification results to obtain abnormal behavior risk level; According to the abnormal behavior risk level, the corresponding level of warning information is triggered, the warning information is confirmed and optimized, and the final warning result is generated, and the final warning result is pushed to the management personnel.

2. A risk warning method for student behavior trajectory and action analysis as claimed in claim 1, characterized in that: The video data and sensor data of student behavior are collected through the camera array and multiple types of sensors, including: Deploy camera arrays in key areas of the campus, determine the installation position and angle of the camera based on the spatial distribution characteristics of the key areas of the campus, according to the field of view coverage requirements and occlusion conditions, optimize the spacing between adjacent cameras based on the spatial distance and field of view overlap, adjust the camera parameter settings based on the lighting conditions of the camera array's collection area, and collect video data of student behavior; Multiple types of sensors are deployed in key areas of the campus, including infrared sensors, sound sensors and motion sensors. The corresponding types of sensor data are collected through the multiple types of sensors. Infrared sensors are installed at the entrances and corridors of buildings to detect the passage of people; sound sensors are installed in classrooms and activity areas to collect abnormal sound signals; motion sensors are installed around fences and dangerous areas to detect abnormal movements.

3. A risk warning method for student behavior trajectory and action analysis as claimed in claim 1, characterized in that: The preprocessing of the video data and the sensor data to obtain the multi-source data of student behavior specifically includes: Performing data cleaning on the video data and the sensor data respectively, including removing noise data, filling missing values ​​and eliminating outliers; Performing image enhancement and noise reduction processing on the video data to obtain enhanced noise-reduced video data; applying a sliding window filtering algorithm to the sensor data to remove burst noise to obtain denoised sensor data; using an interpolation algorithm to repair missing data portions of the enhanced noise-reduced video data and the denoised sensor data, and identifying and eliminating outliers based on a statistical method to obtain cleaned video data and cleaned sensor data; Based on the timestamp information, the sampling time of the cleaned video data and the cleaned sensor data is uniformly converted into a standard time format, and the cleaned video data and the cleaned sensor data are resampled based on the minimum time interval to obtain resampled video data and resampled sensor data. The linear interpolation method is used to synchronize the time of the resampled video data and the resampled sensor data to obtain multi-source data of student behavior.

4. A risk warning method for student behavior trajectory and action analysis as claimed in claim 3, characterized in that: The target detection and target tracking of the multi-source data of student behavior to generate the student movement trajectory specifically includes: Extracting image features and signal features from the multi-source data of student behavior, calculating feature weights of different data sources, performing weighted fusion to obtain behavior fusion features, and inputting the behavior fusion features into a deep learning model to obtain target detection results; The calculation formula of the weighted fusion is: Among them, W i is the fusion weight of the i-th data source, R i is the characteristic reliability index of the i-th data source, S i is the feature significance index of the i-th data source, R j is the characteristic reliability index of the jth data source, S j is the feature significance index of the jth data source, α i , β i , α j , β j are the feature reliability adjustment factor of the i-th data source, the feature significance adjustment factor of the i-th data source, the feature reliability adjustment factor of the j-th data source, and the feature significance adjustment factor of the j-th data source, respectively. N1 is the number of data sources; The optimization calculation formula for target detection confidence is: Among them, C is the optimized confidence, C0 is the original confidence, λ is the confidence adjustment coefficient, N1 is the number of data sources, and W i is the fusion weight of the i-th data source, D i is the detection score of the i-th data source; Based on the target detection result, a target tracking algorithm with multi-feature fusion is used to track the detected target, and a target feature descriptor is constructed. The target feature descriptor includes the appearance feature and motion feature of the target. A Kalman filter algorithm is used to predict the target position, and data association is performed based on matching degree calculation to generate a continuous motion trajectory. The generated trajectory is evaluated for smoothness and optimized to generate a student motion trajectory with spatiotemporal information. The calculation formula for the smoothness evaluation is: Among them, S is the trajectory smoothness, Δθ k,k+1 is the angle change between the kth trajectory point and the k+1th trajectory point, w k is the time weight factor of the kth trajectory point, and N2 is the number of trajectory points minus 1.

5. A risk warning method for student behavior trajectory and action analysis as claimed in claim 1, characterized in that: The extracting of the student's behavior characteristics based on the student's movement trajectory, and performing abnormal behavior recognition on the behavior characteristics based on a preset normal behavior pattern library to obtain abnormal behavior recognition results specifically include: Based on the student's motion trajectory, extract the student's behavior characteristics, the behavior characteristics include spatiotemporal characteristics and posture characteristics, the spatiotemporal characteristics include motion speed, activity range, and dwell time, the posture characteristics include posture angle, limb movement, and motion amplitude, and construct a student behavior feature vector; The calculation formula of the student behavior feature vector is: F=[w t ×V t +w s ×V s +w p ×V p ]; Among them, F is the student behavior feature vector, V t 、V s 、V p are the time feature vector, space feature vector and posture feature vector respectively, w t 、w s 、w p are the weight coefficients of the temporal feature vector, spatial feature vector and posture feature vector, respectively, w t 、w s 、w p The sum is 1; Calculate the feature similarity between the student behavior feature vector and each pattern in the normal behavior pattern library, comprehensively consider the feature similarity and behavior duration, calculate the abnormality score, determine the abnormal behavior type according to the abnormality score, and output the abnormal behavior recognition result, which includes the abnormal behavior type and the abnormality score; The calculation formula of the feature similarity is: Among them, Sim(·) is the similarity function, F1 is a student behavior feature vector, F2 is the feature vector to be compared in the normal behavior pattern library, exp(·) is the exponential function, γ is the distance decay coefficient, σ is the scale parameter, T1 and T2 are the behavior durations corresponding to F1 and F2 respectively, T max is the maximum time difference threshold; The calculation formula of the abnormality score is: Among them, score(F0) is the abnormality score corresponding to another student behavior feature vector F0, max(Sim) is the maximum feature similarity value, β0 is the time influence factor, D(F0) is the behavior duration corresponding to F0, and D0 is the benchmark duration.

6. A risk warning method for student behavior trajectory and action analysis as claimed in claim 5, characterized in that: The risk level assessment of the abnormal behavior recognition result to obtain the abnormal behavior risk level specifically includes: Based on historical data and expert experience, we quantitatively evaluate the risk level of different types of abnormal behaviors, analyze the impact of behavior duration, location sensitivity, environmental complexity, and population density on the risk level, and establish risk level classification standards; The risk level classification standard is used to evaluate the risk level of abnormal behavior identification results, calculate the basic risk score of abnormal behavior, weight the risk based on the duration of abnormal behavior, and comprehensively consider the regulatory effect of scenario characteristics on risk to obtain a comprehensive risk score. Based on a preset level threshold, the comprehensive risk score is mapped to a corresponding abnormal behavior risk level, and the abnormal behavior risk levels include low risk, medium risk and high risk.

7. A risk warning method for student behavior trajectory and action analysis as claimed in claim 1, characterized in that: According to the abnormal behavior risk level, triggering the corresponding level of warning information, confirming and optimizing the warning information, generating a final warning result, and pushing the final warning result to the management personnel specifically includes: Generate warning information including behavior type, location information, risk level and disposal suggestions according to the risk level of abnormal behavior, and conduct credibility assessment and priority sorting on the warning information; Based on the risk level and priority of the warning information, determine the push objects and methods of the warning information, establish a hierarchical push strategy, and push the warning information to managers at the corresponding level.

8. A risk warning system for student behavior trajectory and action analysis, characterized in that: The system comprises: A data acquisition module is used to collect video data and sensor data of student behavior through a camera array and multiple types of sensors; A data processing module, used for preprocessing the video data and the sensor data to obtain multi-source data of student behavior; A trajectory generation module is used to perform target detection and target tracking on the multi-source data of student behavior to generate a student movement trajectory; An abnormality identification module is used to extract the student's behavior characteristics based on the student's movement trajectory, and to identify abnormal behaviors based on the behavior characteristics based on a preset normal behavior pattern library to obtain abnormal behavior identification results; The risk assessment module is used to assess the risk level of abnormal behavior recognition results and obtain the abnormal behavior risk level; The early warning push module is used to trigger the early warning information of the corresponding level according to the risk level of the abnormal behavior, confirm and optimize the early warning information, generate the final early warning result, and push the final early warning result to the management personnel.

9. An electronic device, characterized in that: include: at least one processor, at least one memory, a communication interface, and a bus; The processor, memory, and communication interface communicate with each other via the bus, the memory stores program instructions executable by the processor, and the processor calls the program instructions to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions enable a computer to implement the method according to any one of claims 1 to 7.

Images