Campus bullying behavior intelligent identification method based on multi-view panoramic camera

Abstract

The invention relates to the technical field of campus bullying behavior identification methods, in particular to a campus bullying behavior intelligent identification method based on a multi-view panoramic camera, which comprises the following steps: collecting panoramic image information of a campus specified area through a multi-view panoramic camera device; acquiring spatial position parameters of the multi-view panoramic camera device; performing gridding and coordinate processing on the panoramic image information based on the panoramic image information and the spatial position parameters; performing scene positioning and abnormal behavior detection based on the panoramic image information after gridding and coordinate processing; based on the abnormal behavior detection result, obtaining the abnormal behavior detection result and a probability distribution diagram of abnormal types detected in each frame by using a space-time convolutional network; based on the panoramic image information, interactive postures of the students are analyzed to obtain intimacy among the students, and a student social relation graph is constructed; obtaining a training model based on the training data set and the federated learning framework; and inputting the panoramic image information into the training model to obtain an abnormal behavior prediction result.

Description

Technical Field

[0001] The present invention relates to the technical field of campus bullying behavior recognition methods, and more specifically, to a campus bullying behavior intelligent recognition method based on a multi-view panoramic camera. Background Art

[0002] As society pays more and more attention to campus safety issues, the prevention and identification of campus bullying has become an important topic in the field of education management. Traditional campus safety management mainly relies on manual inspections and fixed camera monitoring, which has many limitations. Manual inspections are limited by human resources and time, making it difficult to achieve all-weather, all-round supervision; while fixed cameras can provide continuous monitoring, there are often blind spots, and the large amount of video data generated needs to be manually reviewed one by one, which is inefficient and easy to miss important information.

[0003] In recent years, with the rapid development of artificial intelligence technology, some researchers have tried to apply computer vision and machine learning technology to the automatic identification of school bullying behavior. These methods are usually based on video data collected by a single camera and detect possible bullying behavior through pre-defined behavior patterns. However, this type of method still has some obvious shortcomings: first, the field of view of a single camera is limited, and it is difficult to fully cover the complex campus environment; second, the pre-defined behavior patterns are often too simplistic and difficult to adapt to the diversity and complexity of bullying behavior; third, these methods usually only focus on the behavioral characteristics of a single time point, ignoring the continuity of bullying behavior in time and space.

[0004] In addition, most existing automatic recognition methods only focus on the behavior itself, while ignoring the environmental context in which the behavior occurs and the social relationships between students. This often makes it difficult for the system to distinguish between normal playful behavior and real bullying, resulting in a large number of false positives and false negatives. At the same time, these methods usually output the recognition results directly and lack the explanation of the recognition process, which makes it difficult for educators to understand and verify the judgment basis of the system, thus affecting the credibility of the system in practical applications.

[0005] Finally, existing technologies still have the problem of insufficient privacy protection. Many systems need to store and process students’ personal information during the identification process, which may not only violate relevant privacy protection laws and regulations, but also cause concerns among students and parents, hindering the widespread application of the system.

[0006] In view of the problems existing in the above-mentioned prior art, there is an urgent need for an intelligent system that can comprehensively, accurately and efficiently identify campus bullying behaviors while taking into account privacy protection. The present invention is proposed to meet this demand. Summary of the invention

[0007] The present invention aims to solve the above-mentioned technical problems. The present invention provides an intelligent identification method for campus bullying behavior based on a multi-eye panoramic camera. The method realizes all-round monitoring of the campus environment and intelligent identification of bullying behavior by innovatively combining multi-eye panoramic camera technology, deep learning algorithm and graph neural network.

[0008] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0009] The intelligent identification method of campus bullying behavior based on multi-view panoramic cameras includes:

[0010] The acquisition steps include:

[0011] Collect panoramic image information of the designated area of ​​the campus through a multi-eye panoramic camera device;

[0012] Obtaining spatial position parameters of a multi-eye panoramic camera device;

[0013] Processing steps include:

[0014] Based on the panoramic image information and spatial position parameters, the panoramic image information is gridded and coordinate processed;

[0015] Based on the panoramic image information after gridding and coordinate processing, scene positioning and abnormal behavior detection are performed;

[0016] Based on the abnormal behavior detection results, a spatiotemporal convolutional network is used to obtain the abnormal behavior detection results and the probability distribution map of the abnormal types detected in each frame;

[0017] Based on the panoramic image information, the interactive postures of students are analyzed to obtain the intimacy between students and construct a social relationship diagram of students;

[0018] Based on the training data set and the federated learning framework, a training model is obtained;

[0019] Output steps include:

[0020] The panoramic image information is input into the training model to obtain the abnormal behavior prediction results.

[0021] Preferably, the gridding and coordinate processing specifically includes:

[0022] Divide the panoramic image into several parts according to a rectangular regular grid;

[0023] Number each section;

[0024] Map the panoramic images into the same coordinate system.

[0025] Preferably, the scene positioning and abnormal behavior detection specifically include:

[0026] Recognize panoramic image information based on convolutional neural network model;

[0027] Determine the type and number of moving objects;

[0028] Determine the position coordinates of the moving object.

[0029] Preferably, the using the spatiotemporal convolutional network to obtain the abnormal behavior detection result specifically includes:

[0030] Detecting behavioral patterns using the first branch network;

[0031] Use the second branch network to detect behavioral features;

[0032] Calculate the feature encoding similarity on the time series;

[0033] Get the behavior type label and the time when the abnormal behavior occurs.

[0034] Preferably, analyzing the student's interaction posture specifically includes:

[0035] Extract the coordinates of key points of each student's head, torso, and limbs;

[0036] Calculate the distance between any two students' key parts;

[0037] Calculate the posture similarity of the two students;

[0038] Judging the intimacy between students based on posture similarity.

[0039] Preferably, obtaining the training model specifically includes:

[0040] The students' status is divided into three categories: normal status, victim status, and violent status;

[0041] Use graph neural network model to process student state vector;

[0042] Calculate the intimacy between students;

[0043] Use Softmax classifier to get the final state of each student.

[0044] Preferably, obtaining the abnormal behavior prediction result specifically includes:

[0045] Representing panoramic image information as vector input;

[0046] The student behaviors are divided into five types: hitting, pushing, blackmailing, ostracizing, and isolating;

[0047] Use graph neural networks to infer the results of abnormal behavior from student states;

[0048] Pass the results to the client corresponding to each student;

[0049] The overall model is updated through the federated learning framework.

[0050] As a preference, it also includes:

[0051] Input the panoramic image information into the spatiotemporal convolutional network;

[0052] Obtain abnormal behavior detection results and abnormal type probability distribution diagram;

[0053] Among them, the abnormal type probability distribution map records the predicted probability of each pixel belonging to the five behaviors of beating, pushing, blackmailing, ostracizing and isolating.

[0054] As a preference, it also includes:

[0055] Perform face recognition on the panoramic image information to obtain the facial features of the students;

[0056] Train emotion recognition models based on facial features;

[0057] Use emotion recognition models to identify students’ emotional states.

[0058] As a preference, it also includes:

[0059] Build a campus map model;

[0060] Associating the panoramic image information with the campus geographical location information;

[0061] Analyze students’ activity trajectories and gathering behaviors based on the campus map model;

[0062] Identify high-risk areas for bullying in schools.

[0063] The method of the present invention has the following significant technical effects:

[0064] First, the invention uses a multi-view panoramic camera network, which greatly expands the monitoring range and effectively eliminates the problem of blind spots in traditional fixed cameras. This all-round coverage ensures that the system can capture suspicious behavior in any corner of the campus, greatly improving the comprehensiveness and reliability of bullying behavior identification.

[0065] Secondly, the present invention introduces advanced deep learning technologies such as spatiotemporal convolutional networks and graph neural networks, which enable the system to automatically learn and adapt to complex and changeable bullying behavior patterns. This method no longer relies on predefined simple behavior patterns, but can capture more subtle and complex behavioral characteristics, significantly improving the accuracy and robustness of recognition. In particular, this method can simultaneously consider the temporal continuity of behavior and the social relationships between students, which enables the system to better understand the context in which the behavior occurs, thereby more accurately distinguishing between normal interactions and bullying behavior.

[0066] Furthermore, the method of the present invention provides a visual display of behavior recognition results by generating an abnormal type probability distribution diagram. This intuitive display method greatly enhances the interpretability of the system, allowing educators to better understand and verify the judgment basis of the system, thereby improving the credibility and acceptance of the system in practical applications.

[0067] In addition, the present invention fully considers the issue of privacy protection during the design process. The system mainly processes abstract information such as behavioral characteristics and social relationships, and does not directly store and process students' personal identity information. This design not only ensures the recognition effect of the system, but also protects students' privacy to the greatest extent, which is conducive to the wide application and promotion of the system.

[0068] Finally, the method of the present invention has good scalability and adaptability. Through continuous online learning and model updating, the system can continuously adapt to emerging bullying behavior patterns and different campus environments, ensuring long-term effectiveness.

[0069] In general, the intelligent identification method of campus bullying behavior based on multi-eye panoramic cameras provided by the present invention comprehensively solves the problems of limited monitoring range, insufficient identification accuracy, poor interpretability, insufficient privacy protection, etc. in the prior art by innovatively combining multiple advanced technologies. This method can not only identify campus bullying behavior more comprehensively, accurately, and efficiently, but also provide valuable decision-making support for educators, which has important practical significance for improving the level of campus safety management and creating a good learning environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 The figure is an overall flow chart of the method of the present invention.

[0071] Figure 2 It is a logic block diagram of the gridding and coordinate processing of the present invention.

[0072] Figure 3 This is a logical block diagram of abnormal behavior identification of the present invention.

[0073] Figure 4 It is a logical block diagram of the posture estimation of the present invention.

[0074] Figure 5 It is a logic block diagram of the learning student identity feature vector of the present invention.

[0075] Figure 6 This is a logic block diagram for judging bullying behavior according to the present invention. DETAILED DESCRIPTION

[0076] like Figure 1-6 As shown, the present invention provides an intelligent identification method for campus bullying behavior based on multi-eye panoramic cameras. The method realizes all-round monitoring of the campus environment by arranging multiple panoramic cameras in the campus environment, and uses advanced artificial intelligence technology to analyze the collected video data, thereby realizing intelligent identification and early warning of campus bullying behavior.

[0077] Specifically, the method of the present invention comprises the following steps:

[0078] First, in the acquisition step, the method collects panoramic video data of the campus environment through a multi-eye panoramic camera. These cameras can be installed in key areas of the campus such as teaching buildings, playgrounds, and canteens to ensure comprehensive coverage of the entire campus environment. Preferably, these panoramic cameras use 360-degree no-dead-angle shooting technology, and the resolution of each camera is not less than 4K, and the frame rate is not less than 30fps to ensure the clarity and smoothness of the video data. At the same time, the method also obtains the spatial position parameters of these multi-eye panoramic cameras. These parameters include information such as the installation position coordinates and shooting angles of each camera, which provide important spatial references for subsequent data processing.

[0079] Next, in the processing step, the method of the present invention first establishes a camera network based on the collected panoramic video data and the camera spatial position parameters. This network integrates the data of all cameras to form a unified data processing platform. Preferably, the present invention adopts a distributed computing architecture to distribute data processing tasks to multiple computing nodes to improve processing efficiency.

[0080] Then, the method performs gridding and coordinate processing on the panoramic video data to obtain gridded video data. Specifically, each frame of the panoramic image is divided into N×M grids, where N and M represent the number of grids in the horizontal and vertical directions, respectively. Preferably, the value range of N and M is between 20 and 50, which can ensure sufficient detail information without causing excessive computational burden. Each grid is assigned a unique coordinate (i, j), where i∈[1,N], j∈[1,M]. This processing method makes it possible to accurately locate any area in the video, laying the foundation for subsequent behavioral analysis.

[0081] Next, the method of the present invention uses a spatiotemporal convolutional network to learn behavior patterns based on gridded video data to obtain behavior pattern features. In this step, an improved 3D convolutional neural network model is used, and the mathematical expression of the model is as follows:

[0082]

[0083] Among them, F(x) is the output feature map, x is the input data, and w ijt is the convolution kernel weight, b is the bias term, and k is half of the convolution kernel size. Preferably, a 3x3x3 convolution kernel is used, that is, k = 1. This setting can reduce the computational complexity while ensuring the model performance.

[0084] Based on the obtained behavioral pattern characteristics, this method further identifies abnormal behaviors and obtains the ID of students with potential bullying behavior. This step uses an anomaly detection algorithm based on the attention mechanism, and its core formula is as follows:

[0085]

[0086] Where Q, K, and V represent the query, key, and value matrices, respectively. k is the dimension of the key vector. This mechanism can effectively capture the long-term dependencies in video sequences and improve the accuracy of abnormal behavior recognition.

[0087] Subsequently, the method of the present invention performs posture estimation on the panoramic video data to obtain the student posture features. An improved OpenPose model is used, which can accurately identify and track the key points of multiple people's bodies. Specifically, for each detected student, the coordinates of 17 key points are extracted, including the head, neck, shoulders, elbows, wrists, hips, knees and ankles. The coordinate information of these key points constitutes the student's posture features.

[0088] Based on the obtained student posture features, this method further extracts behavior sequence features. This step uses the long short-term memory network (LSTM), and its core formula is as follows:

[0089] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0090] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0091]

[0092] o t =σ(W o ·[h t-1 ,x t ]+b o )

[0093] h t =o t *tanh(C t )

[0094] Among them, f t 、i t , o t Respectively represent the forget gate, input gate and output gate, C t Indicates the unit state, h t represents the hidden state, W and b represent the weight matrix and bias term respectively, and σ represents the sigmoid activation function. This network structure can effectively capture the temporal dependencies in the behavior sequence.

[0095] Finally, the method of the present invention uses a graph neural network to learn the student identity feature vector based on the behavior sequence features and student identity labels. The graph convolutional network (GCN) model is used, and its core formula is as follows:

[0096]

[0097] Among them, H (l) represents the feature matrix of the ιth layer, represents the adjacency matrix with self-connection added, express The degree matrix, W (l) represents the weight matrix of the lth layer, and σ represents the activation function. This model can effectively utilize the social relationship information between students and improve the representation ability of identity features.

[0098] In the output step, the method of the present invention first calculates the feature matching degree based on the behavior sequence feature and the student identity feature vector. Here, cosine similarity is used as the matching degree metric, and its formula is as follows:

[0099]

[0100] Among them, A and B represent the behavior sequence feature and student identity feature vector respectively, and ||·|| represents the Euclidean norm of the vector.

[0101] Finally, based on the calculated feature matching degree, this method determines whether bullying behavior has occurred and outputs the judgment result. Specifically, a threshold σ (preferably, σ = 0.8) is set. When the matching degree is greater than σ, it is determined that bullying behavior has occurred. The selection of this threshold is based on a large amount of experimental data, which can reduce the false alarm rate while ensuring the recognition accuracy.

[0102] Through the above steps, the method of the present invention can effectively identify bullying behaviors in the campus environment and provide strong support for campus safety management. Compared with traditional methods, the present invention has the following advantages: first, the use of multi-eye panoramic cameras ensures comprehensive coverage of the campus environment and reduces monitoring blind spots; second, advanced artificial intelligence algorithms can automatically analyze large amounts of video data, greatly improving recognition efficiency; finally, this method can not only identify known bullying patterns, but also adapt to new abnormal behaviors through continuous learning, and has strong adaptability and scalability.

[0103] In a preferred embodiment of the present invention, the gridding and coordinate processing steps are further refined to achieve accurate division and positioning of the panoramic image. Specifically, the steps include the following operations:

[0104] First, the method divides the panoramic image into several parts according to a rectangular regular grid. Preferably, the size of the grid can be adjusted according to the actual application scenario. For example, in a typical campus environment, the 360-degree panoramic image can be divided into an 18×9 grid, that is, one unit for every 20 degrees in the horizontal direction and one unit for every 20 degrees in the vertical direction. This division method can ensure sufficient accuracy without causing excessive computational burden.

[0105] Then, each grid section is numbered. The numbering method can be row-first or column-first, for example, starting from the upper left corner, (1,1), (1,2), ..., (18,9). This numbering method facilitates subsequent area positioning and behavior analysis.

[0106] Finally, the method maps the panoramic image to the same coordinate system. In one embodiment, a spherical coordinate system can be used to represent the coordinates of the center point of each grid as (θ, φ), where θ represents the azimuth angle and φ represents the pitch angle. This mapping method can accurately reflect the spatial position relationship of each part in the panoramic image and provide important spatial information for subsequent behavior analysis.

[0107] Through such gridding and coordinate processing, the method of the present invention can convert complex panoramic images into structured data, greatly improving the efficiency and accuracy of subsequent processing.

[0108] In another embodiment of the present invention, the scene positioning and abnormal behavior detection steps are further refined to achieve accurate recognition and positioning of moving objects in the panoramic image. Specifically, the steps include the following operations:

[0109] First, the method recognizes panoramic image information based on a convolutional neural network model. In a preferred embodiment, an improved YOLOv3 (You Only Look Once version 3) model is used as the infrastructure. The model can effectively recognize targets of different sizes by making predictions at different scales, and is particularly suitable for processing complex campus scenes. In order to adapt to the characteristics of panoramic images, the method adds a module for processing spherical projection deformation on the basis of the original model to improve the accuracy of recognition.

[0110] Secondly, this method determines the category and number of moving objects. In the scenario of school bullying behavior recognition, the main focus is on students. Therefore, this method divides the targets into two categories: "students" and "non-students". For the "student" category, the system will further record each detected student instance to obtain the number of students. This classification method can not only focus on key objects, but also reduce the complexity of the system.

[0111] Finally, this method determines the position coordinates of the moving object. Based on the previous gridding and coordinate processing, the system can accurately locate each detected object. Specifically, for each detected student, the system records the grid number and the relative position within the grid. This positioning method not only retains global information, but also provides local precise location, laying the foundation for subsequent behavior analysis.

[0112] Through such scene positioning and abnormal behavior detection steps, the method of the present invention can quickly and accurately identify key objects and their positions in the panoramic image, providing important input information for subsequent behavior analysis.

[0113] In another embodiment of the present invention, the step of obtaining abnormal behavior detection results using a spatiotemporal convolutional network is further refined to achieve accurate recognition of complex behavior patterns. Specifically, the step includes the following operations:

[0114] First, the method uses the first branch network to detect behavior patterns. This branch network adopts a 3D convolutional neural network structure, which can capture features in both spatial and temporal dimensions.

[0115] In a preferred embodiment, the network contains 5 3D convolutional layers, each followed by a batch normalization layer and a ReLU activation function. This design can effectively capture complex spatiotemporal behavior patterns, such as typical bullying behaviors such as pushing and hitting.

[0116] Secondly, the method uses a second branch network to detect behavioral features. This branch network focuses on extracting more fine-grained behavioral features, such as the speed and acceleration of body movements. In one embodiment, the network adopts a structure similar to the Inception module, including multiple parallel convolution branches, each branch using a convolution kernel of a different size. This design can capture behavioral features of different scales and improve the robustness of detection.

[0117] Then, the method calculates the similarity of feature encodings over the time series. Specifically, the system compares the feature encodings of multiple consecutive frames and calculates the cosine similarity between them. This step aims to capture the persistence and changing trends of the behavior, which helps to distinguish between normal short-term interactions and persistent bullying behavior.

[0118] Finally, this method obtains the behavior type label and the time when the abnormal behavior occurs. Based on the output of the previous steps, the system uses a fully connected layer followed by a softmax layer to predict the behavior type, such as "normal", "potential bullying", "confirmed bullying", etc. At the same time, the system also records the start and end timestamps of the abnormal behavior for subsequent intervention and analysis.

[0119] Through this abnormal behavior detection method based on spatiotemporal convolutional networks, the present invention can effectively capture complex bullying behavior patterns and provide detailed behavior type and time information. This method can not only improve the accuracy of detection, but also provide educators with more valuable information, which is helpful for formulating targeted intervention measures.

[0120] Next, the method of the present invention performs posture estimation on the panoramic video data to obtain the student posture features. In a preferred embodiment of the present invention, the posture estimation step specifically includes the following contents:

[0121] First, the method extracts the key point coordinates of each student's head, torso and limbs. Preferably, an improved OpenPose model is used to achieve this goal. The model can accurately identify and track the key points of multiple students' bodies in complex campus scenes. Specifically, for each detected student, the method extracts the coordinates of 17 key points, including the top of the head, the tip of the nose, the left and right eyes, the left and right ears, the neck, the left and right shoulders, the left and right elbows, the left and right wrists, the left and right hips, the left and right knees and the left and right ankles. The selection of these key points is based on the anatomical characteristics of the human body and can fully reflect the posture information of the students.

[0122] Subsequently, the method of the present invention calculates the distance between the key parts of any two students. The purpose of this step is to quantify the spatial relationship between students and provide an important basis for subsequent behavioral analysis. Specifically, for any two students A and B, the method calculates the following distances:

[0123] 1. Head distance: d h ead=||P A,head -P B,head || 2

[0124] 2. Torso distance: d t orso=||P A,torso -P B,torso || 2

[0125] 3. Limb distance:

[0126] Among them, R X,Y Indicates the center coordinates of the student's X and Y parts, |||·||| 2 represents the Euclidean distance.

[0127] Next, the method calculates the posture similarity of the two students. In one embodiment of the present invention, the calculation formula of the posture similarity S is as follows:

[0128]

[0129] Among them, w 1 、w 2 、w 3 is a weight coefficient used to adjust the importance of distances between different parts. Preferably, w 1 =0.4, w 2 =0.3, w 3 = 0.3. σ is a scaling factor used to control the sensitivity of similarity, preferably, σ = 100. The selection of these parameters is based on a large amount of experimental data and can effectively capture the posture similarity between students.

[0130] Finally, the method of the present invention determines the close relationship between students based on posture similarity. Specifically, a threshold θ (preferably, θ=0.7) is set, and when the posture similarity S>θ, the two students are judged to have a close relationship. The selection of this threshold takes into account the balance between normal social behavior and possible bullying behavior, which can effectively identify students who interact closely without being overly sensitive and causing misjudgment.

[0131] In another embodiment of the present invention, the method uses graph neural network learning to obtain student identity feature vectors. The specific implementation of this step is as follows:

[0132] First, this method divides the status of students into three categories: normal status, victim status, and violent status. This classification is based on the research results of psychology and education, and can fully cover the possible status of students in school bullying scenarios.

[0133] Then, the method of the present invention uses a graph neural network model to process the student state vector. In a preferred embodiment, a graph attention network (GAT) is used to achieve this goal. The core formula of GAT is as follows:

[0134]

[0135] in, represents the feature vector of node i in layer l, represents the neighbor set of node i, W (l) is the weight matrix of the lth layer, α ij is the attention coefficient, σ is the activation function. Attention coefficient α ij The calculation formula is as follows:

[0136]

[0137] Among them, a is a learnable attention vector and || represents a vector concatenation operation.

[0138] Next, the method calculates the intimacy between students. In one embodiment of the present invention, the calculation formula of the intimacy C_{ij} is as follows:

[0139]

[0140] Among them, t ij represents the interaction time between student i and student j, Represents the maximum interaction time between all pairs of students. This calculation method takes into account both the attention weights between students and the impact of interaction time.

[0141] Finally, the method of the present invention uses the Softmax classifier to obtain the final state of each student. Specifically, for student i, the calculation formula of its state probability distribution is as follows:

[0142]

[0143] in, is the feature vector of student i in the last layer of the graph neural network, W s and b s are the weight matrix and bias vector of the Softmax classifier respectively.

[0144] Through the above steps, the method of the present invention can effectively capture the complex social relationships between students and accurately identify the status of each student. This method based on graph neural network has significant advantages over traditional methods: it can automatically learn the interaction patterns between students, adapt to different campus environments and bullying types, and has strong generalization and robustness. In another embodiment of the present invention, the method also includes inputting panoramic video data into a spatiotemporal convolutional network to obtain abnormal behavior detection results and abnormal type probability distribution maps. The introduction of this step further improves the accuracy and interpretability of bullying behavior identification.

[0145] Specifically, the present invention adopts an improved 3D convolutional neural network as the basic architecture of the spatiotemporal convolutional network. The core operation of the network can be expressed as:

[0146]

[0147] Among them, F(x,y,z,t) represents the value of the output feature map at position (x,y,z) and time t, I represents the input data, w represents the convolution kernel weight, k 1 ,k 2 ,k 3 ,k 4 Represent the sizes of the convolution kernel in spatial and temporal dimensions respectively.

[0148] Preferably, the method adopts a multi-scale convolution structure and uses convolution kernels of different sizes to capture spatiotemporal features of different scales. For example, 3x3, 5x5, and 7x7 convolution kernels can be used simultaneously, and then the obtained feature maps are fused. This design can effectively capture bullying behavior patterns of different scales.

[0149] The abnormal type probability distribution map is an important innovation of the present invention. Each pixel of the distribution map corresponds to an area in the original video frame, and its value represents the probability of the area belonging to various abnormal behavior types. Specifically, for the position (x, y) in the video frame, its abnormal type probability distribution can be expressed as:

[0150] P(x,y)=[p 1 (x,y),p 2 (x,y),p 3 (x,y),p 4 (x,y),p 5 (x,y)]

[0151] Among them, p 1 ,p 2 ,p 3 ,p 4 ,p 5The probabilities of the five behaviors of hitting, pushing, extorting, ostracizing, and isolating are shown respectively. This visualization method can not only accurately locate the location where abnormal behaviors occur, but also provide detailed information on the types of behaviors, providing important reference for subsequent intervention measures.

[0152] In another embodiment of the present invention, the method further includes performing face recognition on the panoramic video data to obtain facial features of the students. The introduction of this step enables the method to more accurately identify and track the behavior of each student.

[0153] Preferably, the present invention adopts a deep residual network (ResNet) as the basic model for face recognition. The core idea of ​​ResNet is to introduce residual learning, and its basic structure can be expressed as:

[0154] H(x)=F(x)+x

[0155] Among them, H(x) is the desired underlying mapping, F(x) is the residual mapping, and x is the input. This structure can effectively solve the gradient vanishing problem of deep networks and improve the accuracy of face recognition.

[0156] Based on the obtained facial features, the method further trains an emotion recognition model. In a preferred embodiment, an attention mechanism-enhanced long short-term memory network (LSTM) is used to achieve emotion recognition. The core formula is as follows:

[0157]

[0158] h t =o t ⊙tanh(c t )

[0159] α t =softmax(W α [h t ;c t ]+b α )

[0160]

[0161] Among them, c t and h t Respectively represent the cell state and hidden state of LSTM, f t ,i t ,o t They represent the forget gate, input gate and output gate respectively. represents the candidate unit state, α t represents the attention weight, W α and b α are the weight matrix and bias vector of the attention layer respectively.

[0162] Through this emotion recognition model, this method can monitor students' emotional states in real time, providing important clues for early identification of bullying behavior. For example, if a student is detected to be in a state of fear or sadness for a long time, the system will automatically increase its attention to the student.

[0163] Finally, in one embodiment of the present invention, the method further includes establishing a campus map model to associate the panoramic video data with the campus geographic location information. The introduction of this step enables the method to analyze and predict bullying behavior at a more macro level.

[0164] Specifically, this method first builds a 3D campus model, including all buildings, playgrounds, corridors and other key areas. Then, the field of view of each camera is mapped onto this 3D model. In this way, each detected event can be accurately located at a specific location on campus.

[0165] Based on this campus map model, this method further analyzes student activity trajectories and aggregation behaviors. Preferably, a spatiotemporal clustering algorithm is used to achieve this goal. The core idea is to combine the time and space dimensions and define a spatiotemporal distance metric:

[0166]

[0167] Among them, (x, y) represents the spatial coordinates, t represents time, and λ is a trade-off coefficient used to adjust the relative importance of time and space dimensions.

[0168] By analyzing these spatiotemporal clustering results, this method can identify high-risk areas for school bullying. For example, if a remote corner often sees students gathering and is accompanied by abnormal behavior detection, then this area will be marked as a high-risk area. School administrators can adjust their supervision strategies accordingly, such as increasing the frequency of patrols or installing additional monitoring equipment.

[0169] In general, the intelligent identification method of campus bullying behavior based on multi-view panoramic cameras provided by the present invention realizes the all-round and multi-angle identification of campus bullying behavior by combining advanced computer vision technology, deep learning algorithms and spatial analysis methods. This method can not only accurately detect bullying incidents that have occurred, but also predict potential high-risk areas, providing strong technical support for campus safety management.

[0170] The above description is only a preferred specific implementation manner of the present invention; however, the protection scope of the present invention is not limited thereto; any person familiar with the art who, within the scope disclosed by the present invention, makes equivalent replacements or changes based on the scheme and improved concepts of the present invention shall be covered within the protection scope of the present invention.

Claims

1. An intelligent identification method for campus bullying behavior based on multi-view panoramic cameras, characterized in that: include: The acquisition steps include: Collect panoramic image information of the designated area of ​​the campus through a multi-eye panoramic camera device; Obtaining spatial position parameters of a multi-eye panoramic camera device; Processing steps include: Based on the panoramic image information and spatial position parameters, the panoramic image information is gridded and coordinate processed; Based on the panoramic image information after gridding and coordinate processing, scene positioning and abnormal behavior detection are performed; Based on the abnormal behavior detection results, a spatiotemporal convolutional network is used to obtain the abnormal behavior detection results and the probability distribution map of the abnormal types detected in each frame; Based on the panoramic image information, the interactive postures of students are analyzed to obtain the intimacy between students and construct a social relationship diagram of students; Based on the training data set and the federated learning framework, a training model is obtained; Output steps include: The panoramic image information is input into the training model to obtain the abnormal behavior prediction results.

2. The method according to claim 1, characterized in that The gridding and coordinate processing specifically includes: Divide the panoramic image into several parts according to a rectangular regular grid; Number each section; Map the panoramic images into the same coordinate system.

3. The method according to claim 1, characterized in that The scene positioning and abnormal behavior detection specifically include: Recognize panoramic image information based on convolutional neural network model; Determine the type and number of moving objects; Determine the position coordinates of the moving object.

4. The method according to claim 1, characterized in that: The abnormal behavior detection result obtained by using the spatiotemporal convolutional network specifically includes: Detecting behavioral patterns using the first branch network; Use the second branch network to detect behavioral features; Calculate the feature encoding similarity on the time series; Get the behavior type label and the time when the abnormal behavior occurs.

5. The method according to claim 1, characterized in that The analyzing the student's interaction posture specifically includes: Extract the coordinates of key points of each student's head, torso, and limbs; Calculate the distance between any two students' key parts; Calculate the posture similarity of the two students; Judging the intimacy between students based on posture similarity.

6. The method according to claim 1, characterized in that The obtaining of the training model specifically includes: The students' status is divided into three categories: normal status, victim status, and violent status; Use graph neural network model to process student state vector; Calculate the intimacy between students; Use Softmax classifier to get the final state of each student.

7. The method according to claim 1, characterized in that The abnormal behavior prediction result obtained specifically includes: Representing panoramic image information as vector input; The student behaviors are divided into five types: hitting, pushing, blackmailing, ostracizing, and isolating; Use graph neural networks to infer the results of abnormal behavior from student states; Pass the results to the client corresponding to each student; The overall model is updated through the federated learning framework.

8. The method according to claim 1, characterized in that Also includes: Input the panoramic image information into the spatiotemporal convolutional network; Obtain abnormal behavior detection results and abnormal type probability distribution diagram; Among them, the abnormal type probability distribution map records the predicted probability of each pixel belonging to the five behaviors of beating, pushing, blackmailing, ostracizing and isolating.

9. The method according to claim 1, characterized in that: Also includes: Perform face recognition on the panoramic image information to obtain the facial features of the students; Train emotion recognition models based on facial features; Use emotion recognition models to identify students’ emotional states.

10. The method according to claim 1, characterized in that Also includes: Build a campus map model; Associating the panoramic image information with the campus geographical location information; Analyze students’ activity trajectories and gathering behaviors based on the campus map model; Identify high-risk areas for bullying in schools.

Images