A predictability behavior recognition method based on a neural network model

Abstract

The application discloses a foreseeability behavior recognition method based on a neural network model, and comprises the following steps: collecting a foreseeability behavior data set; constructing a behavior recognition model, improving a neural network model VideoMAE, including an encoder, a decoder and a mask strategy, wherein the encoder comprises input data preprocessing and improved feature extraction, the improved feature extraction changes an original space-time joint self-attention mechanism into a multi-head self-attention feature extraction module and a local convolution attention feature extraction module to realize feature extraction together, and then linear weighting fusion is performed to obtain final behavior features F; training the behavior recognition model; and applying the trained behavior recognition model to perform foreseeability behavior recognition. The application has the characteristics of high recognition accuracy, good real-time performance and strong robustness.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a foreseeability behavior recognition method based on a neural network model. BACKGROUND

[0002] Stroke is a major chronic non-communicable disease that endangers the health of the nation, which is divided into two types of ischemic stroke and hemorrhagic stroke, about 60% to 80% of which is ischemic stroke. Active and effective intervention on the risk factors of stroke can reduce the incidence of stroke, however, the effect of stroke treatment is strongly related to time, if the patient receives intravascular recanalization treatment within 6 hours of onset, good clinical efficacy can be obtained. Elderly patients living alone and lacking awareness of stroke are more likely to have pre-hospital delay, and they are delayed in treatment because they cannot be discovered in time. Therefore, it is of great significance to study the intelligent perception of potential stroke patients in the home environment for the foreseeable behavior of saving lives.

[0003] In the prior art, deep learning technology is applied to the early identification of diseases. Some researches combine the home environment and propose an intelligent recognition method for depression based on voice signals, design a new integrated support vector machine algorithm, and provide strong support for the early diagnosis of depression patients. Some researches propose a cloud-based computerized sepsis clinical decision system, which identifies inflammatory syndromes according to the patient's infectious diseases and comorbidities, and realizes early warning of sepsis. Some researches propose a Squeeze-and-Excitation Residual Networks-based MRI classification and early diagnosis model for children with viral encephalitis, which has an accuracy of 70.83%, providing strong support for the early diagnosis of children with viral encephalitis. In view of the low prediction accuracy of cardiovascular diseases, some researches propose a one-dimensional convolutional neural network-based cardiovascular disease prediction model, which analyzes 13 kinds of physiological data such as blood glucose, and explores the relationship between physiological data and cardiovascular diseases. The accuracy is more than 90%, which is better than the prediction model based on the particle swarm algorithm. Some researches propose a transfer learning model combined with deep learning to identify signs of cognitive diseases. The average accuracy is 67.6%, which reduces the difficulty of screening early signs of cognitive diseases. However, these methods have low recognition accuracy, poor real-time performance, poor robustness, and lack of methods to observe and understand the early foreseeability behavior of diseases through daily activities. SUMMARY

[0004] The present application aims to overcome the above-mentioned shortcomings and proposes a foreseeability behavior recognition method based on a neural network model with high recognition accuracy, good real-time performance and strong robustness.

[0005] The foreseeability behavior recognition method based on a neural network model comprises the following steps:

[0006] Step 1: Collect a dataset of predictable behaviors, predefine a category C for predictable behaviors, and label the data in the dataset with category labels; divide the dataset into a training set, a test set, and a validation set;

[0007] Step 2: Behavior Recognition Model Construction: This behavior recognition model is an improvement upon the VideoMAE neural network model, including an encoder, a decoder, and a masking strategy. The encoder includes input data preprocessing and improved feature extraction; the improved feature extraction includes a multi-head self-attention feature extraction module. Local convolutional attention feature extraction module Its behavior recognition process is as follows:

[0008] Step 2.1: The encoder's input data preprocessing involves dividing the input data in the dataset into individual frame sequences, performing temporal downsampling according to frame t, and then performing center clipping and size normalization to obtain the data sequence T{T1,T2,…,T…}. t Then, the data sequence T{T1,T2,…,T} is processed. t Improved feature extraction from the input encoder;

[0009] Step 2.2: The improved feature extraction first involves processing the input data sequence T{T1,T2,…,T…}. t The process involves dividing the data into multiple smaller blocks based on spatial and temporal dimensions, resulting in P{P1,P2,…,P}. t}, then set P{P1,P2,…,P t Perform a masking operation on the tube, randomly selecting a certain proportion for masking, and then masking the remaining P{P1,P2,…,P}. t After passing through a linear embedding layer, high-dimensional feature information h{h1,h2,…,h is obtained. t};

[0010] Then, the high-dimensional feature information h{h1,h2,…,h t Input multi-head attention feature extraction module Multi-head attention is used to segment the input high-dimensional feature information data sequence to form different attention heads, and behavioral features are captured in the entire spatiotemporal range through these attention heads; behavioral information is captured from a global perspective to obtain a global feature vector F with dimension M·N·3. global Where M is the width of the input image / number of frames, and N is the height of the input image / number of frames, as shown in the following formula:

[0011]

[0012] Where T represents a token data sequence sample, T∈RB×N×C C is the number of channels, N is t × width × height of the input image, B is the number of samples per round; t represents the number of frames input to the model, A n Let be a 1×C×C attention parameter matrix, where ⊙ denotes element-wise multiplication of the matrix elements;

[0013] Step 2.3: Convert the data sequence T{T1,T2,…,T…} t The input is fed into the local convolutional attention feature extraction module. First, consider the input sequence T{T1,T2,…,T…} t The process involves performing a Conv1×1 convolution to obtain local feature information Q and K. Then, Q and K are multiplied by a matrix to obtain static information attn. Attn is then subjected to a classification softmax operation and multiplied by a matrix T to obtain local dynamic features. Finally, after performing a Conv1×1 convolution, LayerNorm normalization, ReLU activation, and another Conv1×1 convolution, a local feature vector F with an output dimension of M·N·3 is obtained. local ;

[0014] Step 2.4: Transfer global features F global and local features F local A linear weighted fusion is performed to generate the final behavioral feature F; the feature fusion formula is:

[0015] F=αF global +(1-α)F local

[0016] Where α∈[0,1] represents the feature fusion weight coefficient, which balances the importance of global features and local features;

[0017] Step 2.5: Input the fused feature F into the decoder, and obtain the predicted category label C of the sequence through classification. t and the corresponding category score S;

[0018] Step 3: Train the behavior recognition model: Input the training dataset into the behavior recognition network model for training; evaluate and verify the trained behavior recognition network model using the test set and validation set;

[0019] Step 4: Apply the trained behavior recognition model to perform predictable behavior recognition.

[0020] The aforementioned predictable behavior recognition method based on a neural network model, wherein in step 1, the dataset is divided into a training set, a test set, and a validation set in a ratio of 6.5:1.5:2.

[0021] In the above-mentioned predictable behavior recognition method based on a neural network model, in step 2.1, the input data is divided into separate frame sequences, and the number of frames is set to 90 frames.

[0022] In the above-mentioned predictable behavior recognition method based on a neural network model, in step 2.1, the temporal downsampling is performed according to t frames, and the sampling rate is set to 8 and 16.

[0023] The aforementioned predictable behavior recognition method based on a neural network model, wherein: in step 2.2, 90% of the randomly selected portion is masked, and the remaining 10% of P{P1,P2,…,P} is masked. t It passes through a linear embedding layer.

[0024] The above-mentioned predictable behavior recognition method based on a neural network model includes the following step: In step 3, the behavior recognition model is trained with an initial learning rate of lr = 1e-3, an optimizer of Adamw, an action category of 8, and a shortest side cropping size of 224.

[0025] The aforementioned predictable behavior recognition method based on a neural network model, wherein: in step 4, the application of the trained behavior recognition model to perform predictable behavior recognition, specifically includes the following steps:

[0026] Step 4.1: Acquire video V from the camera real The trained behavior recognition model is input to perform predictable behavior recognition and classification, and the predicted category label C of the sequence is obtained. t and the corresponding category score S;

[0027] Step 4.2: Add the predicted category label C t The frame sequence containing the corresponding category score S is combined with the current timestamp time to form the final output sequence Vpre t{time,C t ,S};

[0028] Step 4.3: Determine the category label C t Does it belong to the predefined category of predictable behavior C? If C t If the behavior falls under the predictable behavior category C, then the resulting sequence Vpre t{time,C} will be... t Store S in the database and output the result; otherwise, return to step 4.1 and continue the identification process.

[0029] Compared with existing technologies, this invention has significant advantages. As can be seen from the above scheme, this invention addresses the challenge of identifying abnormal behaviors in the early stages of disease, fully considering the needs of early warning, and innovatively optimizes recognition accuracy, real-time performance, and multi-angle modeling of anomaly detection. A dual-path structure combining a global multi-head attention module and a local convolutional attention module is designed. First, the multi-head attention module extracts behavioral features from a full-time-space perspective, dividing the input data into multiple attention heads to capture important features in video frames from different angles, obtaining global features. Global feature extraction helps the model grasp the spatiotemporal dynamic relationship of behavior macroscopically, and is particularly suitable for identifying long-term changes in behavior. Simultaneously, the local convolutional attention module processes video data frame by frame, focusing on short-term local dynamic feature extraction, and can identify subtle changes within frames, such as gait abnormalities and slight swaying. This structure allows the model to consider both macroscopic and microscopic details, effectively improving the model's sensitivity to behavior in early diseases and accurately capturing subtle abnormal manifestations in patients. To fully leverage the advantages of global and local features, this invention uses a linear weighted fusion strategy to weight and combine global and local features to generate the final behavioral features. By assigning weights to global features, the model better balances its focus on macroscopic behavioral trends and local details, ensuring greater adaptability in dealing with different types of abnormal behaviors. This fusion mechanism enhances the model's generalization ability, enabling accurate identification of behaviors such as falls, headaches, and chest pain. It is particularly suitable for early detection of short-term abnormal behaviors during disease onset, making the model more robust in complex scenarios. Furthermore, this invention designs a real-time monitoring and high-confidence feedback process. During video acquisition from a camera, the method processes the video at a specified frame rate and continuously identifies abnormal behaviors. Based on real-time performance, the identified abnormal behaviors are filtered using a confidence matching strategy, ensuring the system records and issues warnings for high-confidence abnormal behaviors. Thus, when abnormal behaviors such as gait abnormalities and falls are identified, the system immediately provides feedback and stores the information in the database for real-time monitoring and subsequent intervention. This provides patients with a stable early warning and behavioral feedback mechanism, helping family members and medical staff to promptly understand the patient's abnormal state.

[0030] In summary, this invention not only excels in accurately identifying early abnormal behaviors in disease patients but also provides real-time and efficient feedback on abnormal states. Through the collaborative extraction, weighted fusion, and confidence feedback mechanism of global and local features, it ensures robustness in complex and diverse behavioral scenarios, effectively reducing the false negative rate and enhancing its practical value for early disease warning. In conclusion, this invention features high accuracy, good real-time performance, and strong robustness.

[0031] The following specific embodiments further illustrate the beneficial effects of the present invention. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the recognition model structure of the present invention. Detailed Implementation

[0033] The following detailed description, in conjunction with the accompanying drawings and preferred embodiments, describes the specific implementation, features, and effects of a predictable behavior recognition method based on a neural network model proposed in accordance with the present invention.

[0034] See Figure 1 The present invention provides a predictable behavior recognition method based on a neural network model, comprising the following steps:

[0035] Step 1: Collect a dataset of predictable behaviors, predefine a category C of predictable behaviors, and label the data in the dataset with category labels; divide the dataset into a training set, a test set, and a validation set; the dataset is divided into the training set, the test set, and the validation set in a ratio of 6.5:1.5:2.

[0036] Step 2: Behavior Recognition Model Construction: This behavior recognition model is an improvement upon the VideoMAE neural network model, including an encoder, a decoder, and a masking strategy. The encoder includes input data preprocessing and improved feature extraction; the improved feature extraction includes a multi-head self-attention feature extraction module. Local convolutional attention feature extraction module Its behavior recognition process is as follows:

[0037] Step 2.1: The encoder's input data preprocessing involves dividing the input data in the dataset into individual frame sequences, performing temporal downsampling according to frame t, and then performing center clipping and size normalization to obtain the data sequence T{T1,T2,…,T…}. t Then, the data sequence T{T1,T2,…,T} is processed. t Improved feature extraction from the input encoder;

[0038] The input data is segmented into a separate frame sequence, with the number of frames set to 90. Timing downsampling is performed according to frame t, with sampling rates set to 8 and 16.

[0039] Step 2.2: The improved feature extraction first involves processing the input data sequence T{T1,T2,…,T…}. t The process involves dividing the data into multiple smaller blocks based on spatial and temporal dimensions, resulting in P{P1,P2,…,P}. t}, then set P{P1,P2,…,P tPerform a masking operation on the tube, randomly selecting a certain proportion for masking, and then masking the remaining P{P1,P2,…,P}. t After passing through a linear embedding layer, high-dimensional feature information h{h1,h2,…,h is obtained. t};

[0040] The random selection of 90% for masking, and the remaining 10% of P{P1,P2,…,P t It passes through a linear embedding layer.

[0041] Then, the high-dimensional feature information h{h1,h2,…,h t Input multi-head attention feature extraction module Multi-head attention is used to segment the input high-dimensional feature information data sequence to form different attention heads, and behavioral features are captured in the entire spatiotemporal range through these attention heads; behavioral information is captured from a global perspective to obtain a global feature vector F with dimension M·N·3. global Where M is the width of the input image / number of frames, and N is the height of the input image / number of frames, as shown in the following formula:

[0042]

[0043] Where T represents a token data sequence sample, T∈R B×N×C C is the number of channels, N is t × width × height of the input image, B is the number of samples per round; t represents the number of frames input to the model, A n Let be a 1×C×C attention parameter matrix, where ⊙ denotes element-wise multiplication of the matrix elements;

[0044] Step 2.3: Convert the data sequence T{T1,T2,…,T…} t The input is fed into the local convolutional attention feature extraction module. First, consider the input sequence T{T1,T2,…,T…} t The process involves performing a Conv1×1 convolution to obtain local feature information Q and K. Then, Q and K are multiplied by a matrix to obtain static information attn. Attn is then subjected to a classification softmax operation and multiplied by a matrix T to obtain local dynamic features. Finally, after performing a Conv1×1 convolution, LayerNorm normalization, ReLU activation, and another Conv1×1 convolution, a local feature vector F with an output dimension of M·N·3 is obtained. local ;

[0045] Step 2.4: Transfer global features F global and local features F local A linear weighted fusion is performed to generate the final behavioral feature F; the feature fusion formula is:

[0046] F=αF global +(1-α)F local

[0047] Where α∈[0,1] represents the feature fusion weight coefficient, which balances the importance of global features and local features.

[0048] Step 2.5: Input the fused feature F into the decoder, and obtain the predicted category label C of the sequence through classification. t and the corresponding category score S;

[0049] Step 3: Train the behavior recognition model: Input the training dataset into the behavior recognition network model for training; evaluate and verify the trained behavior recognition network model using the test set and validation set;

[0050] The training behavior recognition model is configured with an initial learning rate of lr = 1e-3, an optimizer of Adamw, 8 action categories, and a minimum image cropping size of 224.

[0051] Step 4: Apply the trained behavior recognition model to perform predictable behavior recognition. The specific steps are as follows:

[0052] Step 4.1: Acquire video V from the camera real The trained behavior recognition model is input to perform predictable behavior recognition and classification, and the predicted category label C of the sequence is obtained. t and the corresponding category score S;

[0053] Step 4.2: Add the predicted category label C t The frame sequence containing the corresponding category score S is combined with the current timestamp time to form the final output sequence Vpre t{time,C t ,S};

[0054] Step 4.3: Determine the category label C t Does it belong to the predefined category of predictable behavior C? If C t If the behavior falls under the predictable behavior category C, then the resulting sequence Vpre t{time,C} will be... t Store S in the database and output the result; otherwise, return to step 4.1 and continue the identification process.

[0055] Specifically, taking early predictable behaviors in stroke as an example, the workflow of the predictable behavior recognition method based on a neural network model is described as follows:

[0056] Step 1: Data Collection and Processing: Construct the NTU-ST dataset for predictable behaviors in the early stages of stroke. The predefined predictable behavior categories in the dataset are C {gait abnormalities, swaying, falls, headaches, chest pain, neck pain, sneezing, and vomiting}. The NTU-ST dataset is selected from the NTU RGB-D dataset and the SAIL-TUG dataset for the corresponding category C. The dataset is divided into training, testing, and validation sets in a 6.5:1.5:2 ratio.

[0057] Step 2: Building a behavior recognition model (e.g.) Figure 1 This behavior recognition model is an improvement upon the VideoMAE neural network model, including an encoder, a decoder, and a masking strategy. The encoder includes input data preprocessing and improved feature extraction; the improved feature extraction includes a multi-head self-attention feature extraction module. Local convolutional attention feature extraction module

[0058] The improved encoder feature extraction replaces the original spatiotemporal joint self-attention mechanism with a multi-head self-attention feature extraction module. and local convolutional attention feature extraction module The features are extracted together and then obtained as the final behavioral features F through linear weighted fusion.

[0059] Step 2.1: Divide the NTU-ST dataset into individual frame sequences and perform temporal downsampling according to frame t, followed by center cropping and size normalization to obtain the data sequence T{T1,T2,…,T t Then the data sequence T{T1,T2,…,T} is processed. t Input the feature extraction module of the improved encoder.

[0060] Step 2.2: First, process the input data sequence T{T1,T2,…,T…} t The process involves dividing the data into multiple smaller blocks based on spatial and temporal dimensions, resulting in P{P1,P2,…,P}. t}, then P{P1,P2,…,P t Perform a tube masking operation, randomly select 90% for masking, and retain 10% of P{P1,P2,…,P}. t After passing through a linear embedding layer, high-dimensional feature information h{h1,h2,…,h is obtained. t}

[0061] Then, the high-dimensional feature information h{h1,h2,…,h t Input multi-head attention feature extraction module Multi-head attention is used to segment the input high-dimensional feature information data sequence, forming different attention heads. These attention heads are then used to capture behavioral features across the entire temporal and spatial range. From a global perspective, behavioral information is captured, resulting in a global feature vector F of dimension M·N·3 (where M is the width of the input image / number of frames, and N is the height of the input image / number of frames). global The formula is as follows:

[0062]

[0063] Where T represents the token data sequence sample (T∈R) B×N×C C is the number of channels, N is t × width × height of the input image, B is the number of samples per round, t represents the number of frames input to the model, and A n Let be a 1×C×C attention parameter matrix, where ⊙ denotes the element-wise multiplication of the matrix elements.

[0064] The multi-head attention mechanism uses multiple independent attention heads to focus on features in video frames from different angles. This captures both global information and identifies subtle changes, making the model more expressive in spatiotemporal relationship modeling. It enhances the model's ability to capture spatiotemporal features, enabling it to focus on behavioral details in the video from different perspectives.

[0065] Step 2.3: Simultaneously process the data sequence T{T1,T2,…,T…} t The input is fed into the local convolutional attention feature extraction module. First, consider the input sequence T{T1,T2,…,T…} t A Conv1×1 convolution operation is performed to obtain local feature information Q and K. Then, Q and K are multiplied by a matrix to obtain static information attn. Attn is then subjected to a softmax operation and multiplied by matrix T to obtain local dynamic features. Finally, after Conv1×1 convolution, LayerNorm, ReLU, and Conv1×1 convolution operations, a local feature vector F with an output dimension of M·N·3 (M is the width of the input image / number of frames, N is the height of the input image / number of frames) is obtained. local .

[0066] The local convolutional attention feature extraction module shown can efficiently extract local dynamic features in video frames and amplify information in important regions. By capturing intra-frame details through convolution operations, this module helps identify short-term behavioral changes and, combined with an attention mechanism, more accurately focuses on key regions, thereby improving the model's ability to recognize subtle behaviors.

[0067] Step 2.4: Transfer global features F global and local features F localLinear weighted fusion is performed to generate the final behavioral feature F.

[0068] Step 2.5: Input the fused feature F into the decoder, and obtain the sequence prediction label C through classification. t And the corresponding category score S.

[0069] Step 3: Train the behavior recognition model: Input the training dataset into the behavior recognition network model for training; use the test set and validation set to evaluate and verify the trained behavior recognition network model.

[0070] The model was initially set with a learning rate of lr = 1e-3, an optimizer of Adamw, 8 action categories, and a cropped shortest side image size of 224. The behavior recognition model was initialized using the pre-trained model parameters from the pretrain dataset, and then trained on the NTU-ST dataset to obtain the behavior recognition model R. m The pretrain dataset is Kinetics710.

[0071] Step 4: Apply the trained behavior recognition model to perform predictable behavior recognition. The specific steps are as follows:

[0072] Step 4.1: Acquire video V from the camera real The trained behavior recognition model is input to perform predictable behavior recognition and classification, and the predicted category label C of the sequence is obtained. t and the corresponding category score S;

[0073] Step 4.2: Add the predicted category label C t The frame sequence containing the corresponding category score S is combined with the current timestamp time to form the final output sequence Vpre t{time,C t ,S};

[0074] Step 4.3: Determine the category label C t Does it belong to the predefined category of predictable behavior C? If C t If the behavior falls under the predictable behavior category C, then the resulting sequence Vpre t{time,C} will be... t Store S in the database and output the result; otherwise, return to step 4.1 and continue the identification process.

[0075] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications, equivalent changes, and alterations made to the above embodiments without departing from the technical essence of the present invention shall still fall within the scope of the present invention.

Claims

1. A predictable behavior recognition method based on a neural network model, characterized in that: The method includes the following steps: Step 1: Collect a dataset of predictable behaviors, predefine the categories of predictable behaviors, and label the data in the dataset with category labels; divide the dataset into a training set, a test set, and a validation set; Step 2: Behavior Recognition Model Construction: This behavior recognition model is an improvement upon the VideoMAE neural network model, including an encoder, a decoder, and a masking strategy. The encoder includes input data preprocessing and improved feature extraction; the improved feature extraction includes a multi-head self-attention feature extraction module R. global Local convolutional attention feature extraction module R local Its behavior recognition process is as follows: Step 2.1: The encoder's input data preprocessing involves dividing the input data in the dataset into individual frame sequences, performing temporal downsampling according to frame t, and then performing center clipping and size normalization to obtain the data sequence T{T1,T2,…,T…}. t Then, the data sequence T{T1,T2,…,T} is processed. t Improved feature extraction from the input encoder; Step 2.2: The improved feature extraction first involves processing the input data sequence T{T1,T2,…,T…}. t The data is divided into multiple smaller blocks according to spatial and temporal dimensions, resulting in data blocks P{P1,P2,…,P}. m }, then set P{P1,P2,…,P m Perform a masking operation on the tube, randomly selecting 90% of the image for masking, and then masking the remaining 10% of the image (P{P1,P2,…,P...)). m After passing through a linear embedding layer, high-dimensional feature information h{h1,h2,…,h is obtained. m }; Then, the high-dimensional feature information h{h1,h2,…,h m Input multi-head attention feature extraction module R global Multi-head attention is used to segment the input high-dimensional feature information data sequence to form different attention heads, and behavioral features are captured in the entire time and space range through the attention heads; By capturing behavioral information from a global perspective, a global feature vector F with dimensions M·N·3 is obtained. global Where M is the width of the input image / number of frames, and N is the height of the input image / number of frames, as shown in the following formula: ； Where T represents a token data sequence sample, T∈R B×N×C C represents the number of channels, B represents the number of samples per round; t represents the number of frames input to the model, and A n Let be a 1×C×C attention parameter matrix, where ⊙ denotes element-wise multiplication of the matrix elements; Step 2.3: Convert the data sequence T{T1,T2,…,T…} t The input is fed into the local convolutional attention feature extraction module R. local First, for the input sequence T{T1,T2,…,T…} t The process involves performing a Conv1×1 convolution to obtain two local feature information points: query Q and key K. Subsequently, Q and K are multiplied by a matrix to obtain static information attn. Then, attn undergoes a classification softmax operation and is multiplied by the feature matrix extracted from the processed data sequence T to obtain local dynamic features. Finally, after performing a Conv1×1 convolution, LayerNorm normalization, ReLU activation, and another Conv1×1 convolution, a local feature vector F with an output dimension of M·N·3 is obtained. local ; Step 2.4: Transfer global features F global and local features F local Perform linear weighted fusion to generate the final behavioral feature F, as shown in the formula: F=αF global +(1-α)F local Where α∈[0,1] represents the feature fusion weight coefficient, which balances the importance of global features and local features; Step 2.5: Input the fused feature F into the decoder, and obtain the predicted category label of the sequence through classification. and the corresponding category score S; Step 3: Train the behavior recognition model: Input the training dataset into the behavior recognition network model for training; evaluate and verify the trained behavior recognition network model using the test set and validation set; Step 4: Apply the trained behavior recognition model to perform predictable behavior recognition.

2. The predictable behavior recognition method based on a neural network model as described in claim 1, characterized in that: In step 1, the dataset is divided into a training set, a test set, and a validation set in a ratio of 6.5:1.5:

2.

3. The predictable behavior recognition method based on a neural network model as described in claim 1, characterized in that: In step 2.1, the input data is divided into a separate frame sequence, with the number of frames set to 90 frames.

4. The predictable behavior recognition method based on a neural network model as described in claim 1, characterized in that: In step 2.1, the timing downsampling is performed according to frame t, and the sampling rate is set to 8 and 16.

5. The predictable behavior recognition method based on a neural network model as described in claim 1, characterized in that: In step 3, the initial learning rate for training the behavior recognition model is set to lr = 1 * 10. -3 The optimizer used is Adamw, with 8 action categories and a minimum image crop size of 224.

6. The predictable behavior recognition method based on a neural network model as described in claim 1, characterized in that: In step 4, the pre-trained behavior recognition model is used to perform predictable behavior recognition. The specific steps are as follows: Step 4.1: Acquire video V from the camera real The trained behavior recognition model is input to perform predictable behavior recognition and classification, and the predicted category label of the sequence is obtained. and the corresponding category score S; Step 4.2: Add the predicted category labels The frame sequence containing the corresponding category score S is combined with the current timestamp time to form the final output sequence Vpret{time, ,S}; Step 4.3: Determine the category label Does it belong to a predefined category of predictable behavior? If so... If it belongs to the predictable behavior category, then the result sequence Vpret{time, Store S in the database and output the result; otherwise, return to step 4.1 and continue the identification process.

Images