Predictable behavior recognition method based on neural network model

Abstract

The invention discloses a predictable behavior recognition method based on a neural network model. The method comprises the following steps: collecting a predictable behavior data set; a behavior recognition model is constructed, improvement is carried out based on a neural network model Video MAE, an encoder, a decoder and a mask strategy are included, the encoder comprises input data preprocessing and improved feature extraction, and the improved feature extraction comprises feature extraction. An original space-time joint self-attention mechanism is changed into a multi-head self-attention feature extraction module # imgabs0 # # imgabs1 # and a local convolution attention feature extraction module # imgabs2 # to jointly realize feature extraction, and then a final behavior feature F is obtained through linear weighted fusion; training a behavior recognition model; and carrying out predictable behavior recognition by using the trained behavior recognition model. The method has the advantages of being high in recognition accuracy, good in real-time performance and high in robustness.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a predictable behavior recognition method based on a neural network model. Background Art

[0002] Stroke is a major chronic non-communicable disease that endangers national health. It is divided into two types: ischemic stroke and hemorrhagic stroke, of which about 60% to 80% are ischemic stroke. Active and effective intervention in the risk factors of stroke can reduce the incidence of stroke. However, the treatment effect of stroke is strongly correlated with time. If the patient undergoes endovascular recanalization treatment within 6 hours of onset, good clinical efficacy can be achieved. Elderly patients who live alone and lack stroke awareness are more likely to experience pre-hospital delays. They are delayed in treatment because they cannot be discovered in time. Therefore, studying the predictable behavior of potential stroke patients with intelligent perception in the home environment is of great significance to saving lives.

[0003] In the existing technology, deep learning technology is applied to the early identification of diseases. Some studies have proposed an intelligent recognition method for depression based on voice signals in combination with the home environment, and designed a new integrated support vector machine algorithm to provide strong support for the early diagnosis of patients with depression. Some studies have proposed a cloud-based computerized sepsis clinical decision system to identify inflammatory syndromes based on the patient's infectious diseases and comorbidities, and to achieve early warning of sepsis. Some studies have proposed a pediatric viral encephalitis MRI classification and early diagnosis model based on Squeeze-and-Excitation Residual Networks, with a model accuracy of 70.83%, providing strong support for the early diagnosis of pediatric viral encephalitis. In response to the problem of low prediction accuracy of cardiovascular diseases, some studies have proposed a cardiovascular disease prediction model based on a one-dimensional convolutional neural network, analyzing 13 physiological data such as blood sugar to explore the relationship between physiological data and cardiovascular diseases. Its accuracy rate is more than 90%, which is better than the prediction model based on the particle swarm algorithm. Some studies have proposed a transfer learning model combined with deep learning to identify signs of cognitive diseases. Its average accuracy is 67.6%, which reduces the difficulty of screening early signs of cognitive diseases. However, these methods have shortcomings such as low recognition accuracy, poor real-time performance, and poor robustness, and lack methods to gain insight into the early predictable behaviors of the disease by observing and understanding people's daily activities. Summary of the invention

[0004] The purpose of the present invention is to overcome the above shortcomings and propose a predictable behavior recognition method based on a neural network model with high recognition accuracy, good real-time performance and strong robustness.

[0005] The present invention provides a predictable behavior recognition method based on a neural network model, wherein: the method comprises the following steps:

[0006] Step 1: Collect a predictable behavior dataset, predefine a predictable behavior category C, and annotate the predictable behavior dataset data with category labels; divide the dataset into a training set, a test set, and a validation set;

[0007] Step 2: Construction of behavior recognition model: The behavior recognition model is improved based on the neural network model VideoMAE, including an encoder, a decoder, and a mask strategy, wherein the encoder includes input data preprocessing and improved feature extraction; the improved feature extraction includes a multi-head self-attention feature extraction module Local convolutional attention feature extraction module The behavior recognition process is as follows:

[0008] Step 2.1: The encoder input data is preprocessed by dividing the input data into separate frame sequences, and performing time-series downsampling according to t frames, and performing center cropping and size normalization to obtain a data sequence T{T1, T2, …, T t}, and then the data sequence T{T1,T2,…,T t} Improved feature extraction of input encoder;

[0009] Step 2.2: The improved feature extraction first performs feature extraction on the input data sequence T{T1, T2, …, T t} is divided into multiple small blocks according to the spatial and temporal dimensions, and P{P1,P2,…,P t}, and then P{P1,P2,…,P t} performs masking operation on tube, randomly selects a certain proportion for masking, and masks the remaining part of P{P1,P2,…,P t}After the linear embedding layer, we get the high-dimensional feature information h{h1,h2,…,h t};

[0010] Then, the high-dimensional feature information h{h1,h2,…,h t} Input multi-head attention feature extraction module The multi-head attention is used to process the input high-dimensional feature information data sequence in segments to form different attention heads, and the behavior characteristics are captured in the full time and space range through the attention heads; the behavior information is captured from a global perspective to obtain a global feature vector F with a dimension of M·N·3 global , where M is the width / number of frames of the input image, and N is the height / number of frames of the input image. The formula is as follows:

[0011]

[0012] Among them, T represents the token data sequence sample, T∈RB×N×C , C is the number of channels, N is t×the width of the input image×the height of the input image, B is the number of samples per round; t represents the number of frames of the input model, A n is a 1×C×C attention parameter matrix, ⊙ represents the bitwise multiplication of matrix elements;

[0013] Step 2.3: Transform the data sequence T{T1,T2,…,T t} Input to the local convolutional attention feature extraction module First, the input sequence T{T1,T2,…,T t}Convolution Conv1×1 operation is performed to obtain local feature information Q and K; then, Q and K are matrix multiplied to obtain static information attn, and then attn is classified softmax operation and multiplied with T matrix to obtain local dynamic features; finally, after convolution Conv1×1, normalization LayerNorm, activation function ReLU, convolution Conv1×1 operation, the local feature vector F with output dimension M·N·3 is obtained. local ;

[0014] Step 2.4: Transform the global feature F global and local features F local Perform linear weighted fusion to generate the final behavior feature F; the feature fusion formula is:

[0015] F=αF global +(1-α)F local

[0016] Among them, α∈[0,1] represents the feature fusion weight coefficient, balancing the importance of global features and local features;

[0017] Step 2.5: Input the fusion feature F into the decoder and classify it to obtain the sequence prediction category label C t and the corresponding category score S;

[0018] Step 3: training the behavior recognition model: inputting the training data set into the behavior recognition network model for training; using the test set and the validation set to evaluate and verify the trained behavior recognition network model;

[0019] Step 4: Apply the trained behavior recognition model to perform predictable behavior recognition.

[0020] The above-mentioned predictable behavior identification method based on the neural network model, wherein: in step 1, the data set is divided into a training set, a test set and a validation set in a ratio of 6.5:1.5:2.

[0021] The above-mentioned predictable behavior recognition method based on the neural network model, wherein: in step 2.1, the input data is divided into separate frame sequences, and the number of frames is set to 90 frames.

[0022] The above-mentioned predictable behavior recognition method based on the neural network model, wherein: in step 2.1, the time-series downsampling is performed according to t frames, and the sampling rate is set to 8 or 16.

[0023] The predictable behavior recognition method based on the neural network model, wherein: in step 2.2, the random selection of 90% ratio for masking, the remaining 10% of P{P1, P2, ..., P t}After a linear embedding layer.

[0024] The above-mentioned predictable behavior recognition method based on the neural network model, wherein: in step 3, the training behavior recognition model sets the initial learning rate lr=1e-3, the optimizer uses Adamw, the number of action categories is 8, and the shortest side cropping size of the image is 224.

[0025] The foreseeable behavior recognition method based on the neural network model, wherein: in step 4, the trained behavior recognition model is applied to perform foreseeable behavior recognition, and the specific steps are:

[0026] Step 4.1: Get video V from the camera real , input the trained behavior recognition model to perform predictable behavior recognition classification, and obtain the sequence prediction category label C t and the corresponding category score S;

[0027] Step 4.2: Take the predicted class label C t The frame sequence of the corresponding category score S is combined with the current timestamp time to form the final output result sequence Vpre t{time,C t ,S};

[0028] Step 4.3: Determine the category label C t Does it belong to the predefined foreseeable behavior category C? If C t Belongs to the predictable behavior category C, then the result sequence Vpre t{time,C t ,S} is stored in the database and the result is output; otherwise, it returns to step 4.1 and continues the recognition.

[0029] Compared with the prior art, the present invention has obvious beneficial effects. It can be seen from the above scheme that the present invention solves the problem of abnormal behavior recognition in the early stage of the disease, fully considers the needs of early warning, and makes innovative optimization in terms of recognition accuracy, real-time performance and multi-angle modeling of abnormal detection. A dual-path structure combining a global multi-head attention module and a local convolutional attention module is designed. First, the multi-head attention module extracts behavioral features from a full spatiotemporal perspective, divides the input data into multiple attention heads, captures important features in the video frame from different angles, and obtains global features. Global feature extraction helps the model grasp the spatiotemporal dynamic relationship of behavior at a macro level, and is particularly suitable for identifying long-term changes in behavior. At the same time, the local convolutional attention module processes video data frame by frame, focuses on short-term local dynamic feature extraction, and can identify subtle changes in the frame, such as abnormal gait, slight shaking and other behaviors. This structure enables the model to take into account both macro and micro details, effectively improve the model's sensitivity to behavior in early diseases, and accurately capture the subtle abnormal manifestations of patients with diseases. In order to give full play to the advantages of global and local features, the present invention uses a linear weighted fusion strategy to weightedly combine global features and local features to generate the final behavioral features. By assigning weights to global features, the model's attention to macro-behavior trends and local detail changes can be better balanced, ensuring that the model has higher adaptability when dealing with different types of abnormal behaviors. This fusion mechanism improves the generalization ability of the model, and can accurately identify behaviors such as falls, headaches, chest pain, etc., and is particularly suitable for early detection of short-term abnormal behaviors during the onset of the disease, so that the model of the present invention exhibits stronger robustness in complex scenarios. In addition, the present invention designs a set of real-time monitoring and high-confidence feedback processes. The method processes the video at a specified frame rate and continuously identifies abnormal behaviors during the process of acquiring the video from the camera. This process is based on real-time performance, and the identified abnormal behavior results are screened through a confidence matching strategy to ensure that the system records and warns of abnormal behaviors with high confidence. In this way, when abnormal behaviors such as abnormal gait and falls are identified, the system will immediately feedback and store them in the database for real-time monitoring and subsequent intervention, thereby providing a stable early warning and behavior feedback mechanism for patients with the disease, which helps family members and medical staff to know the abnormal state of the patient in a timely manner.

[0030] In summary, the present invention not only performs well in accurately identifying the early abnormal behaviors of disease patients, but also can provide real-time and efficient feedback of abnormal states. Through the collaborative extraction, weighted fusion and confidence feedback mechanism of global and local features, the robustness in complex and diverse behavioral scenarios is ensured, which can effectively reduce the missed detection rate and enhance the practical application value of early warning of diseases. In short, the present invention has the characteristics of high recognition accuracy, good real-time performance and strong robustness.

[0031] The beneficial effects of the present invention are further illustrated below through specific implementation modes. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of the recognition model structure of the present invention. DETAILED DESCRIPTION

[0033] The following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation, features and efficacy of a predictable behavior recognition method based on a neural network model proposed in the present invention.

[0034] See also Figure 1 The present invention provides a predictable behavior recognition method based on a neural network model, which comprises the following steps:

[0035] Step 1: Collect a predictable behavior dataset, predefine a predictable behavior category C, and annotate the predictable behavior dataset data with category labels; divide the dataset into a training set, a test set, and a validation set; the dataset is divided into a training set, a test set, and a validation set in a ratio of 6.5:1.5:2.

[0036] Step 2: Construction of behavior recognition model: The behavior recognition model is improved based on the neural network model VideoMAE, including an encoder, a decoder, and a mask strategy, wherein the encoder includes input data preprocessing and improved feature extraction; the improved feature extraction includes a multi-head self-attention feature extraction module Local convolutional attention feature extraction module The behavior recognition process is as follows:

[0037] Step 2.1: The encoder input data is preprocessed by dividing the input data into separate frame sequences, and performing time-series downsampling according to t frames, and performing center cropping and size normalization to obtain a data sequence T{T1, T2, …, T t}, and then the data sequence T{T1,T2,…,T t} Improved feature extraction of input encoder;

[0038] The input data is divided into separate frame sequences, and the number of frames is set to 90. The time sequence downsampling is performed according to t frames, and the sampling rate is set to 8 or 16.

[0039] Step 2.2: The improved feature extraction first performs feature extraction on the input data sequence T{T1, T2, …, T t} is divided into multiple small blocks according to the spatial and temporal dimensions, and P{P1,P2,…,P t}, and then P{P1,P2,…,P t} performs masking operation on tube, randomly selects a certain proportion for masking, and masks the remaining part of P{P1,P2,…,P t}After the linear embedding layer, we get the high-dimensional feature information h{h1,h2,…,h t};

[0040] The random selection of 90% ratio is used for masking, and the remaining 10% of P{P1,P2,…,P t}After a linear embedding layer.

[0041] Then, the high-dimensional feature information h{h1,h2,…,h t} Input multi-head attention feature extraction module The multi-head attention is used to process the input high-dimensional feature information data sequence in segments to form different attention heads, and the behavior characteristics are captured in the full time and space range through the attention heads; the behavior information is captured from a global perspective to obtain a global feature vector F with a dimension of M·N·3 global , where M is the width / number of frames of the input image, and N is the height / number of frames of the input image. The formula is as follows:

[0042]

[0043] Among them, T represents the token data sequence sample, T∈R B×N×C , C is the number of channels, N is t×the width of the input image×the height of the input image, B is the number of samples per round; t represents the number of frames of the input model, A n is a 1×C×C attention parameter matrix, ⊙ represents the bitwise multiplication of matrix elements;

[0044] Step 2.3: Transform the data sequence T{T1,T2,…,T t} Input to the local convolutional attention feature extraction module First, the input sequence T{T1,T2,…,T t}Convolution Conv1×1 operation is performed to obtain local feature information Q and K; then, Q and K are matrix multiplied to obtain static information attn, and then attn is classified softmax operation and multiplied with T matrix to obtain local dynamic features; finally, after convolution Conv1×1, normalization LayerNorm, activation function ReLU, convolution Conv1×1 operation, the local feature vector F with output dimension M·N·3 is obtained. local ;

[0045] Step 2.4: Transform the global feature F global and local features F local Perform linear weighted fusion to generate the final behavior feature F; the feature fusion formula is:

[0046] F=αF global +(1-α)F local

[0047] Among them, α∈[0,1] represents the feature fusion weight coefficient, which balances the importance of global features and local features.

[0048] Step 2.5: Input the fusion feature F into the decoder and classify it to obtain the sequence prediction category label C t and the corresponding category score S;

[0049] Step 3: training the behavior recognition model: inputting the training data set into the behavior recognition network model for training; using the test set and the validation set to evaluate and verify the trained behavior recognition network model;

[0050] The training behavior recognition model sets the initial learning rate lr=1e-3, uses Adamw as the optimizer, the number of action categories is 8, and the image shortest side cropping size is 224.

[0051] Step 4: Apply the trained behavior recognition model to identify predictable behaviors. The specific steps are as follows:

[0052] Step 4.1: Get video V from the camera real , input the trained behavior recognition model to perform predictable behavior recognition classification, and obtain the sequence prediction category label C t and the corresponding category score S;

[0053] Step 4.2: Take the predicted class label C t The frame sequence of the corresponding category score S is combined with the current timestamp time to form the final output result sequence Vpre t{time,C t ,S};

[0054] Step 4.3: Determine the category label C t Does it belong to the predefined foreseeable behavior category C? If C t Belongs to the predictable behavior category C, then the result sequence Vpre t{time,C t ,S} is stored in the database and the result is output; otherwise, it returns to step 4.1 and continues the recognition.

[0055] Specifically, taking the early predictable behavior of stroke as an example, the predictable behavior identification method based on the neural network model is described as follows:

[0056] Step 1: Data collection and processing: Construct the NTU-ST dataset of predictable behaviors in early stroke. The predefined dataset predictable behaviors are C {abnormal gait, staggering, falling, headache, chest pain, neck pain, sneezing, and vomiting}. The NTU-ST dataset is selected from the NTU RGB-D dataset and the SAIL-TUG dataset. The dataset is divided into training set, test set, and validation set in a ratio of 6.5:1.5:2.

[0057] Step 2: Behavior recognition model construction (such as Figure 1 ): The behavior recognition model is improved based on the neural network model VideoMAE, including an encoder, a decoder, and a mask strategy, wherein the encoder includes input data preprocessing and improved feature extraction; the improved feature extraction includes a multi-head self-attention feature extraction module Local convolutional attention feature extraction module

[0058] The improved encoder feature extraction changes the original spatiotemporal joint self-attention mechanism to a multi-head self-attention feature extraction module And local convolutional attention feature extraction module Feature extraction is achieved together, and then the final behavior feature F is obtained through linear weighted fusion.

[0059] Step 2.1: Split the NTU-ST dataset into separate frame sequences and perform temporal downsampling according to t frames, and perform center cropping and size normalization to obtain the data sequence T{T1,T2,…,T t Then the data sequence T{T1,T2,…,T t} Input the feature extraction module of the improved encoder.

[0060] Step 2.2: First, the input data sequence T{T1,T2,…,T t} is divided into multiple small blocks according to the spatial and temporal dimensions, and P{P1,P2,…,P t}, then P{P1,P2,…,P t} performs tube masking, randomly selects 90% for masking, and retains 10% of P{P1,P2,…,P t}After the linear embedding layer, we get high-dimensional feature information h{h1,h2,…,h t}.

[0061] Then the high-dimensional feature information h{h1,h2,…,h t} Input multi-head attention feature extraction module The multi-head attention is used to segment the input high-dimensional feature information data sequence to form different attention heads, and the behavior characteristics are captured in the full time and space range through the attention heads. The behavior information is captured from a global perspective, and the global feature vector F with dimensions M·N·3 (M is the width / number of frames of the input image, N is the height / number of frames of the input image) is obtained. global The formula is as follows:

[0062]

[0063] Among them, T represents the token data sequence sample (T∈R B×N×C , C is the number of channels, N is t×the width of the input image×the height of the input image, B is the number of samples per round), t represents the number of frames of the input model, A n is a 1×C×C attention parameter matrix, and ⊙ represents the bitwise multiplication of matrix elements.

[0064] The multi-head attention mechanism uses multiple independent attention heads to focus on features in video frames from different angles, which can capture global information and identify changes in details, making the model more expressive in modeling spatiotemporal relationships. It improves the model's ability to capture spatiotemporal features, allowing it to focus on behavioral details in the video from different perspectives.

[0065] Step 2.3: Simultaneously convert the data sequence T{T1, T2, …, T t} Input to the local convolutional attention feature extraction module First, the input sequence T{T1,T2,…,T t}Conv1×1 convolution operation is performed to obtain local feature information Q and K. Subsequently, Q and K are matrix multiplied to obtain static information attn. Then, attn is softmaxed and multiplied with the T matrix to obtain local dynamic features. After that, after Conv1×1 convolution, LayerNorm, ReLU, and Conv1×1 convolution operations, the local feature vector F with an output dimension of M·N·3 (M is the width / number of frames of the input image, and N is the height / number of frames of the input image) is obtained. local .

[0066] The local convolutional attention feature extraction module shown can efficiently extract local dynamic features in video frames and amplify information in important areas. The local convolutional attention feature extraction module captures details within the frame through convolution operations, which helps to identify short-term behavioral changes and combines the attention mechanism to focus on key areas more accurately, thereby improving the model's ability to recognize subtle behaviors.

[0067] Step 2.4: Transform the global feature F global and local features F localPerform linear weighted fusion to generate the final behavior feature F.

[0068] Step 2.5: Input the fusion feature F into the decoder and obtain the sequence prediction label C after classification t and the corresponding category score S.

[0069] Step 3: training the behavior recognition model: inputting the training data set into the behavior recognition network model for training; using the test set and the validation set to evaluate and verify the trained behavior recognition network model.

[0070] The model sets the initial learning rate lr = 1e-3, uses Adamw as the optimizer, the number of action categories = 8, and the image shortest side cropping size = 224. The behavior recognition model is initialized using the pretrained model parameters of the pretrain dataset, and then the behavior recognition model is trained on the NTU-ST dataset to obtain the behavior recognition model R m The pretrain dataset is Kinetics710.

[0071] Step 4: Apply the trained behavior recognition model to identify predictable behaviors. The specific steps are as follows:

[0072] Step 4.1: Get video V from the camera real , input the trained behavior recognition model to perform predictable behavior recognition classification, and obtain the sequence prediction category label C t and the corresponding category score S;

[0073] Step 4.2: Take the predicted class label C t The frame sequence of the corresponding category score S is combined with the current timestamp time to form the final output result sequence Vpre t{time,C t ,S};

[0074] Step 4.3: Determine the category label C t Does it belong to the predefined foreseeable behavior category C? If C t Belongs to the predictable behavior category C, then the result sequence Vpre t{time,C t ,S} is stored in the database and the result is output; otherwise, it returns to step 4.1 and continues the recognition.

[0075] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still falls within the scope of the technical solution of the present invention.

Claims

1. A predictable behavior recognition method based on a neural network model, characterized in that: The method comprises the following steps: Step 1: Collect a predictable behavior dataset, predefine a predictable behavior category C, and annotate the predictable behavior dataset data with category labels; divide the dataset into a training set, a test set, and a validation set; Step 2: Construction of behavior recognition model: The behavior recognition model is improved based on the neural network model VideoMAE, including an encoder, a decoder, and a mask strategy, wherein the encoder includes input data preprocessing and improved feature extraction; the improved feature extraction includes a multi-head self-attention feature extraction module Local convolutional attention feature extraction module The behavior recognition process is as follows: Step 2.1: The encoder input data is preprocessed by dividing the input data into separate frame sequences, and performing time-series downsampling according to t frames, and performing center cropping and size normalization to obtain a data sequence T{T1, T2, …, T t }, and then the data sequence T{T1,T2,…,T t } Improved feature extraction of input encoder; Step 2.2: The improved feature extraction first performs feature extraction on the input data sequence T{T1, T2, …, T t } is divided into multiple small blocks according to the spatial and temporal dimensions, and P{P1,P2,…,P t }, and then P{P1,P2,…,P t } performs masking operation on tube, randomly selects a certain proportion for masking, and masks the remaining part of P{P1,P2,…,P t }After the linear embedding layer, we get the high-dimensional feature information h{h1,h2,…,h t }; Then, the high-dimensional feature information h{h1,h2,…,h t } Input multi-head attention feature extraction module Use multi-head attention to segment the input high-dimensional feature information data sequence to form different attention heads, and use the attention heads to capture behavioral characteristics in the full time and space range; Capture behavioral information from a global perspective and obtain a global feature vector F with dimensions M·N·3 global , where M is the width / number of frames of the input image, and N is the height / number of frames of the input image. The formula is as follows: Among them, T represents the token data sequence sample, T∈R B×N×C , C is the number of channels, N is t×the width of the input image×the height of the input image, B is the number of samples per round; t represents the number of frames of the input model, A n is a 1×C×C attention parameter matrix, ⊙ represents the bitwise multiplication of matrix elements; Step 2.3: Transform the data sequence T{T1,T2,…,T t } Input to the local convolutional attention feature extraction module First, the input sequence T{T1,T2,…,T t }Convolution Conv1×1 operation is performed to obtain local feature information Q and K; then, Q and K are matrix multiplied to obtain static information attn, and then attn is classified softmax operation and multiplied with T matrix to obtain local dynamic features; finally, after convolution Conv1×1, normalization LayerNorm, activation function ReLU, convolution Conv1×1 operation, the local feature vector F with output dimension M·N·3 is obtained. local ; Step 2.4: Transform the global feature F global and local features F local Perform linear weighted fusion to generate the final behavior feature F, the formula is: F=αF global +(1-α)F local Among them, α∈[0,1] represents the feature fusion weight coefficient, balancing the importance of global features and local features; Step 2.5: Input the fusion feature F into the decoder and classify it to obtain the sequence prediction category label C t and the corresponding category score S; Step 3: training the behavior recognition model: inputting the training data set into the behavior recognition network model for training; using the test set and the validation set to evaluate and verify the trained behavior recognition network model; Step 4: Apply the trained behavior recognition model to perform predictable behavior recognition.

2. A predictable behavior recognition method based on a neural network model as claimed in claim 1, characterized in that: In step 1, the data set is divided into a training set, a test set, and a validation set in a ratio of 6.5:1.5:

2.

3. The predictable behavior identification method based on a neural network model as claimed in claim 1, characterized in that: In step 2.1, the input data is divided into separate frame sequences, and the number of frames is set to 90 frames.

4. The predictable behavior identification method based on a neural network model as claimed in claim 1, characterized in that: In step 2.1, the time sequence downsampling is performed according to t frames, and the sampling rate is set to 8 or 16.

5. The predictable behavior identification method based on a neural network model as claimed in claim 1, characterized in that: In step 2.2, the random selection of 90% ratio is performed for masking, and the remaining 10% of P{P1,P2,…,P t }After a linear embedding layer.

6. The predictable behavior identification method based on a neural network model as claimed in claim 1, characterized in that: In step 3, the training behavior recognition model sets the initial learning rate lr=1e-3, the optimizer uses Adamw, the number of action categories is 8, and the shortest side cropping size of the image is 224.

7. The predictable behavior identification method based on a neural network model as claimed in claim 1, characterized in that: In step 4, the trained behavior recognition model is applied to perform predictable behavior recognition, and the specific steps are as follows: Step 4.1: Get video V from the camera real , input the trained behavior recognition model to perform predictable behavior recognition classification, and obtain the sequence prediction category label C t and the corresponding category score S; Step 4.2: Pass the predicted class label C t The frame sequence of the corresponding category score S is combined with the current timestamp time to form the final output result sequence Vpre t{time,C t ,S}; Step 4.3: Determine the category label C t Does it belong to the predefined foreseeable behavior category C? If C t Belongs to the predictable behavior category C, then the result sequence Vpre t{time,C t ,S} is stored in the database and the result is output; otherwise, it returns to step 4.1 and continues the recognition.

Images