A method and system for identifying abnormal behavior of crew members

Abstract

This invention relates to a method and system for identifying abnormal crew behavior, belonging to the fields of computer vision and artificial intelligence. The method employs a large-scale action recognition dataset to pre-train a backbone model, obtaining a pre-trained backbone model. A lightweight adapter module is embedded into the pre-trained backbone model to construct a domain adaptation model. This domain adaptation model is then trained based on the acquired abnormal crew behavior dataset, obtaining a trained domain adaptation model. Channel-level pruning is used to compress the trained domain adaptation model, obtaining a pruned domain adaptation model. Based on this pruned model, a crew abnormal behavior identification model is obtained. This model is then used to identify acquired real-time ship monitoring video streams to determine abnormal crew behavior, improving identification accuracy and ensuring safe ship operations.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, and in particular to a method and system for identifying abnormal behavior of crew members. Background Technology

[0002] In modern ship navigation management, the compliance of crew members' operational behavior is crucial to maritime safety. Behaviors such as using mobile phones, sleeping on duty, smoking, or leaving their posts without authorization during watch can easily lead to operational errors, fires, explosions, or delayed emergency response, ultimately resulting in serious maritime accidents. To achieve 24 / 7, contactless risk control, an increasing number of studies are attempting to introduce computer vision technology to automate crew behavior monitoring.

[0003] Current mainstream behavior recognition methods based on Transformer have shown good performance in spatiotemporal feature modeling. However, existing methods are mostly geared towards behavior recognition tasks with large data volumes and general categories, and are difficult to directly transfer to fine-grained, small-sample, and complex scenarios such as ship operations. On the one hand, the abnormal behavior samples of crew members are sparsely distributed and the duration of the behavior is uneven, which makes the recognition model prone to overfitting and has poor generalization ability. On the other hand, mainstream structures such as Transformer have large parameters and high computational costs, making them unsuitable for deployment on edge devices such as ship local terminals. In addition, shipping monitoring videos have obvious spatiotemporal continuity and semantic ambiguity of behavior. Traditional models are insufficient in modeling key behavior patterns, especially when dealing with long-term dependent behaviors such as "absent from duty" or "sleeping on duty", the recognition accuracy is low. Summary of the Invention

[0004] In view of this, it is necessary to provide a method and system for identifying abnormal behavior of crew members, so as to solve the technical problems of low recognition accuracy and limited deployment of models at the edge.

[0005] To address the aforementioned problems, in a first aspect, the present invention provides a method for identifying abnormal behavior of crew members, comprising:

[0006] The backbone model is pre-trained using a large-scale action recognition dataset to obtain a pre-trained backbone model, wherein the backbone model includes lightweight local blocks and lightweight global blocks.

[0007] The lightweight adapter module is embedded into the pre-trained backbone model to construct a domain adaptation model. The domain adaptation model is trained based on the acquired abnormal crew behavior dataset to obtain a trained domain adaptation model.

[0008] Channel-level pruning is used to compress the structure of the trained domain adaptation model to obtain a pruned domain adaptation model, and a crew abnormal behavior recognition model is obtained based on the pruned domain adaptation model.

[0009] The abnormal behavior of the crew members is identified by using the abnormal crew behavior identification model to identify the real-time monitoring video stream of the ship.

[0010] In one possible implementation, the backbone model's network structure is LA-UniFormerV2. LA-UniFormerV2 includes a 12-layer functional unit module composed of multiple lightweight local blocks and multiple lightweight global blocks stacked layer by layer, a dynamic fusion mechanism, and a classification module. The 12-layer functional unit module includes lightweight local blocks in the first 8 layers and a combination of lightweight local blocks and lightweight global blocks in the last 4 layers. Pre-training the constructed backbone model using a large-scale action recognition dataset includes:

[0011] The large-scale action recognition dataset is input into LA-UniFormerV2, and the large-scale action recognition dataset is preprocessed to obtain a spatiotemporal encoded sequence;

[0012] Short-time local features are obtained by extracting features from the spatiotemporal encoded sequence using lightweight local blocks.

[0013] Global video features are obtained by extracting features from the short-term local features using lightweight global blocks;

[0014] The local features and global video features are fused through a dynamic fusion mechanism to obtain video spatiotemporal semantic features;

[0015] The spatiotemporal semantic features of the video are classified and predicted using a classification module to obtain prediction results.

[0016] In one possible implementation, the lightweight local block includes a T-SCA submodule, an S-FCA submodule, and an FFN submodule; the step of extracting features from the spatiotemporal encoded sequence using the lightweight local block to obtain short-time local features includes:

[0017] The spatiotemporal encoded sequence is input into a lightweight local block. The spatiotemporal encoded sequence is divided by the T-SCA submodule to obtain attention sub-features and residual sub-features. After performing local temporal single-head attention calculation on the attention sub-features, feature fusion is performed with the residual sub-features to obtain the output features of the T-SCA submodule.

[0018] The spatiotemporal encoded sequence is residually concatenated with the output features of the T-SCA submodule to obtain the input features of the S-FCA submodule. The S-FCA submodule calculates the input features of the S-FCA submodule through a multi-head self-attention mechanism to obtain global spatial dimension features.

[0019] The global spatial dimension features are residually connected with the output features of the T-SCA submodule to obtain the input features of the FFN submodule. The input features of the FFN submodule are then enhanced through the fully connected layer of the FFN submodule. Finally, the enhanced features are residually connected with the input features of the FFN submodule to obtain short-time local features.

[0020] In one possible implementation, the lightweight global block includes a DPE submodule, an ST-SCA submodule, and an FFN submodule, wherein the DPE submodule includes a 3D convolutional layer; the step of extracting features from the local features using the lightweight global block to obtain global video features includes:

[0021] The short-time local features are input into the DPE submodule. The 3D convolutional layer of the DPE submodule performs position encoding on the short-time local features. The short-time local features are then residually connected with the position-encoded features to obtain the spatiotemporal position features.

[0022] Learnable vectors are obtained, and the spatiotemporal location features and learnable vectors are divided by the ST-SCA submodule to obtain attention information sub-features, attention query sub-features, residual information sub-features and residual query sub-features. After global spatiotemporal single-head attention calculation is performed on the attention information sub-features and attention query sub-features, feature fusion is performed with the residual information sub-features to obtain the output features of the ST-SCA submodule.

[0023] The output features of the ST-SCA submodule are enhanced by the fully connected layer of the FFN submodule, and the output features of the ST-SCA submodule are then residually connected with the enhanced features to obtain global video features.

[0024] In one possible implementation, embedding the lightweight adapter module into the pre-trained backbone model to construct a domain-adaptive model includes:

[0025] The lightweight adapter module is embedded before the T-SCA submodule in the lightweight local block and between the DPE submodule and the ST-SCA submodule in the lightweight global block to construct a domain adaptation model.

[0026] In one possible implementation, the network structure of the lightweight adapter module is an FST-Adapter, which includes a downsampling layer, a DPE layer, an offset network layer, a sparse self-attention layer, an activation function layer, and an upsampling layer; the training of the domain adaptation model based on the acquired abnormal crew behavior dataset includes:

[0027] Input features are obtained based on a dataset of abnormal crew behavior. These input features are then fed into a lightweight adapter module, where a downsampling layer performs downsampling and normalization operations on the input features.

[0028] The DPE layer performs positional encoding on the normalized features, fuses the positionally encoded features with the normalized features, and projects the fused encoded features to obtain the query features.

[0029] The input features are uniformly sampled to obtain initial sampling points. The query features are learned through an offset network layer to obtain the offset of the initial sampling points. The initial sampling points are moved based on the offset of the initial sampling points to obtain a set of moved sampling points. Feature interpolation is performed on the set of moved sampling points to obtain the features of the moved sampling points. The offset network layer includes a depthwise 3D convolutional layer, a GELU layer, and a 3D convolutional layer.

[0030] Project the features of the moved sampling points to obtain multiple key-value features. Then, perform sparse self-attention calculation on the multiple key-value features and the query features through a sparse self-attention layer to obtain spatiotemporal adaptation features.

[0031] The spatiotemporal adaptation features are nonlinearly transformed by an activation function layer, and then upsampled by an upsampling layer to obtain upsampled features. The upsampled features are then fused with the input features to obtain output features.

[0032] In one possible implementation, the step of performing structural compression on the trained domain adaptation model using channel-level pruning to obtain a pruned domain adaptation model includes:

[0033] Determine the channels of the 3D convolutional layer in the trained domain adaptation model, and calculate the importance score and redundancy index of each channel;

[0034] The importance score and redundancy index are combined to obtain the channel pruning score index;

[0035] The channels are sorted and filtered based on the channel pruning score index to obtain the pruned domain adaptation model.

[0036] In one possible implementation, the channel pruning scoring metric is:

[0037] ,

[0038] in, The scoring index for channel pruning. Score the importance of each channel. As a redundancy index, For weight hyperparameters.

[0039] In one possible implementation, the step of identifying abnormal crew behavior based on the acquired real-time ship monitoring video stream using the crew abnormal behavior identification model includes:

[0040] The real-time monitoring video stream of the ship is acquired, and the abnormal behavior of the crew is identified based on the abnormal behavior identification model of the crew to determine the abnormal behavior of the crew. The abnormal behavior of the crew includes smoking, playing with mobile phones, leaving the post, sleeping on the post, walking in dangerous areas, and redundant conversation.

[0041] When the duration of the abnormal behavior of the crew member exceeds a preset threshold, a linkage response mechanism is triggered. When the duration of the abnormal behavior of the crew member is less than or equal to the preset threshold, the abnormal behavior recognition model of the crew member is used to identify the real-time monitoring video stream of the ship.

[0042] Secondly, the present invention also provides a system for identifying abnormal behavior of crew members, comprising:

[0043] The backbone model acquisition module is used to pre-train the constructed backbone model using a large-scale action recognition dataset to obtain a pre-trained backbone model, wherein the backbone model includes lightweight local blocks and lightweight global blocks.

[0044] The domain adaptation model building module is used to embed the lightweight adapter module into the pre-trained backbone model to build a domain adaptation model. The domain adaptation model is trained based on the acquired abnormal behavior dataset of crew members to obtain a trained domain adaptation model.

[0045] The abnormal behavior recognition model acquisition module uses channel-level pruning to perform structural compression on the trained domain adaptation model to obtain a pruned domain adaptation model, and obtains a crew abnormal behavior recognition model based on the pruned domain adaptation model.

[0046] The abnormal behavior identification module is used to identify the acquired real-time monitoring video stream of the ship based on the abnormal behavior identification model of the crew members, so as to determine the abnormal behavior of the crew members.

[0047] The beneficial effects of this invention are as follows: A lightweight adapter module is embedded into a pre-trained backbone model to construct a domain adaptation model. The domain adaptation model is trained based on the acquired abnormal crew behavior dataset to obtain a well-trained model. Short-term behavioral details are extracted through lightweight local blocks of the backbone model, and global video features are extracted through lightweight global blocks of the backbone model. This effectively addresses the high-risk state of abnormal crew behavior, significantly improving recognition accuracy and temporal alignment capability. Embedding the lightweight adapter module into the backbone model enables rapid modeling of specific behavioral distributions in ship scenarios, effectively reducing the risk of overfitting and improving generalization stability and training efficiency in small sample environments. Channel-level pruning is used to structurally compress the trained domain adaptation model. The process involves obtaining a pruned domain adaptation model, and then using this pruned model to develop a crew abnormal behavior recognition model. Channel-level pruning effectively compresses the model parameter size, reducing computational overhead and significantly improving model deployment efficiency. The model identifies abnormal crew behavior by analyzing real-time ship monitoring video streams. A domain adaptation model is constructed using a backbone model and a lightweight adapter module. Channel-level pruning further refines this model, resulting in a crew abnormal behavior recognition model. This approach ensures recognition accuracy while achieving dual optimization of structural compression and efficient inference, enabling real-time intelligent monitoring and abnormal alarm functions for crew operations, thus guaranteeing safe ship operations. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 A flowchart illustrating an embodiment of the abnormal behavior identification method for crew members provided by the present invention;

[0050] Figure 2 A schematic diagram of the backbone model of the abnormal crew behavior recognition method provided by the present invention;

[0051] Figure 3 A schematic diagram of the lightweight adapter module of the abnormal crew behavior recognition method provided by the present invention;

[0052] Figure 4 A schematic diagram of the structure of a lightweight adapter module embedded in a lightweight local block for the crew abnormal behavior recognition method provided by the present invention;

[0053] Figure 5A schematic diagram of the structure of a lightweight adapter module embedded in a lightweight global block for the crew abnormal behavior recognition method provided by the present invention;

[0054] Figure 6 A flowchart illustrating an embodiment of the domain adaptation model training for the abnormal crew behavior recognition method provided by the present invention;

[0055] Figure 7 This is a schematic diagram of an embodiment of the crew abnormal behavior recognition system provided by the present invention. Detailed Implementation

[0056] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0057] In this document, the term "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0058] This invention discloses a method and system for identifying abnormal behavior of crew members. One specific embodiment of this invention discloses a method for identifying abnormal behavior of crew members, such as... Figure 1 As shown, the methods for identifying abnormal crew behavior include:

[0059] S101. The constructed backbone model is pre-trained using a large-scale action recognition dataset to obtain a pre-trained backbone model, wherein the backbone model includes lightweight local blocks and lightweight global blocks.

[0060] It should be noted that the backbone model is LA-UniFormerV2, which includes Lightweight Local Blocks (LLBs) and Lightweight Global Blocks (LGBs). Lightweight Local Blocks include the T-SCA, S-FCA, and FFN submodules, while Lightweight Global Blocks include the DPE, ST-SCA, and FFN submodules.

[0061] S102. Embed the lightweight adapter module into the pre-trained backbone model to build a domain adaptation model. Train the domain adaptation model based on the acquired abnormal crew behavior dataset to obtain the trained domain adaptation model.

[0062] It should be noted that the lightweight adapter module includes a downsampling layer, a DPE layer, an offset network layer, a sparse self-attention layer, an activation function layer, and an upsampling layer. By embedding the lightweight adapter module on the backbone model, it is possible to quickly model the distribution of specific behaviors in the ship scene, effectively reduce the risk of overfitting, and improve the generalization stability and training efficiency in small sample environments.

[0063] S103. Channel-level pruning is used to compress the structure of the trained domain adaptation model to obtain the pruned domain adaptation model, and the abnormal behavior recognition model of the crew is obtained based on the pruned domain adaptation model.

[0064] It should be noted that channel-level pruning is a channel-level CI-R pruning strategy, which effectively compresses the model parameter size and significantly reduces computational overhead while ensuring modeling capabilities.

[0065] S104. Identify abnormal crew behavior by using a crew abnormal behavior identification model to identify the acquired real-time monitoring video stream of the ship.

[0066] It should be noted that a domain adaptation model is constructed by using a backbone model and a lightweight adapter module. By pruning the domain adaptation model through channel-level pruning, a crew abnormal behavior recognition model is constructed. While ensuring recognition accuracy, the model achieves dual optimization of structural compression and efficient inference, enabling real-time intelligent monitoring and abnormal alarm functions for crew work behavior, thus ensuring safe ship operations.

[0067] In some embodiments, in step S101, the constructed backbone model is pre-trained using a large-scale action recognition dataset to obtain a pre-trained backbone model. The backbone model includes lightweight local blocks and lightweight global blocks. A Lite Attention UniFormerV2 (LA-UniFormerV2) model is constructed as the backbone model. The LA-UniFormerV2 model is pre-trained using the large-scale action recognition dataset Kinetics-400. The LA-UniFormerV2 model includes lightweight local blocks, lightweight global blocks, a dynamic fusion mechanism, and a classification module. The main body of LA-UniFormerV2 is composed of multiple Lite Local Blocks (LLBs) and Lite Global Blocks (LGBs) stacked layer by layer, forming a 12-layer feature modeling structure. In the first 8 layers, each LLB is considered a functional unit layer; in the last 4 layers, each LLB-LGB combination is considered a functional unit layer. For a schematic diagram of the backbone model structure, please refer to [link to schematic diagram]. Figure 2The local branch, composed of LLB, focuses on short-term dependency modeling between neighboring video frames, while the global branch, composed of LGB, realizes long-term dependency modeling between global video frames. After the feature extraction stage is completed, the model will extract local feature representations and global video representations from the last layer's LLB and LGB respectively, and integrate the two through a dynamic fusion mechanism to generate the final video spatiotemporal semantic expression.

[0068] The backbone model is pre-trained using a large-scale action recognition dataset. This dataset is then input into LA-UniFormerV2 for preprocessing to obtain spatiotemporal coding features. During preprocessing, LA-UniFormerV2 uses frame-level video clips as input. These frame-level video clips are as follows:

[0069] ,

[0070] in, In terms of time dimension, For height dimension, For the width dimension, For channel dimensions;

[0071] Use size 3D convolution kernels, and with The step size is used to perform spatial downsampling and temporal modeling operations on the input frame-level video segments, thereby extracting low-level local spatiotemporal features. A global semantic representation is added before the feature sequence obtained by downsampling. The feature map is marked and a learnable positional encoding is introduced to preserve spatiotemporal structure information. The feature map is then expanded into a spatiotemporal encoded sequence through dimensionality rearrangement. The spatiotemporal encoded sequence is as follows:

[0072] ,

[0073] ,

[0074] in, It is a spatiotemporal encoded sequence. The number of sequences after flattening out the spatiotemporal dimension;

[0075] Feature extraction of spatiotemporal encoded sequences is performed using lightweight local blocks to obtain short-term local features. The lightweight global blocks include the T-SCA (Temporal Sub-Channel Attention) sub-module, the S-FCA (Spatial Full-Channel Attention) sub-module, and the FFN (Feed-Forward) sub-module. The Network submodule, with each submodule concatenated sequentially via residual connections, takes the spatiotemporal encoded sequence as input to a lightweight local block. The T-SCA submodule partitions the spatiotemporal encoded sequence, obtaining attention sub-features and residual sub-features. After performing local temporal single-head attention computation on the attention sub-features, they are fused with the residual sub-features to obtain the output features of the T-SCA submodule. The spatiotemporal encoded features are then residually connected with the output features of the T-SCA submodule to obtain the input features of the S-FCA submodule. The S-FCA submodule computes the input features using a multi-head attention mechanism to obtain global spatial dimension features. These global spatial dimension features are then residually connected with the output features of the T-SCA submodule to obtain the input features of the FFN submodule. The fully connected layers of the FFN submodule enhance the input features of the FFN submodule to obtain short-time local features. Specifically, the spatiotemporal encoded sequence... The input is fed into the T-SCA submodule, where it undergoes local time single-head attention computation on some features, and is then added to the original input features to form new features. The T-SCA submodule receives input features Then, the input features are divided into two sub-features along the channel dimension. The features of each channel constitute attention sub-features The features of the remaining channels constitute the residual sub-features. ,in, ,and The default value is For attention sub-features, T-SCA performs batch normalization on the attention sub-features, and then uses a single-head attention mechanism based on the local affinity matrix to perform weighted aggregation of the encoded sequence within the local time range to obtain the local time modeling result. The calculation formula is:

[0076] ,

[0077] ,

[0078] in, For linear projection, For batch normalization processing, Let be the local affinity matrix, representing the... The static weight relationship between codes within a local time range. It is Learnable parameter matrix within a local time range ,

[0079] ,

[0080] in, , They are respectively Two encoded sequences within a local time range, For The central time domain window;

[0081] Local time modeling results With residual sub-characteristics The features are concatenated along the channel dimension and fused using a learnable fusion matrix to obtain the output features of the T-SCA submodule, calculated as follows:

[0082] ,

[0083] in, For the output characteristics of the T-SCA submodule, For a learnable fusion matrix, ;

[0084] The spatiotemporal encoded sequence is residually concatenated with the output features of the T-SCA submodule to obtain the input features of the S-FCA submodule. The calculation formula is:

[0085] ,

[0086] in, As input features, It is a spatiotemporal encoded sequence;

[0087] The S-FCA submodule uses a multi-head self-attention mechanism to compute the input features, obtaining global spatial dimension features. The S-FCA submodule then processes these input features... Layer normalization is performed, and then the multi-head self-attention mechanism in standard ViT is followed for computation to obtain the features of the encoded sequence in the global spatial dimension, i.e., global spatial dimension features. The calculation formula is:

[0088] ,

[0089] ,

[0090] in, For a learnable fusion matrix, To attract attention, Let n be the number of attention heads. For n attention heads, the formula is:

[0091] ,

[0092] ,

[0093] in, , , For the distinct linear transformations in the nth attention head, it is shown that each attention head is in Modeling the encoded sequence within the global spatial scope of a single frame;

[0094] The global spatial dimension features are residually concatenated with the output features of the T-SCA submodule to obtain the input features of the FFN submodule. The formula for its calculation is:

[0095] ,

[0096] ,

[0097] The input features of the FFN submodule are enhanced by a fully connected layer. The enhanced features are then residually connected to the input features of the FFN submodule to obtain short-time local features. The FFN then receives these input features. Then, the input features After normalization, the data is passed through two fully connected layers connected by the GELU activation function to obtain the enhanced feature representation. , and then with By performing residual connections, the final output features of the LLB, namely short-time local features, are obtained. The formula for its calculation is:

[0098] ,

[0099] ,

[0100] ,

[0101] in, This is the up-dimensional matrix of the feedforward network. This is the dimensionality reduction matrix of the feedforward network. The default value is 4C.

[0102] Lightweight Global Blocks (LGBs) extract short-term local features to obtain global video features. LGBs are primarily used to capture long-range spatiotemporal dependencies between video frames, compensating for the limitations of local modeling in cross-frame modeling. Each LGB block consists of three sub-modules: Dynamic Position Encoding (DPE), Spatial-Temporal Sub-Channel Attention (ST-SCA), and Feed-Forward Network (FFN). These sub-modules are sequentially connected in a stacked manner, and residual connections are used between non-attention modules to enhance representation stability. The LGB receives output features from the LLB within the same layer. As input, The DPE module performs positional encoding, and the positional encoding result is added to the original input to obtain new features. ,Will And an additional learnable vector The input is fed into the ST-SCA module, and after partial feature global spatiotemporal single-head attention calculation, new features are obtained. ,Will The input FFN is used for final feature enhancement, and then combined with... Perform residual connections to obtain the final output features of the LLB. Specifically, short-term local features are input into the DPE submodule. The 3D convolutional layer of the DPE submodule performs positional encoding on these short-term local features. A residual connection is then performed between the short-term local features and the positionally encoded features to obtain spatiotemporal positional features and a learnable vector. The ST-SCA submodule further divides the spatiotemporal positional features and the learnable vector to obtain attention information sub-features, attention query sub-features, residual information sub-features, and residual query sub-features. Global spatiotemporal single-head attention computation is performed on the attention information sub-features and attention query sub-features, and then they are fused with the residual information sub-features to obtain the output features of the ST-SCA submodule. The fully connected layer of the FFN submodule enhances the output features of the ST-SCA submodule. A residual connection is then performed between the output features of the ST-SCA submodule and the enhanced features to obtain global video features. In essence, the DPE receives the input features. Then, a size of 3D convolution kernels, and with Step size for input features By performing depthwise 3D convolution, features embedded with spatiotemporal location information are obtained. ,Will Input features Adding them together yields the spatiotemporal location features. The formula for its calculation is:

[0103] ,

[0104] ,

[0105] ST-SCA Reception Characteristics Used to generate key vectors Sum value vector Receive learnable vectors Used to generate the query vector Q, where In the LGB block of layer 9, this refers to an additional learnable vector that is initialized with zero. In subsequent LGB blocks, it refers to the output of the LGB block in the previous layer; respectively and Divided into two sub-features along the channel dimension, the front The features of each channel constitute the attention information sub-features. With attention query sub-features The features of the remaining channels constitute the residual information sub-features. With residual query sub-features ,in, ,and The default value is For attention information sub-features With attention query sub-features ST-SCA first performs layer normalization on the attention information sub-features and attention query sub-features. Then, through a global spatiotemporal single-head attention mechanism, it performs weighted aggregation on the encoded sequence within the global spatiotemporal scope to obtain the global spatiotemporal modeling result. The formula for its calculation is:

[0106] ,

[0107] ,

[0108] ,

[0109] in, For linear projection, The cross-attention weights between the query vector and the global spatiotemporal encoding are calculated as follows:

[0110] ,

[0111] in, and These are different linear transformations;

[0112] For residual information sub-features ST-SCA performs global average pooling on the residual information sub-features along the encoding dimension to obtain global residual features. The formula for its calculation is:

[0113] ,

[0114] global spatiotemporal modeling results With global residual features The features are concatenated along the channel dimension and then fused using a learnable fusion matrix to obtain the final ST-SCA output features. The formula for its calculation is:

[0115] ,

[0116] in, A learnable fusion matrix;

[0117] FFN receives input features Then, process the input. After normalization, the data is passed through two fully connected layers connected by the GELU activation function to obtain the enhanced feature representation. Then, combined with input features Perform residual connections to obtain the final output features of the LLB, i.e., global video features. The formula for its calculation is:

[0118] ,

[0119] ,

[0120] ,

[0121] in, , These are the dimension-upgrading and dimension-reducing matrices of the feedforward network, respectively. The default value is 4C;

[0122] A dynamic fusion mechanism is used to fuse local and global video features to obtain video spatiotemporal semantic features. A classification module then performs classification and prediction on these features to obtain the prediction results. After feature extraction, the model extracts local feature representations and global video representations from the last layer's LLB and LGB, respectively. Specifically, the local feature representations are derived from the [CLS] encoding extracted from the last layer's LLB, denoted as... The global video representation originates from the video encoding output from the last LGB layer, denoted as... The two are fused using a weighted summation method to obtain the final video semantic feature F, which is used as the input to the downstream classifier. The calculation formula is as follows:

[0123] ,

[0124] in, These are the learnable parameters processed by the Sigmoid function.

[0125] In some embodiments, in step S102, the lightweight adapter module is embedded into the pre-trained backbone model to construct a domain adaptation model. The domain adaptation model is trained based on the acquired abnormal crew behavior dataset to obtain a trained domain adaptation model. The network structure of the lightweight adapter module is FST-Adapter. For a schematic diagram of the lightweight adapter module, please refer to [link to schematic diagram]. Figure 3 The FST-Adapter comprises a downsampling layer, a DPE layer, an offset network layer, a sparse self-attention layer, an activation function layer, and an upsampling layer. A crew abnormal behavior dataset is constructed, and a lightweight adapter module, Flexible Spatio-Temporal Adapter (FST-Adapter), is inserted into the pre-trained backbone model LA-UniFormerV2 to build a domain-adapted model for crew abnormal behavior recognition. The parameters of the domain-adapted model are then efficiently fine-tuned using the crew abnormal behavior dataset, thereby endowing the model with efficient recognition capabilities for crew abnormal behavior tasks. The crew abnormal behavior dataset is constructed based on real-world ship monitoring data from enterprises. The original data is collected from monitoring videos of multiple different types of ships during daily navigation, with shooting locations including core operational areas such as the bridge, engine room, and equipment control area. Sampling covers both day and night lighting environments, and video recording follows maritime regulatory standards with a uniform resolution. The frame rate was stabilized at 25 fps to ensure image clarity and temporal continuity. Abnormal crew behaviors were defined as six categories: smoking, using mobile phones, leaving their posts, sleeping on duty, walking in dangerous areas, and redundant conversations. 100 video samples were collected and labeled for each category, plus a control group of normal behaviors, totaling 700 samples. Each video was approximately 15 seconds long to ensure complete behavioral processes and clear semantics. Furthermore, all samples in the dataset underwent manual review to ensure label accuracy and scene consistency. Due to the limited sample size of the crew abnormal behavior dataset, to improve the model's generalization ability in small-sample scenarios, a lightweight adapter module, FST-Adapter, was inserted into the pre-trained backbone model LA-UniFormerV2 based on an adapter fine-tuning strategy, constructing a ship-oriented... This is a domain-adaptive model for identifying abnormal behavior in video personnel. The FST-Adapter utilizes a dynamic sampling mechanism to adaptively model the spatiotemporal dimensions of the video, achieving both parameter efficiency and adaptation flexibility. In terms of model structure design, the adapter module follows a full-coverage insertion strategy, being embedded into each LLB and LGB of the backbone model. Specifically, the lightweight adapter module is embedded before the T-SCA submodule in the lightweight local block and between the DPE and ST-SCA submodules in the lightweight global block to construct the domain-adaptive model. In each LLB, the adapter is embedded at the beginning of the module, before the T-SCA, to perform preliminary adjustments on short-term local features. For a schematic diagram of the structure of the lightweight adapter module embedded in the lightweight local block, please refer to [link to schematic diagram]. Figure 4 Within each LGB, an adapter is embedded between the DPE and ST-SCA, acting on the encoded spatiotemporal feature representation to guide it more effectively in completing cross-frame semantic modeling tasks; for a schematic diagram of the structure of the lightweight adapter module embedded in the lightweight global block, please refer to [link to schematic diagram]. Figure 5 After the domain adaptation model is built, it is trained using a crew abnormal behavior dataset. During the training process, only a small number of parameters inside the adapter module are updated, while the backbone model remains completely frozen. This enhances the model's semantic perception and behavior discrimination capabilities for crew abnormal behavior categories without interfering with the original modeling capabilities.

[0126] In some embodiments, please refer to Figure 6 The steps for training a domain-adaptive model include:

[0127] S601. Based on the abnormal behavior dataset of crew members, the input features are obtained and input into the lightweight adapter module. The input features are downsampled and normalized through the downsampling layer.

[0128] S602. The normalized features are encoded using the DPE layer, the encoded features are fused with the normalized features, and the fused encoded features are projected to obtain the query features.

[0129] S603. Uniformly sample the input features to obtain initial sampling points. Learn the query features through the offset network layer to obtain the offset of the initial sampling points. Move the initial sampling points based on the offset of the initial sampling points to obtain the set of moving sampling points. Perform feature interpolation on the set of moving sampling points to obtain the features of the moving sampling points. The offset network layer includes a depthwise 3D convolutional layer, a GELU layer, and a 3D convolutional layer.

[0130] S604. Project the features of the moved sampling points to obtain multiple key-value features. Perform sparse self-attention calculation on the multiple key-value features and the query features through a sparse self-attention layer to obtain spatiotemporal adaptation features.

[0131] S605. The spatiotemporal adaptation features are nonlinearly transformed by the activation function layer, and the spatiotemporal adaptation features after nonlinear transformation are upsampled by the upsampling layer to obtain the upsampled features. The upsampled features are then fused with the input features to obtain the output features.

[0132] In some embodiments, in step S601, input features are obtained based on the crew abnormal behavior dataset, and the input features are input to the lightweight adapter module. The input features are downsampled and normalized through a downsampling layer. The input features of the FST-Adapter are uniformly set to... The FST-Adapter downsamples the features and then performs a layer normalization operation, the calculation formula of which is:

[0133] ,

[0134] in, The features are normalized. This is the downsampling matrix. As input features, This is a layer normalization operation.

[0135] In some embodiments, in step S602, the normalized features are subjected to position encoding operation through the DPE layer, the position encoded features are fused with the normalized features, and the fused encoded features are projected to obtain the query features. The position encoding operation is performed through the DPE module, and the position encoding result is compared with... Adding together yields the fused features. Then use matrix right Projection yields query features The formula for its calculation is:

[0136] ,

[0137] ,

[0138] in, For query features.

[0139] In some embodiments, in step S603, the input features are uniformly sampled to obtain initial sampling points. The query features are learned through an offset network layer to obtain the offset of the initial sampling points. The initial sampling points are moved based on their offsets to obtain a set of moved sampling points. Feature interpolation is performed on the moved set of sampling points to obtain the moved sampling point features. The offset network layer includes a depthwise 3D convolutional layer, a GELU layer, and a 3D convolutional layer. The FST-Adapter operates in the time, height, and width directions of the input features. The density was uniformly sampled, and a total of Let there be an initial set of sampling points:

[0140] ,

[0141] in, , , These are the coordinates of the m-th initial sampling point in time, height, and width, respectively. The FST-Adapter uses a layer consisting of depthwise 3D convolutional layers, GELU layers, and... An offset network composed of 3D convolutional layers learns the query feature Q to obtain the offset of the initial sampling points, which is calculated as follows:

[0142] ,

[0143] in, This is the offset of the initial sampling point;

[0144] The initial sampling points are moved to areas of greater interest based on their respective offsets, resulting in a set of moved sampling points. This set of moved sampling points is as follows:

[0145] ,

[0146] in, , , Let be the coordinates of the m-th moving sampling point in terms of time, height, and width, respectively. The offset of the m-th initial sampling point is given by the FST-Adapter. Interpolation yields the features of the shifted sampled points. For the m-th shifted sampled point... The formula for its calculation is:

[0147] ,

[0148] in, Indicates traversal characteristics All spatiotemporal locations, .

[0149] In some embodiments, in step S604, the features of the moved sampling points are projected to obtain multiple key-value features. A sparse self-attention layer is then used to perform sparse self-attention calculations on the multiple key-value features and the query features to obtain spatiotemporal adaptation features. A matrix is ​​then used... and To each Projection yields key features and Sparse self-attention computation is performed with the query feature Q to obtain the spatiotemporal adaptation feature. The formula for its calculation is:

[0150] ,

[0151] ,

[0152] ,

[0153] in, For spatiotemporal adaptation features, This is the matrix transpose.

[0154] In some embodiments, in step S605, the spatiotemporal adaptation features are nonlinearly transformed by an activation function layer, and then upsampled by an upsampling layer to obtain upsampled features. These upsampled features are then fused with the input features to obtain output features. The spatiotemporal adaptation features are then fed into the GELU activation function and processed by a matrix... Upsample the features, then combine the upsampled features with the original input features. Add them together to obtain the output characteristics of the FST-Adapter. The formula for its calculation is:

[0155] ,

[0156] in, This is the upsampling matrix.

[0157] In some embodiments, in step S103, channel-level pruning is used to structurally compress the trained domain adaptation model to obtain a pruned domain adaptation model. The channel-level pruning is a CI-R (Channel Importance-Redundancy) pruning strategy. This involves determining the channels of the 3D convolutional layers in the trained domain adaptation model and calculating the importance score and redundancy index for each channel. The importance score and redundancy index are then fused to obtain a channel pruning score index. Based on the channel pruning score index, the channels are sorted and filtered to obtain the pruned domain adaptation model. CI-R pruning is implemented for all 3D convolutional layers in the domain adaptation model, including three steps: channel importance assessment, channel redundancy assessment, and fused score sorting and channel filtering. In the channel importance assessment stage, for the target layer, N sets of output features are first obtained using samples from the crew abnormal behavior dataset. These features are then statistically analyzed to calculate the average response intensity of each channel across all samples, thus obtaining the importance score for each channel. Subsequently, in the channel redundancy evaluation phase, for each channel's corresponding convolutional filter, cosine similarity is used to calculate its structural similarity with other channels, quantifying the channel's redundancy level R. Finally, channel importance and redundancy are integrated into a channel pruning score index. The specific integration method is as follows:

[0158] ,

[0159] in, The weight hyperparameter is set to 0.6 to balance the impact of importance and redundancy. The channels are sorted according to their channel pruning score values, and the top K most important channels are retained. The parameters of the remaining channels are pruned. After pruning, the model is fine-tuned and trained using the crew abnormal behavior dataset to restore the model's representation ability and discrimination performance, thereby obtaining a compact and stable crew abnormal behavior recognition model.

[0160] Calculate channel importance index Let the output features of the target layer be... , First, the kernel norm of the output features of the target layer is calculated to measure the overall information content of the features:

[0161] ,

[0162] in, for The Construct a series of binary mask matrices using singular values. , The first in the matrix Line 1 The elements of a column are defined as follows:

[0163] ,

[0164] in, To mask the c-th channel using a binary mask matrix, and Performing a Hadamard product operation masks the information representation of the c-th channel. Then, calculating the nuclear norm of the matrix after masking the c-th channel information, and subtracting the newly obtained matrix nuclear norm from the nuclear norm of the original feature matrix, we can obtain the importance index of the c-th channel in the n-th sample. Finally, averaging the results of n samples, we can obtain the importance index of the c-th channel.

[0165] ,

[0166] To calculate the channel redundancy index R, for the filter matrix corresponding to the c-th channel in the target layer, flatten it into a vector form, and represent it as follows: If cosine similarity is used to measure the similarity between two filters, then the first... , The similarity between the filters corresponding to each channel is:

[0167] ,

[0168] in, For the first , The similarity between the filters corresponding to each channel is calculated; the average similarity between the c-th channel and the filters corresponding to all other channels in the target layer is used to obtain the redundancy index of the c-th channel.

[0169] ,

[0170] in, This is the redundancy index for the c-th channel.

[0171] Building upon the domain-adaptive model, a channel-level pruning mechanism is introduced to compress the model's structure, resulting in the final model for recognizing abnormal crew behavior. CI-R pruning, by jointly considering the expressive power of channels in task response and the redundancy of convolutional kernel structures, constructs a fusion scoring metric to filter and retain channels. This significantly compresses model parameters and computational load while minimizing performance loss, thereby improving model deployment efficiency.

[0172] In some embodiments, in step S104, the acquired real-time ship monitoring video stream is identified based on the crew abnormal behavior recognition model to determine abnormal crew behavior. The abnormal crew behavior includes smoking, using a mobile phone, leaving one's post, sleeping on duty, walking in dangerous areas, and engaging in redundant conversations. When abnormal crew behavior is detected, if the duration of the abnormal behavior exceeds a preset threshold, a linkage response mechanism is triggered; if the duration of the abnormal behavior is less than or equal to the preset threshold, the crew abnormal behavior recognition model is used to identify the abnormal ship monitoring video stream. If no abnormal crew behavior is detected, the model is used to identify the monitoring video. Data is received from key areas such as the bridge and engine room. The system monitors real-time video streams from ships and uses a crew abnormal behavior recognition model to detect them. If the model determines that abnormal behaviors such as smoking, using mobile phones, leaving one's post, sleeping on duty, walking in dangerous areas, or engaging in redundant conversations exist, and the duration of such behavior exceeds a set threshold, a multi-faceted response mechanism is immediately triggered, including audible and visual alarms, SMS and email notifications, or console prompts. Simultaneously, information such as the type of abnormal behavior detected, the time of occurrence, and video clips is uniformly stored to support subsequent behavior analysis, accident retrospection, and crew management assessment. This mechanism significantly enhances the operational safety assurance capabilities during ship navigation, achieving a systematic leap from "passive monitoring" to "proactive early warning." The trained crew abnormal behavior recognition model is deployed in the actual ship monitoring system to achieve real-time intelligent monitoring and abnormal alarm functions for crew operational behavior.

[0173] This paper compares and analyzes the performance of the abnormal crew behavior recognition model with that of the existing representative Transformer-based method, Timesformer, on a crew abnormal behavior dataset. The Timesformer model, based on the Transformer architecture, consists of 12 layers. To adapt the Timesformer model to the crew abnormal behavior recognition task, a common method for transferring general video recognition models to specific small-sample domains was adopted for its training. First, Timesformer was pre-trained on the large-scale action recognition dataset Kinetics-400. Then, during the fine-tuning stage, the model weights of layers 3 to 8 were frozen, and only the parameters of the remaining layers were optimized to achieve fine-grained domain adaptation to crew abnormal behavior. The trained crew abnormal behavior recognition model is compared with the Timesformer model. The results are shown in Table 1.

[0174] Table 1

[0175]

[0176] As shown in Table 1, the crew abnormal behavior recognition model achieved a Top-1 accuracy of 88.76% while maintaining a parameter size of only 103M. In contrast, the Timesformer model achieved an accuracy of only 48.33% when the number of parameters reached 121M. The crew abnormal behavior recognition model reduced the number of parameters by about 14.9% and improved the recognition accuracy by more than 40 percentage points. In terms of computational overhead, the FLOPs of the crew abnormal behavior recognition model in the inference stage were only 148G, which is more than 75% less than the computational overhead of the Timesformer model. This significantly improved the model's operating efficiency in resource-constrained scenarios. The above results show that the crew abnormal behavior recognition model achieves dual optimization of structural compression and efficient inference while ensuring recognition accuracy, and has stronger engineering deployability and practical application value.

[0177] In summary, the abnormal crew behavior recognition method provided by this invention pre-trains a backbone model using a large-scale action recognition dataset to obtain a pre-trained backbone model; embeds a lightweight adapter module into the pre-trained backbone model to construct a domain adaptation model; trains the domain adaptation model based on the acquired abnormal crew behavior dataset to obtain a trained domain adaptation model; performs channel-level pruning to compress the structure of the trained domain adaptation model to obtain a pruned domain adaptation model; and obtains a crew abnormal behavior recognition model based on the pruned domain adaptation model. Finally, the method identifies abnormal crew behavior in the acquired real-time ship monitoring video stream based on the crew abnormal behavior recognition model, thereby improving recognition accuracy and ensuring safe ship operations.

[0178] To better implement the abnormal crew behavior identification method in this embodiment of the invention, based on the abnormal crew behavior identification method, correspondingly, such as... Figure 7 As shown, this embodiment of the invention also provides a crew abnormal behavior identification system. The crew abnormal behavior identification system 700 includes:

[0179] The backbone model acquisition module 701 is used to pre-train the constructed backbone model using a large-scale action recognition dataset to obtain a pre-trained backbone model, wherein the backbone model includes lightweight local blocks and lightweight global blocks.

[0180] The domain adaptation model construction module 702 is used to embed the lightweight adapter module into the pre-trained backbone model to construct a domain adaptation model. The domain adaptation model is trained based on the acquired abnormal behavior dataset of crew members to obtain a trained domain adaptation model.

[0181] The abnormal behavior recognition model acquisition module 703 uses channel-level pruning to perform structural compression on the trained domain adaptation model to obtain a pruned domain adaptation model, and obtains a crew abnormal behavior recognition model based on the pruned domain adaptation model.

[0182] The abnormal behavior recognition module 704 is used to identify the acquired real-time monitoring video stream of the ship based on the abnormal behavior recognition model of the crew members, so as to determine the abnormal behavior of the crew members.

[0183] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A crew abnormal behavior recognition method characterized by, The application relates to a method for constructing a LA-UniFormerV2 model. The method comprises the following steps: a large-scale action recognition data set is used to pre-train a constructed backbone model, wherein the backbone model comprises a lightweight local block and a lightweight global block, the network structure of the backbone model is LA-UniFormerV2, the LA-UniFormerV2 comprises 12-layer functional unit modules composed of a plurality of lightweight local blocks and a plurality of lightweight global blocks stacked by layers, a dynamic fusion mechanism and a classification module, the 12-layer functional unit modules comprise lightweight local blocks in the first 8 layers and a combination of lightweight local blocks and lightweight global blocks in the last 4 layers; the method comprises the following steps: a large-scale action recognition data set is input into the LA-UniFormerV2, the large-scale action recognition data set is pre-processed, and a space-time coding sequence is obtained; a short-time local feature is obtained by performing feature extraction on the space-time coding sequence through a lightweight local block, wherein the lightweight local block comprises a T-SCA submodule, an S-FCA submodule and an FFN submodule; the method comprises the following steps: the space-time coding sequence is input into the lightweight local block, the space-time coding sequence is divided through the T-SCA submodule, attention sub-features and residual sub-features are obtained, local time single-head attention calculation is performed on the attention sub-features, the residual sub-features are fused with the output features of the T-SCA submodule, and the output features of the T-SCA submodule are obtained; the space-time coding sequence and the output features of the T-SCA submodule are residually connected, the input features of the S-FCA submodule are obtained, the input features of the S-FCA submodule are calculated through a multi-head self-attention mechanism, and global spatial dimension features are obtained; the global spatial dimension features and the output features of the T-SCA submodule are residually connected, the input features of the FFN submodule are obtained, the input features of the FFN submodule are enhanced through a full connection layer of the FFN submodule, the features after feature enhancement are residually connected with the input features of the FFN submodule, and a short-time local feature is obtained; a global video feature is obtained by performing feature extraction on the short-time local feature through a lightweight global block, wherein the lightweight global block comprises a DPE submodule, an ST-SCA submodule and an FFN submodule, and the DPE submodule comprises a 3D convolution layer; the method comprises the following steps: the short-time local feature is input into the DPE submodule, the short-time local feature is positionally encoded through the 3D convolution layer of the DPE submodule, the short-time local feature and the features after position encoding are residually connected, and a space-time position feature is obtained; The learnable vector is obtained, the space-time position feature and the learnable vector are divided by the ST-SCA submodule respectively, the attention information sub-feature, the attention query sub-feature, the residual information sub-feature and the residual query sub-feature are obtained, the attention information sub-feature and the attention query sub-feature are subjected to global space-time single-head attention calculation, and then the residual information sub-feature is subjected to feature fusion to obtain the output feature of the ST-SCA submodule; The output feature of the ST-SCA submodule is subjected to feature enhancement by the full connection layer of the FFN submodule, and the output feature of the ST-SCA submodule and the feature after feature enhancement are subjected to residual connection to obtain the global video feature; The local feature and the global video feature are fused by the dynamic fusion mechanism to obtain the video space-time semantic feature; The video space-time semantic feature is classified and predicted by the classification module to obtain a prediction result; The lightweight adapter module is embedded into the pre-trained backbone model to construct a domain adaptation model, and the domain adaptation model is trained based on the obtained crew abnormal behavior data set to obtain a trained domain adaptation model, wherein the network structure of the lightweight adapter module is FST-Adapter, the FST-Adapter includes a down-sampling layer, a DPE layer, an offset network layer, a sparse self-attention layer, an activation function layer and an up-sampling layer; the embedding of the lightweight adapter module into the pre-trained backbone model to construct the domain adaptation model comprises: The lightweight adapter module is embedded into the T-SCA submodule before the lightweight local block and between the DPE submodule and the ST-SCA submodule of the lightweight global block to construct the domain adaptation model; The training of the domain adaptation model based on the obtained crew abnormal behavior data set to obtain the trained domain adaptation model comprises: Based on the crew abnormal behavior data set, an input feature is obtained, the input feature is input into the lightweight adapter module, and the input feature is subjected to down-sampling and normalization operation by the down-sampling layer; The normalized feature is subjected to position coding operation by the DPE layer, the position coded feature is fused with the normalized feature, and the fused coded feature is projected to obtain a query feature; The input feature is uniformly sampled to obtain an initial sampling point, the query feature is learned by the offset network layer to obtain an offset of the initial sampling point, the initial sampling point is moved based on the offset of the initial sampling point to obtain a set of moved sampling points, and the set of moved sampling points is subjected to feature interpolation to obtain moved sampling point features, wherein the offset network layer includes a depth-wise 3D convolution layer, a GELU layer and a 3D convolution layer; The moved sampling point features are projected to obtain a plurality of key-value features, and the plurality of key-value features and the query feature are subjected to sparse self-attention calculation by the sparse self-attention layer to obtain a space-time adaptation feature; The spatio-temporal adaptive feature is nonlinearly transformed by an activation function layer, and an up-sampling layer is used to up-sample the nonlinearly transformed spatio-temporal adaptive feature to obtain an up-sampled feature; the up-sampled feature is fused with the input feature to obtain an output feature; The trained domain adaptive model is structurally compressed by adopting channel-level pruning to obtain a pruned domain adaptive model, and a crew abnormal behavior recognition model is obtained based on the pruned domain adaptive model; The obtained ship real-time monitoring video stream is recognized based on the crew abnormal behavior recognition model to determine the crew abnormal behavior.

2. The crew abnormal behavior recognition method according to claim 1, characterized by, The trained domain adaptive model is structurally compressed by adopting channel-level pruning to obtain a pruned domain adaptive model, and a crew abnormal behavior recognition model is obtained based on the pruned domain adaptive model; The channels of the 3D convolution layer in the trained domain adaptive model are determined, and the importance score and redundancy index of each channel are calculated; The importance score and the redundancy index are fused to obtain a channel pruning score index; The channels are sorted and screened based on the channel pruning score index to obtain a pruned domain adaptive model.

3. The crew abnormal behavior recognition method according to claim 2, characterized by, The channel pruning score index is: ， wherein, is a channel pruning score indicator, is an importance score for each channel, is a redundancy indicator, is a weight hyperparameter.

4. The crew abnormal behavior recognition method according to claim 1, characterized by, The obtained ship real-time monitoring video stream is recognized based on the crew abnormal behavior recognition model to determine the crew abnormal behavior, including: The ship real-time monitoring video stream is obtained, and the ship real-time monitoring video stream is recognized based on the crew abnormal behavior recognition model to determine the crew abnormal behavior, wherein the crew abnormal behavior includes smoking, playing a mobile phone, leaving a post, sleeping on duty, walking in a dangerous area, and redundant conversation; When the duration of the crew abnormal behavior is greater than a preset threshold, a linkage response mechanism is triggered; when the duration of the crew abnormal behavior is less than or equal to the preset threshold, the recognition of the ship real-time monitoring video stream by the crew abnormal behavior recognition model is performed.

5. A crew abnormal behavior recognition system characterized by, It includes: The backbone model obtaining module is configured to pre-train a constructed backbone model using a large-scale action recognition dataset to obtain a pre-trained backbone model, wherein the backbone model includes lightweight local blocks and lightweight global blocks, and the network structure of the backbone model is LA-UniFormerV2, the LA-UniFormerV2 includes 12-layer functional unit modules stacked by multiple lightweight local blocks and multiple lightweight global blocks, a dynamic fusion mechanism, and a classification module, the 12-layer functional unit modules include the first 8 layers of lightweight local blocks and the last 4 layers of lightweight local blocks and lightweight global blocks in combination; the backbone model is pre-trained using the large-scale action recognition dataset, including: The large-scale action recognition dataset is input into the LA-UniFormerV2, and the large-scale action recognition dataset is preprocessed to obtain a spatio-temporal coding sequence; The spatio-temporal coding sequence is feature-extracted by a lightweight local block to obtain short-time local features, and the lightweight local block includes a T-SCA submodule, an S-FCA submodule, and an FFN submodule; the spatio-temporal coding sequence is feature-extracted by the lightweight local block to obtain short-time local features, including: The spatio-temporal coding sequence is input into a light local block, the spatio-temporal coding sequence is divided by a T-SCA submodule, attention sub-features and residual sub-features are obtained, the attention sub-features are calculated by local time single-head attention, and then the residual sub-features are fused to obtain output features of the T-SCA submodule; The spatio-temporal coding sequence and the output features of the T-SCA submodule are connected in residual, input features of an S-FCA submodule are obtained, the input features of the S-FCA submodule are calculated by a multi-head self-attention mechanism, and global spatial dimension features are obtained; The global spatial dimension features and the output features of the T-SCA submodule are connected in residual, input features of an FFN submodule are obtained, the input features of the FFN submodule are enhanced by a fully connected layer of the FFN submodule, the enhanced features are connected in residual with the input features of the FFN submodule, and short-time local features are obtained; Global video features are obtained by extracting the short-time local features by a light global block, the light global block includes a DPE submodule, an ST-SCA submodule and an FFN submodule, the DPE submodule includes a 3D convolution layer, and the extracting the short-time local features by the light global block to obtain the global video features includes: The short-time local features are input into the DPE submodule, the short-time local features are position encoded by the 3D convolution layer of the DPE submodule, and the short-time local features and the position encoded features are connected in residual to obtain spatio-temporal position features; A learnable vector is obtained, the spatio-temporal position features and the learnable vector are divided by the ST-SCA submodule respectively, attention information sub-features, attention query sub-features, residual information sub-features and residual query sub-features are obtained, the attention information sub-features and the attention query sub-features are calculated by global spatio-temporal single-head attention, and then the residual information sub-features are fused to obtain output features of the ST-SCA submodule; The output features of the ST-SCA submodule are enhanced by a fully connected layer of the FFN submodule, and the output features of the ST-SCA submodule and the enhanced features are connected in residual to obtain global video features; The local features and the global video features are fused by a dynamic fusion mechanism to obtain video spatio-temporal semantic features; The video spatio-temporal semantic features are classified and predicted by a classification module to obtain a prediction result. The field adaptation model construction module is configured to embed the lightweight adapter module into the pre-trained backbone model to construct a field adaptation model, train the field adaptation model based on the obtained seafarer abnormal behavior dataset, and obtain a trained field adaptation model. The lightweight adapter module is embedded into the T-SCA submodule in the lightweight local block and between the DPE submodule and the ST-SCA submodule in the lightweight global block to construct the field adaptation model. The field adaptation model is trained based on the obtained seafarer abnormal behavior dataset to obtain a trained field adaptation model. The input features are obtained based on the seafarer abnormal behavior dataset, and the input features are input into the lightweight adapter module. The normalized features are subjected to position encoding operation through the DPE layer, and the position encoded features are fused with the normalized features. The initial sampling points are obtained by uniformly sampling the input features, and the offset of the initial sampling points is obtained by learning the query features through the offset network layer. The initial sampling points are moved based on the offset of the initial sampling points to obtain a set of moved sampling points, and the moved sampling point features are obtained by feature interpolation on the set of moved sampling points. The moved sampling point features are projected to obtain a plurality of key-value features, and sparse self-attention calculation is performed on the plurality of key-value features and the query features through the sparse self-attention layer to obtain a space-time adaptation feature. The space-time adaptation feature is subjected to nonlinear transformation through the activation function layer, and the nonlinear transformed space-time adaptation feature is upsampled through the upsampling layer to obtain an upsampled feature. The upsampled feature is fused with the input feature to obtain an output feature. The abnormal behavior recognition model obtaining module is configured to perform channel-level pruning on the trained field adaptation model to obtain a pruned field adaptation model, and obtain a seafarer abnormal behavior recognition model based on the pruned field adaptation model. The abnormal behavior recognition module is configured to identify the obtained ship real-time monitoring video stream based on the seafarer abnormal behavior recognition model to determine the seafarer abnormal behavior.

Images