A semantic analysis model training method, video data transmission method, electronic device and computer-readable storage medium

Abstract

The present invention discloses a semantic analysis model training method, a video data transmission method, an electronic device and a computer-readable storage medium, which belong to the field of artificial intelligence and video transmission technology. The training method constructs a multi-task fusion network based on the MobileViT‑S architecture, combines a lightweight Transformer with a convolutional module, and synchronously extracts semantic information of targets, actions and scenes in the video. The training adopts a two-stage strategy: the pre-training stage uses a general data set to optimize basic capabilities; the fine-tuning stage introduces curriculum learning and temporal smoothing loss, gradually increases the weight of complex ocean scene samples, jointly optimizes the multi-task loss function, and dynamically adjusts the weight to balance accuracy and bandwidth adaptation. The model is deployed on the ship side to extract structured semantic information; the shore-based system restores the video through the GAN+Diffusion model, and combines optical flow interpolation and Kalman filtering to improve temporal consistency. This solves the problem of real-time video transmission in low-bandwidth scenarios, and can be applied to fields such as ship monitoring and remote collaboration, significantly improving transmission efficiency and reliability.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and video transmission technology, and specifically relates to a semantic analysis model training method, a video data transmission method, an electronic device and a computer-readable storage medium. Background Art

[0002] Video transmission between ships and shore is a core technology for maritime safety monitoring, remote collaboration, and emergency response. By transmitting real-time video data on the vessel's surroundings, equipment status, and personnel operations, shore-based command centers can quickly make decisions and respond to emergencies (such as people falling overboard and equipment failures). However, maritime communication networks are limited by satellite links or cellular network coverage, resulting in extremely low bandwidth (typically 10kbps to 1Mbps) and high latency fluctuations (500ms to 5s). Traditional video transmission technologies struggle to meet real-time and reliability requirements.

[0003] Existing technologies utilize deep learning models for semantic analysis of videos; however, they suffer from numerous limitations, such as: Single-task model dominance: Existing semantic analysis models often focus on a single task (such as object detection or action recognition) and lack multi-task joint modeling of objects, actions, and scenes. This results in fragmented model output information, requiring multiple inferences to obtain complete semantics, increasing computational latency (single-frame processing time ≥ 100ms). Poor scene adaptability: Common video datasets (such as COCO and Kinetics) lack coverage of marine environments (such as waves, fog, and low light), resulting in a significant decrease in semantic extraction accuracy in ship-to-shore scenarios (experiments show that conventional models experience a ≥30% decrease in target detection mean average performance (mAP) in fog). Lack of temporal continuity: Existing models do not explicitly constrain the semantic coherence of adjacent frames, resulting in jumps in the transmitted semantic description (such as sudden changes in object position), compromising the fidelity of shore-based video reconstruction.

[0004] At the same time, existing video data transmission technologies also have some problems, such as: High bandwidth dependence: Although traditional video compression technologies (such as H.265) can reduce data volume, they still require sacrificing resolution or frame rate (such as down to 240p@5fps) under extremely low bandwidth (<50kbps), resulting in significant loss of key details (such as small floating objects and human movements). Static encoding strategy: Existing transmission solutions use a fixed compression rate and cannot dynamically adjust semantic granularity based on bandwidth fluctuations. For example, when bandwidth drops sharply, complete scene information is still transmitted, resulting in a surge in transmission delay (≥1s). Insufficient end-to-end optimization: Existing "video→text→video" solutions (such as reverse applications based on text-based image technology) do not have dedicated codec protocols designed for ship-shore scenarios. This results in poor compatibility between semantic information and the restoration model, resulting in deformation or logical errors in the restored video (such as distortion of the ship's motion trajectory).

[0005] Combined with the shortcomings of the aforementioned technologies, they fail to address the following core issues: The conflict between low bandwidth and high fidelity: How can the accuracy of shore-based reconstruction of key video content (objects, actions, and scenes) be guaranteed to be ≥95% while compressing the transmitted data volume to 1%-5% of the original video? Semantic robustness in complex environments: How can the model's semantic extraction accuracy be improved in dynamic ocean environments (such as waves, fog, and changing lighting) to avoid false and missed detections? Lack of a real-time guarantee mechanism: How can end-to-end latency of ≤200ms (including ship-side processing, transmission, and shore-based reconstruction) be achieved to meet the real-time response requirements for emergency events (such as man overboard)?

[0006] Therefore, there is an urgent need for a full-link optimization solution from model training to transmission protocol to break through the technical bottlenecks of low bandwidth, high real-time performance, and adaptation to complex scenarios. Summary of the Invention

[0007] The purpose of the present invention is to provide a semantic analysis model training method, a video data transmission method, an electronic device and a computer-readable storage medium to solve the adaptability of the network model to complex marine environments and the real-time transmission of video in low-bandwidth scenarios.

[0008] The present invention provides a semantic analysis model training method for transmitting video data between a ship and a shore, the method comprising the following steps:

[0009] Step 1: Construct a semantic analysis model, which includes:

[0010] Input module, which receives RGB images and optical flow information and splices them into 5 channels for input;

[0011] The backbone network uses a lightweight Transformer and convolutional hybrid architecture MobileVit-S, and embeds a spatiotemporal attention mechanism;

[0012] The multi-task fusion branch includes the target detection branch, the action recognition branch, and the scene coding branch, which output target category and location, action label, and scene label respectively;

[0013] Dynamic feature selection module, dynamically shutting down non-critical task branches based on the current bandwidth status;

[0014] Step 2: Train the semantic analysis model: A two-stage training strategy is adopted, including:

[0015] Phase 1: Pre-training on a general video dataset using the AdamW optimizer and Focal Loss;

[0016] Phase 2: Fine-tuning on ship-shore scenario data, using a curriculum learning strategy to gradually increase the weight of complex samples and adding a time series smoothing loss;

[0017] The MobileViT-S consists of six stages, Stage 1 to Stage 6, consisting of alternating stacks of convolutional layers and Transformer modules. The model types of Stage 1, Stage 3, and Stage 5 are Conv2D+BN+Swish, the model types of Stage 2 and Stage 4 are MobileViT Block, and the model type of Stage 6 is Global Context Transformer.

[0018] The spatiotemporal attention mechanism introduces 3D spatiotemporal position coding in the Transformer layer to capture the temporal dependencies between video frames. The 3D spatiotemporal position coding consists of two parts: 2D spatial coding and 1D temporal coding. The specific design is to embed local window spatial attention in Stage 2 and Stage 4, and global temporal attention in Stage 6.

[0019] The semantic analysis model training method described above is further preferably configured such that the specific structure of the MobileViT Block includes: CNN, Unfold, Transformer, Fold, and CNN, which fuses local-global features, divides the image into local blocks through the Unfold operation, inputs the image into the Transformer to capture the global relationship, and then restores the spatial structure through Fold. At the same time, some fully connected layers are replaced by depthwise separable convolutions to reduce the number of parameters.

[0020] In the above-mentioned semantic analysis model training method, it is further preferred that the implementation of the dynamic feature selection module includes the following steps:

[0021] Bandwidth status monitoring and classification: Real-time monitoring of ship-to-shore communication bandwidth, and classification of bandwidth into high, medium, and low levels based on predefined thresholds;

[0022] Dynamically configure task branches: When the bandwidth is high, all task branches are enabled; when the bandwidth is medium, the scene encoding branch is disabled, and the object detection and action recognition branches are retained; when the bandwidth is low, only the object detection branch is enabled, and the detection granularity is reduced;

[0023] An adaptive adjustment mechanism is introduced: based on a lightweight decision tree model, the bandwidth fluctuation trend is predicted and the task branch configuration is adjusted in advance; if the key target is detected to be lost for three consecutive frames, the action recognition branch is forcibly started and a high-optimization level transmission is triggered. The above-mentioned semantic analysis model training method is further preferably, before performing the semantic analysis model training, it also includes a preprocessing operation on the video data to extract RGB images and optical flow information from the video data.

[0024] The semantic analysis model training method described above is further preferably configured such that the first stage specifically includes:

[0025] Configure a general dataset, using the public video dataset Kinetics-400, covering daily actions and complex scenes;

[0026] Data augmentation using random cropping, color jittering, and motion blur simulation;

[0027] Set the initial learning rate of the optimizer AdamW to , weight decay is 0.05, batch size is 32, and input resolution is 256×256; set the training objectives to object detection mAP@0.5 ≥ 75%, action recognition Top-1 accuracy ≥ 82%, and scene classification F1-score ≥ 0.88.

[0028] In the above-mentioned semantic analysis model training method, it is further preferred that the second stage specifically includes:

[0029] Collect real ship-shore surveillance videos and annotate them. The annotation information includes target category, action label and scene label;

[0030] Simulate sea fog, wave vibration, and salt spray noise for data enhancement;

[0031] Set a course learning strategy, including using only simple scene samples from epochs 1-50; gradually adding complex scene samples from epochs 51-100, increasing the proportion from 20% to 80%; and training on all complex scenes from epochs 101-150, adding adversarial examples;

[0032] Design a multi-task joint loss function and introduce temporal smoothing constraint loss. The joint loss function is: ,in : Focal Loss for target detection; : Cross entropy loss for action recognition; : Label smoothing loss for scene classification; : Temporal smoothing constraint loss;

[0033] Dynamically adjust the weights of multi-task branches: the initial weights are α=1.0, β=0.8, γ=0.5, and λ=0.2, where α represents the weight corresponding to target detection, β represents the weight corresponding to action recognition, γ represents the weight corresponding to scene classification, and λ represents the weight of the temporal smoothing constraint loss; increase λ linearly as the training progresses, and finally λ=0.6.

[0034] The present invention also discloses a video data transmission method, which is applied to data communication between ship and shore, comprising: deploying a semantic analysis model trained by the semantic analysis model training method described above on a ship-end edge device; generating video frames based on a GAN+Diffusion model on a shore-based end, and performing a video restoration operation.

[0035] The video data transmission method described above is further preferably configured to include the following steps:

[0036] Extracting video semantic information: Using semantic analysis models to perform semantic analysis on video data, extract key information, and generate structured semantic descriptions. Key information includes objects, actions, and scenes.

[0037] Semantic information compression: Encode and compress semantic descriptions to reduce data transmission volume;

[0038] Data transmission: The compressed semantic information is sent to the shore-based end according to the transmission protocol;

[0039] Semantic information restoration: The shore-based end decodes the semantic information and uses the generation module to restore it to video frames;

[0040] Video reconstruction: Combine the restored video frames in chronological order to generate complete video data;

[0041] Video post-processing: Use optical flow-guided interpolation to insert intermediate frames into low-frequency clips to increase the frame rate; apply time domain filtering to smooth the target position; and use objective indicators to evaluate the quality of the generated video.

[0042] The present invention also discloses an electronic device, comprising one or more processors; a storage device for storing one or more computer programs, wherein when the one or more computer programs are executed by the one or more processors, the electronic device implements the above-mentioned semantic analysis model training method or the above-mentioned video data transmission method.

[0043] The present invention also discloses a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor of an electronic device, the electronic device executes the above-mentioned semantic analysis model training method or the above-mentioned video data transmission method.

[0044] The beneficial effects of the present invention are: the semantic analysis model training method provided by the present invention is used for video data transmission between ship and shore, and the method comprises the following steps: Step 1: constructing a semantic analysis model, the model comprising: an input module, receiving RGB images and optical flow information and splicing them into 5 channels for input; a backbone network, adopting a lightweight Transformer and convolutional hybrid architecture MobileVit-S, and embedding a spatiotemporal attention mechanism; a multi-task fusion branch, including a target detection branch, an action recognition branch, and a scene encoding branch, which output target categories and positions, action labels, and scene labels respectively; a dynamic feature selection module, which dynamically closes non-critical task branches according to the current bandwidth status; Step 2: training the multi-task fusion network: adopting a two-stage training strategy, including: Stage 1: pre-training on a general video dataset, using the AdamW optimizer and Focal Loss; Stage 2: fine-tuning on ship-shore scene data, adopting a curriculum learning strategy to gradually increase the weights of complex samples, and adding temporal smoothing loss. Furthermore, a video data transmission method is provided, which deploys the trained semantic analysis model to the ship's edge device, generates video frames based on the GAN+Diffusion model on the shore, and performs video restoration operations. This invention constructs a multi-task fusion branch to synchronously extract target, action, and scene semantic information from the video, and improves the model's adaptability to complex marine environments based on a two-stage training strategy. It also solves the problem of real-time video transmission in low-bandwidth scenarios and can be applied to fields such as ship monitoring and remote collaboration, significantly improving transmission efficiency and reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 A flowchart of the steps of the training method of the semantic analysis model of the present invention;

[0047] Figure 2 A flowchart of the steps of the dynamic feature selection module of the present invention;

[0048] Figure 3 This is a flowchart of the steps of the video data transmission method of the present invention. DETAILED DESCRIPTION

[0049] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation methods, structures, features and effects of the semantic analysis model training method and video data transmission method proposed in the present invention. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics of one or more embodiments may be combined in any suitable form.

[0050] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0051] The following describes in detail the specific schemes of the semantic analysis model training method and the video data transmission method provided by the present invention with reference to the accompanying drawings.

[0052] See also Figure 1 , which shows a flowchart of the steps of the training method of the semantic analysis model provided by one embodiment of the present invention, such as Figure 1 As shown, the training method of the semantic analysis model provided in the embodiment of the present application, which is used for video data transmission between ship and shore, includes the following steps 1 to 2:

[0053] Step 1: Construct a semantic analysis model, which includes:

[0054] Input module, which receives RGB images and optical flow information and splices them into 5 channels for input;

[0055] The backbone network uses a lightweight Transformer and convolution hybrid architecture MobileViT-S, embedded with a spatiotemporal attention mechanism;

[0056] The multi-task fusion branch includes the target detection branch, the action recognition branch, and the scene encoding branch, which output the target category and position, the action label, and the scene status respectively;

[0057] Dynamic feature selection module, dynamically shutting down non-critical task branches based on the current bandwidth status;

[0058] It should be noted that to address the multiple inferences and redundant computations caused by traditional approaches that employ separate target detection, action recognition, and scene classification models, this paper proposes a semantic analysis model. Through feature sharing, joint optimization, and dynamic control, this model addresses core issues such as low semantic extraction efficiency, poor environmental adaptability, and insufficient transmission flexibility in ship-to-shore scenarios. This model is the technical cornerstone for achieving the full-link optimization of "high-precision semantic extraction, low-bandwidth transmission, and real-time restoration." This network model comprises an input module, a backbone network, a multi-task fusion branch, and a dynamic feature selection module.

[0059] To simultaneously capture both static spatial features and temporal dynamic features, thereby improving the model's understanding of video content, the input module concatenates the captured RGB images with optical flow information to produce a corresponding 5-channel input. The RGB images (3 channels) provide static spatial information such as color, texture, and object shape, such as the color of a ship or the shape of waves. Optical flow information (2 channels) represents temporal motion information, such as the direction of a ship or the frequency of waves, by calculating pixel displacements (X / Y motion vectors) between adjacent frames. By fusing these two, both the "what" and "how" of motion can be modeled. Channel concatenation can be performed using the torch.cat function in the deep learning framework PyTorch or the tf.concat function in TensorFlow, along the channel dimension.

[0060] The backbone network of this invention adopts a lightweight Transformer and convolution hybrid architecture (MobileViT-S), which reduces computational complexity while ensuring model accuracy and adapts to the deployment requirements of ship-side edge devices. The detailed structural design is as follows:

[0061] 1. Input preprocessing layer

[0062] Input format: 5-channel tensor (3 channels RGB + 2 channels optical flow), resolution 384×384.

[0063] Preprocessing operations:

[0064] Channel normalization: RGB channel: pixel values ​​are normalized to [0, 1]; optical flow channel: displacement values ​​are normalized to [-1, 1].

[0065] Spatial alignment: Bilinear interpolation is used to ensure that the optical flow strictly matches the RGB image size.

[0066] 2. Feature extraction backbone

[0067] The backbone network consists of alternating convolutional neural network (CNN) and Transformer modules, and consists of six stages (Stage 1 to 6). The specific structure is shown in Table 1:

[0068] Table 1

[0069]

[0070] Key module description:

[0071] 1. The structure of MobileVit Block includes: CNN, Unfold, Transformer, Fold and CNN, which is: (1) local-global feature fusion, the image is divided into local blocks (such as 8×8) through the Unfold operation, the input is Transformer to capture the global relationship, and then the spatial structure is restored through Fold; (2) lightweight design: compared with the standard ViT, the number of parameters is reduced by 70% (by replacing some fully connected layers with depthwise separable convolution).

[0072] 2.Global Context Transformer: Its function is to perform global context modeling on the 12×12×256 feature map output by Stage 5.

[0073] It's important to note that alternating convolution and Transformer stacking refers to the use of CNN-based and Transformer-based modules alternately across different stages of the backbone network, rather than a strict restriction that each stage must be purely CNN or purely Transformer. The alternating logic is as follows: Pure CNN stages (Stages 1 / 3 / 5) are responsible for fast downsampling, feature compression, and low-level / high-level feature extraction; Hybrid module stages (Stages 2 / 4) achieve local-global feature fusion through the collaboration of CNN and Transformer in MobileViTBlock; Pure Transformer stage (Stage 6) focuses on modeling long-term temporal dependencies. This design forms an alternating pattern of "CNN → Hybrid → CNN → Hybrid → CNN → Transformer" at the stage level. This design not only improves computational efficiency but also achieves functional complementarity. Specifically, the pure CNN stages (Stages 1, 3, and 5) quickly process high-resolution inputs, avoiding the computational overhead of global attention. The hybrid module stages (Stages 2 and 4) introduce Transformer at appropriate resolutions (such as 96×96 and 24×24) to balance computational complexity and semantic modeling requirements.

[0074] At the same time, the backbone network also embeds a spatiotemporal attention mechanism, which is implemented through 3D position encoding and local-global attention separation. It is divided into two parts: spatial attention (within a local window) and temporal attention (global across frames). Specifically:

[0075] The position encoding of spatial attention adds 2D sinusoidal position encoding to each pixel in the local window to capture the relative position in space. The calculation scope only calculates self-attention within the local window, reducing the amount of calculation and the complexity from down to .

[0076] Temporal attention's temporal position encoding adds a 1D temporal code to the same spatial position in consecutive video frames to represent temporal order. Cross-frame attention, in Stage 6's Global Context Transformer, performs global self-attention on multi-frame feature sequences to capture long-term temporal dependencies (such as the trajectory of a ship).

[0077] The specific design is to embed local window spatial attention in Stage 2 and Stage 4, and global temporal attention (across frames) in Stage 6. This design, which applies attention only in Stages 2, 4, and 6, is the result of a comprehensive optimization based on computational efficiency, feature abstraction requirements, and real-time constraints: 1. Early Stage (Stages 1 / 3): Convolution efficiently extracts low-level features, avoiding computational explosion at high resolutions; 2. Mid-Stage (Stages 2 / 4): Local window attention models medium-granularity semantics, balancing accuracy and speed; 3. Late Stage (Stage 6): Global attention captures long-term temporal dependencies, improving action and scene understanding.

[0078] Therefore, the backbone network proposed in this paper utilizes a lightweight hybrid Transformer and convolutional architecture, embedded with a spatiotemporal attention mechanism, providing the core technical support for achieving the trifecta of high accuracy, low latency, and low power consumption in ship-to-shore scenarios. Furthermore, the design goal of MobileViT-S is to minimize the number of parameters and computational complexity while maintaining high accuracy, adapting to the real-time inference requirements of edge devices (such as the Jetson AGX Xavier onboard).

[0079] The multi-task fusion branch is the backbone network output multi-scale feature map (Stage2~Stage6), which is distributed to three branches through the task adapter: target detection branch, action recognition branch and scene encoding branch, where:

[0080] The target detection branch takes Stage 4 output (24×24×128) and Stage 5 output (12×12×256) as input. The structure is as follows: 1. Feature fusion: Upsampling the Stage 5 features to 24×24 through deconvolution and concatenating them with the Stage 4 features (24×24×384). 2. The detection head uses a YOLO-style anchor box mechanism to output: object categories (80 categories + 10 ship and shore extension categories); bounding box categories (x1, y1, x2, y2, normalized to [0, 1]); and confidence scores (0–1). 3. Post-processing: Non-Maximum Suppression (NMS) is used, with an IOU threshold of 0.5.

[0081] Action recognition branch: The input is the Stage 3 output (48×48×64) + temporal optical flow features (5 consecutive frames). Its structure is as follows: 1. Spatiotemporal feature extraction, including spatial features (global average pooling of the Stage 3 output to obtain a 64-dimensional vector) and temporal features (the optical flow field of 5 consecutive frames is input into the GRU network to output a 128-dimensional temporal vector); 2. Fusion classification, including splicing spatial and temporal features (64+128=192 dimensions) and outputting 20 types of action probabilities through a fully connected layer (192→64→20).

[0082] Scene encoding branch: The input is the Stage 6 output (1×1×512 global features). Its structure is as follows: 1. Multi-label classification, including fully connected layer 1 (512→256, activation function Swish) and fully connected layer 2 (256→32, output scene labels (such as "wave level" and "visibility")); 2. The output format is a 32-dimensional vector, with each bit corresponding to a binary classification of a scene attribute (Sigmoid activation).

[0083] The dynamic feature selection module implements branch-level control and adaptive adjustment mechanisms to achieve real-time response under bandwidth fluctuations. The specific structure is as follows:

[0084] The input is the intermediate feature map of each branch (such as the 24×24×384 features of the target detection branch); the control logic is as follows: 1. Bandwidth monitoring, real-time acquisition of network bandwidth (such as 10Kbps), and division of gears according to thresholds; 2. Branch switch, the specific operations are as follows: when the bandwidth is high, all branches are started; when the bandwidth is medium, the scene encoding branch is closed and its input features (24×24×384) are cached; when the bandwidth is low, only the target detection branch is enabled, and the output granularity is reduced (confidence threshold 0.5→0.8); 3. Introducing an adaptive adjustment mechanism: Based on a lightweight decision tree model, the bandwidth fluctuation trend is predicted and the task branch configuration is adjusted in advance; if the key target is detected to be lost for three consecutive frames, the action recognition branch is forcibly started and a high-optimization level transmission is triggered.

[0085] Step 2: Train the semantic analysis model: A two-stage training strategy is adopted, including: Stage 1: pre-training on a general video dataset, using the AdamW optimizer and Focal Loss; Stage 2: fine-tuning on ship-shore scene data, using a curriculum learning strategy to gradually increase the weight of complex samples and adding a temporal smoothing loss.

[0086] It should be noted that the model training strategy includes three parts: dataset construction, two-stage training method, and loss function design. Specifically:

[0087] Dataset construction: A dedicated dataset for ship-shore scenarios collects video data of scenes including ships, marine environments, and human operations, and annotates them with semantic labels (objects, actions, and scenes). Data enhancement simulates marine environmental interference (such as fog and wave jitter) to improve model robustness.

[0088] Two-stage training method:

[0089] Phase 1 (pre-training): Train a multi-task model on a general video dataset to initially learn semantic extraction capabilities.

[0090] Phase 2 (fine-tuning): A curriculum learning strategy is used on the ship-shore dataset to gradually increase the weight of complex scene samples and optimize the model's adaptability to the marine environment.

[0091] The two-stage training strategy is specifically designed as follows:

[0092] Phase 1: General Dataset Pre-training

[0093] Dataset configuration:

[0094] Use public video datasets (Kinetics-400, AVA v2.2) to cover daily actions and complex scenes.

[0095] Data augmentation: random cropping, color jittering, and motion blur simulation (intensity 5%~15%).

[0096] Loss function design:

[0097] Object detection branch: Focal Loss (α=0.25, γ=2) is used to address the problem of class imbalance (such as missed detection of small objects);

[0098] Action recognition branch: uses cross entropy loss (with category weights, giving higher weights to rare actions);

[0099] Scene classification branch: Use label smoothing loss (smooth=0.1) to improve generalization.

[0100] Training parameters:

[0101] Optimizer: AdamW (initial learning rate , weight decay 0.05).

[0102] Batch size: 32 (single GPU training), input resolution 256×256.

[0103] Training goal: The model should achieve the following performance in common scenarios: mAP@0.5 for object detection ≥ 75%; Top-1 accuracy for action recognition ≥ 82%; and F1-score for scene classification ≥ 0.88.

[0104] Phase 2: Fine-tuning the ship-shore scene

[0105] Dataset construction:

[0106] Collect real ship-shore surveillance videos (≥10,000 segments) and annotate them with the following extended information: Target categories: Add ship-shore specific categories such as "lifeboat," "buoy," and "oil pollution area"; Action tags: Define emergency events such as "cable break" and "cargo tilt"; Scene tags: Mark wave height (0.5m~5m) and visibility (100m~1000m).

[0107] Data augmentation:

[0108] Simulate marine environmental interference: add fog (random concentration), wave jitter (random affine transformation), and salt spray noise (SPNoise probability 5%).

[0109] Course Learning Strategies:

[0110] Epochs 1-50: Only simple scene samples (visibility ≥ 500m, waves ≤ 1m) are used; Epochs 51-100: Complex scene samples (visibility < 500m, waves > 1m) are gradually added, with the proportion increasing from 20% to 80%; Epochs 101-150: Full complex scene training is performed, and adversarial examples are added (extreme weather videos generated using GAN).

[0111] Loss function adjustment:

[0112] Loss function design: multi-task joint loss, the function is as follows:

[0113] ,in, : FocalLoss for target detection; : Cross entropy loss for action recognition; : Label smoothing loss for scene classification; : Temporal smoothness constraint loss (penalizing semantic jumps between adjacent frames).

[0114] Dynamically adjust multi-task weights:

[0115] Initial weight: α=1.0, β=0.8, γ=0.5, λ=0.2;

[0116] Increase λ (time-smoothing loss weight) linearly as training progresses, and finally λ=0.6.

[0117] This invention achieves progressive optimization from general capabilities to scenario-specific capabilities. The two-stage strategy has clear division of labor: pre-training, which builds the model's initial robustness to noise to avoid overfitting to specific transmission conditions; and fine-tuning, which achieves precise optimization for the extremely low-bandwidth and highly dynamic nature of ship-to-shore communications.

[0118] This design ensures that the model has both semantic extraction capabilities for general scenarios and deep adaptation to the transmission requirements of ship-to-shore communications. It is the core basis for the "training-transmission" full-link protection in this invention.

[0119] At the same time, in order to adapt to the real-time constraints of ship-shore communication, it is necessary to preprocess the video data and extract RGB images and optical flow information from the original video data.

[0120] 1. Video framing and decoding

[0121] Through hardware-accelerated decoding, such as NVDEC, the video stream data is decoded into an RGB frame sequence, each maintaining the original 1080p. Then, key frames are extracted according to the preset sampling rate (such as 5fps) to reduce the subsequent processing load.

[0122] 2. Optical flow calculation and optimization

[0123] Based on the continuous RGB frame sequence obtained above, the TV-L1 optical flow algorithm (implemented in OpenCV) is used to balance accuracy and computational efficiency. The RGB frames are downsampled to the model input size (such as 384×384) to reduce the amount of optical flow calculation. The optical flow calculation is accelerated by CUDA (such as using the OpticalFlow API) to achieve real-time processing.

[0124] 3. Data format standardization

[0125] Normalize the RGB image and optical flow respectively: normalize the pixel value from [0,255] to [0,1]; normalize the optical flow

[0126] The displacement values ​​are normalized to [-1, 1] to adapt to the model input range.

[0127] See also Figure 2 , which shows a flow chart of the steps of a dynamic feature selection module provided by an embodiment of the present invention, such as Figure 2 As shown, the dynamic feature selection module provided in the embodiment of the present application includes three steps, namely:

[0128] 1. Bandwidth monitoring and classification: Real-time monitoring of ship-to-shore communication bandwidth, and classification of bandwidth into three levels: high, medium, and low based on predefined thresholds: high bandwidth: ≥100kbps; medium bandwidth: 10kbps~100kbps; low bandwidth: <10kbps.

[0129] 2. Dynamic configuration of task branches: High bandwidth: Enable all task branches (target detection, action recognition, and scene encoding); Medium bandwidth: Disable the scene encoding branch, retaining the target detection and action recognition branches; Low bandwidth: Enable only the target detection branch and reduce the detection granularity (increase the confidence threshold from 0.5 to 0.8).

[0130] 3. Adaptive adjustment mechanism: Based on a lightweight decision tree model, it predicts bandwidth fluctuation trends and adjusts task branch configurations in advance. If a key target is lost (such as a person falling into the water) for three consecutive frames, the motion recognition branch is forcibly enabled and high-priority transmission is triggered.

[0131] Through the implementation of the above-mentioned dynamic feature selection module, non-critical branches can be dynamically closed, balancing the amount of semantic information and transmission efficiency under bandwidth fluctuations; at the same time, critical branches are forcibly enabled and redundantly transmitted to ensure reliable handling of emergency events such as personnel safety.

[0132] See also Figure 3 , which shows a flowchart of the steps of a video data transmission method provided by an embodiment of the present invention, such as Figure 3 As shown, the video data transmission method provided in the embodiment of the present application includes five steps, namely:

[0133] Extract video semantic information: Use semantic analysis models to perform semantic analysis on video data, extract key information, and

[0134] Generate structured semantic descriptions, where key information includes: objects, actions, and scenes;

[0135] Semantic information compression: Encode and compress semantic descriptions to reduce data transmission volume;

[0136] Data transmission: The compressed semantic information is sent to the shore-based end according to the transmission protocol;

[0137] Semantic information restoration: The shore-based end decodes the semantic information and uses the generation module to restore it to video frames;

[0138] Video reconstruction: Combine the restored video frames in chronological order to generate complete video data.

[0139] It should be noted that the transmission of the aforementioned video data requires the prior deployment of hardware and software in a ship-to-shore application scenario. The trained semantic analysis model is deployed on the ship's edge device, while video restoration is performed on shore, generating video frames based on the GAN+Diffusion model.

[0140] 1. Ship-side deployment and semantic extraction:

[0141] Step 1.1 Ship-side edge device configuration

[0142] Hardware platform: The ship uses the NVIDIA Jetson AGX Xavier edge computing device with 32GB of storage and 8GB of memory.

[0143] Model deployment: The trained semantic analysis model (MobileViT-S architecture, 25MB in size) was deployed to the device using the TensorRT acceleration engine. The input resolution was set to 384×384, supporting real-time processing (≤35ms / frame).

[0144] Step 1.2 Video semantic extraction

[0145] Input video stream: The ship's onboard camera captures 1080P@30fps video stream.

[0146] To meet the real-time constraints of ship-to-shore communication, video data preprocessing is required to extract RGB images and optical flow information from the raw video data. This includes video framing and decoding, optical flow calculation and optimization, and data format standardization. These three steps have been described previously and will not be repeated here.

[0147] Semantic analysis process:

[0148] Object detection: Identify objects such as ships, people, and lifeboats, and output bounding box coordinates (normalized to [0,1]) and confidence levels.

[0149] Action recognition: Determines the action category (e.g., "ship accelerating," "person waving") based on five consecutive frames of optical flow information.

[0150] Scene coding: classification of wave level (1-5), visibility (low / medium / high).

[0151] Structured output: Generates semantic description in JSON format

[0152] {

[0153] "timestamp": 1678901234,

[0154] "objects": [

[0155] {"class": "ship", "bbox": [0.2, 0.5, 0.8, 0.7], "confidence": 0.92},

[0156] {"class": "person", "bbox": [0.4, 0.3, 0.5, 0.6], "confidence": 0.88}

[0157] ],

[0158] "actions": ["ship_accelerating"],

[0159] "scene": {"wave_level": 3, "visibility": "low"}

[0160] }

[0161] 2. Semantic information compression and transmission:

[0162] Step 2.1 Semantic Compression

[0163] Encoding protocol: Convert JSON data to Protobuf binary format. The fields are defined as follows:

[0164] message SemanticData {

[0165] uint64 timestamp = 1;

[0166] repeated Object objects = 2;

[0167] repeated string actions = 3;

[0168] Scene scene = 4;

[0169] }

[0170] Compression algorithm:

[0171] Level 1 semantics (target information): Huffman coding (pre-trained dictionary based on high-frequency words in ship-shore scenes);

[0172] Secondary semantics (scene information): LZ77 algorithm compression, compression rate ≥ 60%.

[0173] Step 2.2 Data Transfer

[0174] Transmission protocol: Based on the MQTT protocol, supports low-bandwidth adaptation.

[0175] Block transmission: Split the Protobuf data into 1KB data blocks, and append a CRC check code to each block.

[0176] Priority scheduling: Critical semantics (such as "man overboard") are marked as high priority and sent first.

[0177] Bandwidth adaptation:

[0178] Bandwidth > 100kbps: Transmits complete semantics (target + action + scene); Bandwidth 10-100kbps: Transmits only targets and actions; Bandwidth < 10kbps: Transmits only high-confidence targets (confidence ≥ 0.8).

[0179] 3. Shore-based video restoration and reconstruction:

[0180] Step 3.1 Semantic Decoding

[0181] Data verification: Restore the complete Protobuf data through CRC verification and decode it into a structured semantic description (including target, action, and scene tags).

[0182] Error handling: If the packet loss rate is >5%, a retransmission request is triggered.

[0183] Step 3.2 Video frame restoration

[0184] The core of this step is to restore the received structured semantic information (such as object, action, and scene labels) into high-quality video frames through a hybrid architecture of a generative adversarial network (GAN) and a diffusion model. In particular, action information is explicitly integrated into the generation process in this step to ensure cross-frame motion coherence and physical plausibility. The following are the specific implementation details:

[0185] Step 3.2.1 Generate model architecture design

[0186] Overall architecture: A hybrid design of StyleGAN2+Latent Diffusion Model (LDM).

[0187] GAN branch (target generation): Input: target category, location and action label;

[0188] Action condition injection:

[0189] class ActionEncoder(nn.Module):

[0190] def __init__(self):

[0191] super().__init__()

[0192] self.embed = nn.Embedding(num_actions, 256)

[0193] self.fc = nn.Linear(256, 512)

[0194] def forward(self, action_label):

[0195] action_embed = self.embed(action_label)

[0196] return self.fc(action_embed)

[0197] Conditional encoding: Map the target semantics (category, position) to a latent vector (Latent code). The code is as follows:

[0198] class ObjectEncoder(nn.Module):

[0199] def forward(self, class_label, bbox):

[0200] class_embed = nn.Embedding(num_classes, 512)(class_label)

[0201] pos_embed = fourier_embedding(bbox)

[0202] latent = torch.cat([class_embed, pos_embed], dim=1)

[0203] return latent

[0204] StyleGAN2 generator input: basic latent vector z is sampled from normal distribution; semantic latent vector c modulates the style parameters of the generator through the fully connected layer; fusion of basic latent vector z, semantic latent vector c (category + position), action latent vector Among them, the basic latent vector z is a random noise vector used to control the diversity of generated content (such as target posture and texture details); the semantic latent vector c is a conditional vector obtained by encoding the target semantics (category, position), which is used to control the semantic attributes of the generated content (such as target category, position, size); the action latent vector , is the conditional vector obtained by encoding the action label, which is used to explicitly control the action attributes of the generated content (such as ship acceleration and people waving).

[0205] Diffusion branch (background generation): Based on LDM, progressive denoising generates dynamic background (such as waves, sky),

[0206] Supports fine-grained texture control.

[0207] Input: decoded semantic information (target category, location, action label, scene description);

[0208] Output: Single-frame image (resolution 1080p), frame rate 30fps (increased to 90fps through interpolation).

[0209] Generation process:

[0210] 1. Generate the target initial image (256×256) based on the latent vector;

[0211] 2. Gradually increase the resolution to 1024×1024 through the upsampling module;

[0212] 3. Apply Adaptive Instance Normalization (AdaIN) to incorporate scene lighting conditions (such as fog concentration).

[0213] Spatial Constraint: The generated object is precisely cropped and fit into the specified area of ​​the background according to the bounding box coordinates in the semantics.

[0214] The specific details of the implementation are:

[0215] Scene encoding: The scene label (e.g., “wave level 3,” “low visibility”) is input into the CLIP text encoder to generate a scene description vector.

[0216] Diffusion model input: Initial noisy image (resolution 1024×1024); scene description vector guided by the crisscross attention mechanism. Denoising step: 1000 steps of DDPM (denoising Diffusion Probabilistic Models) are used to gradually refine background details.

[0217] Key optimizations: Dynamic noise scheduling, which adaptively adjusts noise intensity based on scene complexity (for example, increasing noise diversity in high-wave areas and enhancing wake detail when a ship accelerates); and Local Refinement, which performs an additional 50 steps of local denoising around target areas (such as wake spray at the stern) to enhance realism.

[0218] Step 3.2.2 Optical flow guided temporal generation

[0219] The optical flow field of future frames is predicted based on the action label to guide the target displacement.

[0220] The optical flow constraint loss is:

[0221]

[0222] Step 3.2.3 Physical Constraint Modeling

[0223] Apply physical constraints, such as a maximum speed limit, to the resulting target displacement.

[0224] def apply_speed_limit(prev_bbox, curr_bbox, max_speed):

[0225] dx = curr_bbox.x - prev_bbox.x

[0226] dy = curr_bbox.y - prev_bbox.y

[0227] speed = np.sqrt(dx××2 + dy××2)

[0228] if speed>max_speed:

[0229] dx × = max_speed / speed

[0230] dy × = max_speed / speed

[0231] return dx, dy

[0232] Step 3.2.4 Target-background fusion and temporal consistency optimization

[0233] Step 3.2.4.1 Optical flow guided interpolation

[0234] Input: two consecutive frames to generate images and its optical flow field .

[0235] Interpolation method, the code is as follows:

[0236] def interpolate(I0, I1, F, alpha=0.5):

[0237] F0_1 = alpha × F

[0238] F1_0 = -(1 - alpha) × F

[0239] I0_warp = warp(I0, F0_1)

[0240] I1_warp = warp(I1, F1_0)

[0241] return (I0_warp + I1_warp) / 2

[0242] Output: Insert intermediate frames (such as ), increasing the frame rate from 30fps to 90fps.

[0243] Step 3.2.4.2 Time Domain Filtering

[0244] Kalman filter: smooth the target position coordinates to eliminate the generated jitter;

[0245] Motion consistency loss: constrains the motion acceleration of objects in adjacent frames during the generation process.

[0246] Step 3.3 Video reconstruction

[0247] The goal of this step is to combine the discrete video frames restored by the shore-based end into a continuous video stream in chronological order to ensure temporal integrity. It includes the following three steps:

[0248] Step 3.3.1 Frame Buffer and Sorting

[0249] Receive shore-based restored video frames (single-frame resolution is 1080p), each frame is accompanied by a timestamp (parsed from semantic information);

[0250] Frames are sorted according to timestamps. If out-of-order frames are detected (such as caused by network jitter), a reordering mechanism is triggered.

[0251] Step 3.3.2 Frame rate synchronization

[0252] Based on the preset output frame rate (e.g. 30fps), calculate the theoretical duration of each frame (33.3ms / frame);

[0253] If the time interval between adjacent frames exceeds a threshold (such as 50ms), duplicate frames or interpolated frames are inserted to fill the gap.

[0254] Step 3.3.3 Generate video stream

[0255] Use FFmpeg or GStreamer libraries to encode ordered frame sequences into standard video formats (such as MP4, HLS).

[0256] Step 3.4 Video post-processing

[0257] The goal of this step is to improve the smoothness and video quality of the generated video stream and verify that it meets the application standards.

[0258] Timing consistency optimization:

[0259] Use optical flow-guided interpolation to insert intermediate frames into low-frame-rate clips, increasing the frame rate to 90fps;

[0260] Apply time domain filtering (such as Kalman filtering) to smooth the target position coordinates and eliminate generated jitter;

[0261] Quality evaluation: Objective metrics (peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and perceptual similarity (LPIPS)) are used to evaluate the quality of the generated videos.

[0262] Adversarial post-processing: Apply the super-resolution model ESRGAN to enhance details in low-quality areas (such as blurred targets); use the pre-trained PatchGAN discriminator for adversarial fine-tuning to improve the realism of the generated images.

[0263] Step 3.5 Real-time guarantee technology

[0264] Step 3.5.1 Distributed Generation Pipeline

[0265] GPU cluster division of labor: Node1, responsible for target generation (GAN); Node2, responsible for background generation (Diffusion); Node3, responsible for fusion and interpolation.

[0266] Step 3.5.2 Caching and Pre-generation

[0267] Predictive rendering: Generates a 3-frame buffer in advance based on the semantics of future frames predicted by LSTM;

[0268] Dynamic load balancing: Dynamically distribute generation tasks based on GPU load (e.g. pre-generating non-critical backgrounds when load is low).

[0269] The ship-to-shore video data transmission method provided by the present invention achieves the following technical effects: 1. Hybrid generation architecture: Through the hybrid generation architecture GAN+Diffusion, it focuses only on high-resolution target generation while also refining the background, taking into account both efficiency and quality; optical flow interpolation and Kelman filtering ensure temporal smoothness. 2. Semantic-generation deep coupling: Semantic labels directly modulate generation style parameters to achieve precise and controllable video synthesis; scene description vectors guide diffusion model denoising to enhance environmental realism. 3. Real-time optimization: Distributed pipeline + predictive rendering breaks through the computing power bottleneck of a single device; GPU cluster dynamic load ensures 90fps high frame rate output.

[0270] In the above embodiments, all or part of the embodiments can be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, hard disk, tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive).

[0271] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A semantic analysis model training method for ship-to-shore video data transmission, characterized in that: The method comprises the following steps: Step 1: Construct a semantic analysis model, which includes: Input module, which receives RGB images and optical flow information and splices them into 5 channels for input; The backbone network uses a lightweight Transformer and convolutional hybrid architecture MobileViT-S, and embeds a spatiotemporal attention mechanism; The multi-task fusion branch includes the target detection branch, the action recognition branch, and the scene coding branch, which output target category and location, action label, and scene label respectively; Dynamic feature selection module, dynamically shutting down non-critical task branches based on the current bandwidth status; Step 2: Train the semantic analysis model: A two-stage training strategy is adopted, including: Phase 1: Pre-training on a general video dataset using the AdamW optimizer and Focal Loss; Phase 2: Fine-tuning on ship-shore scenario data, using a curriculum learning strategy to gradually increase the weight of complex samples and adding a time series smoothing loss; specifically, this includes: Collect real ship-shore surveillance videos and annotate them. The annotation information includes target categories, action labels, and scene labels. Simulate sea fog, wave vibration, and salt spray noise for data enhancement; Set a course learning strategy, including using only simple scene samples from epochs 1-50; gradually adding complex scene samples from epochs 51-100, increasing the proportion from 20% to 80%; and training on all complex scenes from epochs 101-150, adding adversarial examples; Design a multi-task joint loss function and introduce temporal smoothing constraint loss. The joint loss function is: ,in : Focal Loss for target detection; : Cross entropy loss for action recognition; : Label smoothing loss for scene classification; : Temporal smoothing constraint loss; Dynamically adjust the weights of multi-task branches: the initial weights are α=1.0, β=0.8, γ=0.5, and λ=0.2, where α represents the weight corresponding to object detection, β represents the weight corresponding to action recognition, γ represents the weight corresponding to scene classification, and λ represents the weight of the temporal smoothing constraint loss. λ is linearly increased as training progresses, and finally reaches λ=0.

6. The MobileViT-S consists of six stages, Stage 1 to Stage 6, consisting of alternating stacks of convolutional layers and Transformer modules. The model types of Stage 1, Stage 3, and Stage 5 are Conv2D+BN+Swish, the model types of Stage 2 and Stage 4 are MobileViT Block, and the model type of Stage 6 is Global Context Transformer. The spatiotemporal attention mechanism introduces 3D spatiotemporal position coding in the Transformer layer to capture the temporal dependencies between video frames. The 3D spatiotemporal position coding consists of two parts: 2D spatial coding and 1D temporal coding. The specific design is to embed local window spatial attention in Stage 2 and Stage 4, and global temporal attention in Stage 6.

2. The semantic analysis model training method according to claim 1, characterized in that: The specific structure of the MobileViT Block is: the first layer is CNN, the second layer is Unfold, the third layer is Transformer, the fourth layer is Fold, and the fifth layer is CNN. It fuses local-global features, divides the image into local blocks through the Unfold operation, inputs them into the Transformer to capture the global relationship, and then restores the spatial structure through Fold. At the same time, some fully connected layers are replaced by depthwise separable convolution to reduce the number of parameters.

3. The semantic analysis model training method according to claim 1, characterized in that: The implementation of the dynamic feature selection module includes the following steps: Monitor bandwidth status and classify it: monitor the communication bandwidth between ship and shore in real time and classify it into High, medium and low gears; Dynamically configure task branches: When the bandwidth is high, all task branches are enabled; when the bandwidth is medium, the scene encoding branch is closed. Keep the object detection and action recognition branches; when the bandwidth is low, only enable the object detection branch and reduce the detection granularity; Introducing an adaptive adjustment mechanism: Based on a lightweight decision tree model, it predicts bandwidth fluctuation trends and adjusts task branches in advance. Configuration: If the key target is lost for three consecutive frames, the action recognition branch is forced to start and the high optimization level transmission is triggered.

4. The semantic analysis model training method according to claim 1, characterized in that: Before training the semantic analysis model, the video data is preprocessed to extract RGB images and optical flow information from the video data.

5. The semantic analysis model training method according to claim 1, characterized in that: The first stage specifically includes: Configure a general dataset, using the public video dataset Kinetics-400, covering daily actions and complex scenes; Data augmentation using random cropping, color jittering, and motion blur simulation; Set the initial learning rate of the optimizer AdamW to , weight decay is 0.05, batch size is 32, and input resolution is 256×256; set the training objectives to target object detection mAP@0.5 ≥ 75%, action recognition Top-1 accuracy ≥ 82%, and scene classification F1-score ≥ 0.

88.

6. A video data transmission method, applied to ship-to-shore data communication, characterized in that: include: Deploying a semantic analysis model trained by the semantic analysis model training method according to any one of claims 1 to 5 on a ship-side edge device; On the shore side, video frames are generated based on the GAN+Diffusion model and video restoration operations are performed.

7. The video data transmission method according to claim 6, wherein: The steps include: Extracting video semantic information: Using the semantic analysis model to perform semantic analysis on video data, extract key information, and generate a structured semantic description, where the key information includes objects, actions, and scenes; Semantic information compression: Encode and compress semantic descriptions to reduce data transmission volume; Data transmission: The compressed semantic information is sent to the shore-based end according to the transmission protocol; Semantic information restoration: The shore-based end decodes the semantic information and uses the generation module to restore it to video frames; Video reconstruction: Combine the restored video frames in chronological order to generate complete video data; Video post-processing: Use optical flow to guide interpolation to insert intermediate frames into low-frequency clips to improve the frame rate; apply time domain filtering to The target position is smoothed; the quality of the generated video is evaluated using objective indicators.

8. An electronic device, characterized in that: It includes one or more processors; a storage device for storing one or more computer programs, when the one or more computer programs are executed by the one or more processors, the electronic device implements the semantic analysis model training method according to any one of claims 1 to 5 or the video data transmission method according to any one of claims 6 to 7.

9. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor of an electronic device, the electronic device executes the semantic analysis model training method as described in any one of claims 1-5 or the video data transmission method as described in any one of claims 6-7.

Images