A video sentiment analysis method based on timing mask distillation

Abstract

This invention relates to the field of video processing technology, and more particularly to a video sentiment analysis method based on temporal mask distillation. The method includes: manually creating a sentiment dictionary to acquire video sentiment data; extracting audio data and visual images; mapping the audio data and visual images to a joint coding space using a multimodal encoder; identifying sentiments from the joint coding space using a sentiment classifier; calculating sentiment scores in the temporal dimension based on the sentiment data; analyzing the video sentiment data to obtain sentiment categories, sentiment levels, and sentiment intensity; designing a temporal sentiment complementarity learning scheme; and constructing a mask reconstruction task to predict video sentiment. This invention proposes a novel mask sentiment modeling scheme that utilizes the temporal sentiment features between multimodalities and uses the temporal dimension sentiment score as the target to learn and identify sentiment representations, thereby improving the performance of existing video sentiment analysis methods.

Description

Technical Field

[0001] This invention relates to the field of video processing technology, and in particular to a video sentiment analysis method based on temporal mask distillation. Background Technology

[0002] Video sentiment analysis can be used to analyze and identify the emotional state of people or objects in videos, and to classify emotional states and expressions. This technology has a wide range of applications, particularly in areas such as mental health protection (e.g., video violence detection), online education toolkits (e.g., automated analysis of student attitudes), and business analytics (e.g., consumer sentiment analysis).

[0003] State-of-the-art video sentiment analysis methods leverage deep learning to learn sentiment representations and train networks in a fully supervised manner. A critical drawback of this approach is the need for large-scale labeled datasets, yet labeling videos with sentiment is both time-consuming and extremely expensive. Especially considering the subjective nature of perceived sentiment, a single video requires multiple annotations. Masked autoencoders (MAEs) are widely used due to their excellent performance in uncovering correlations between local patches. MAEs recover low-level representations of embedded information in masked videos, such as pixels, contours, or trajectories. However, unlike common content such as objects and actions presented in videos, sentiment is highly abstract due to its reliance on human cognition. Therefore, a sentiment gap exists between low-level information and high-level sentiment semantics. Existing autoencoder-based video analysis methods exhibit performance degradation on sentiment modeling tasks. Summary of the Invention

[0004] This invention aims to at least solve one of the technical problems existing in related technologies. To this end, this invention provides a video sentiment analysis method based on temporal mask distillation.

[0005] This invention provides a video sentiment analysis method based on temporal mask distillation, comprising the following steps: S1: Create an emotional dictionary; S2: Acquire the video data to be analyzed, extract audio data and visual images, map the audio data and visual images to the joint coding space through a multimodal encoder, identify emotions from the joint coding space through an emotion classifier, and calculate the emotion score in the time dimension based on the emotions. S3: Obtain the text information of the audio data by transcribing it; S4: Parse the obtained text information, and obtain the emotion category, sentiment level, and sentiment intensity of each text message by querying the sentiment dictionary; by verifying and filtering the emotion category, sentiment level, and sentiment intensity, remove the parts of the text information that are not related to the emotion. S5: Guided by the sentiment category, emotion category, and emotion level of the text information, relevant visual and audio features are extracted from the video data. Visual features are input into a multi-head attention layer to obtain dictionary-aware visual linguistic features; audio features are input into a multi-head attention layer to obtain dictionary-aware acoustic linguistic features; and visual and acoustic linguistic features are mapped to the same semantic space and aligned using a modality-aware mapping scheme to obtain temporal sentiment features. S6: Construct a mask reconstruction task based on temporal sentiment features, aiming at the sentiment score in the spatiotemporal dimension, learn to identify sentiment representations, and obtain predicted video sentiment.

[0006] The video sentiment analysis method based on temporal mask distillation provided by the present invention further includes, in step S2, the following:

[0007] S21: The visual image and audio data of the video are mapped to the joint coding space by a multimodal encoder to obtain the frame-by-frame features of the video. The expression for the frame-by-frame features of the video is as follows: In the formula, For frame-by-frame features of the video, It is a multi-mode encoder. For visual images, For audio data, For the set of real numbers, The length of the video. The dimension of the feature. For a joint coding space set;

[0008] S22: Identify emotions using an emotion classifier; the emotion expression is: In the formula, For emotions, It is a fully connected layer. The softmax activation function is used. Global video features generated by performing max pooling operations in the time dimension;

[0009] S23: Sentiment score is calculated based on the spatiotemporal dimension of the video. The sentiment score expression is: In the formula, Let T be a vector repeated T times, and · be the Hadamard product. For video emotion collection, For emotional data.

[0010] According to the present invention, a video sentiment analysis method based on temporal mask distillation further includes, in step S3, transcribing audio data to obtain text information of the audio data, wherein the expression of the text information is: In the formula, The text information is for audio data. For the audio data One word, The number of linguistic features.

[0011] The video sentiment analysis method based on temporal mask distillation provided by the present invention further includes, in step S4, the following: S41: Analyze the text information in the video, search for corresponding sentiment words through the sentiment dictionary, extract language vectors and sentiment vectors, and obtain the sentiment category, sentiment level, and sentiment intensity; S42: Calculate the matching score between sentiment words and corresponding video content usage.

[0012] By calculating the emotion category and the emotion category conveyed by the video, positive and negative emotions are obtained. The calculation expression for matching the emotion category and the emotion category conveyed by the video is as follows: In the formula, According to the emotion category, The category of emotion conveyed by the video. Functions with 0s and 1s;

[0013] The emotion expressions that users can recognize are: In the formula, For fine-grained sentiment categories, For the fine-grained emotional categories conveyed by the video;

[0014] The consistency expression between the fine-grained sentiment score and the fine-grained sentiment score conveyed by the video is as follows: In the formula, For fine-grained sentiment scoring, The video is scored based on the fine-grained emotions it conveys. The threshold value is used.

[0015] The video sentiment analysis method based on temporal mask distillation provided by the present invention further includes step S5, which includes: S51: Extract visual and audio features.

[0016] The visual modality, by inputting the extracted visual features into a multi-head attention layer, retrieves relevant content from the visual feature set to obtain dictionary-aware visual linguistic features. The expression for calculating dictionary-aware visual linguistic features is as follows: In the formula, Visual features The number of linguistic features, This refers to the visual linguistic features of dictionary perception. The number of heads in the multi-head attention layer. For the first Features of the attention head output For learnable matrices, For the first The attention matrix output by each attention head. For visual feature set, , For visual images, Visual features truncated by a time window. For the learnable matrix of values ​​in the attention mechanism, For the learnable matrix of the query in the attention mechanism, For the learnable matrix of keys in the attention mechanism, As the dimension of the feature, This is a matrix transpose operation;

[0017] Acoustic modality is obtained by inputting the extracted audio features into a multi-head attention layer, retrieving relevant content from the acoustic feature set, and obtaining dictionary-aware acoustic linguistic features. The dictionary-aware acoustic linguistic features are calculated using the following expression: In the formula: For audio data, Acoustic linguistic features of dictionary perception. Acoustic features truncated by a time window. For acoustic feature set, ; S52: By aligning the obtained dictionary-aware visual linguistic features and dictionary-aware acoustic linguistic features, corresponding regions are found in multiple modalities to obtain attention maps; visual linguistic features and acoustic linguistic features are mapped to the same semantic space through a modality-aware mapping scheme to obtain mapped visual features and mapped audio features; the mapped visual features and mapped audio features are pushed to the emotional level language embedding to obtain the associated complementary parts; by extracting features between visual features and audio features, the inherent parts of the modality are preserved.

[0018] The visual contrast loss due to emotional complementarity and intrinsic learning is: In the formula, For visual contrast loss, For the mean, For from Time window, To compare distances, For the mapped visual features, As a linguistic feature, For a video's time index, The mapped audio features, For the time index of another video, for Not here Inside, the contrast distance of temperature τ ; For distance features, For distance features, · represents the Hadamard product;

[0019] The audio contrast loss for emotional complementarity and intrinsic learning is:

[0020] The total loss is: In the formula, For visual contrast loss, For audio contrast loss, This represents the total loss.

[0021] The video sentiment analysis method based on temporal mask distillation provided by the present invention further includes step S6, which comprises:

[0022] S61: Based on the cross-modal attention map between language tags and video content tags, obtain the emotion attention map, expressed as: In the formula, For emotional attention diagrams, The number of heads in the multi-head attention layer. For the first The attention matrix output by each attention head. For real numbers, The number of linguistic features, The resolution of the corresponding modality, i.e., the visual features are The audio features are Emotional Attention Map Indicates and The likelihood of emotional cues corresponding to locational features;

[0023] S62: Sort the attention map in descending order to obtain the corresponding index of the label. Apply high-attention labeling to the mask to encourage the model to uncover and recover high-level sentiment cues. Utilize the label with the highest attention score to provide recovery cues. Mask the labeling in a progressive manner. The expression for the masking strategy is: In the formula, Decide whether to [do something] Each marker is used to mask it. High scores exceeding the threshold are marked. To obtain the first The corresponding index of each tag, To determine the ratio of training time to total training time, Low scores that do not exceed the threshold are marked. for For other cases; S63: Reconstruct the temporal sentiment distribution based on the time score, and model the sentiment distribution of each spatiotemporal segment with the sentiment score in the time dimension as the target.

[0024] The video sentiment analysis method based on temporal mask distillation provided by the present invention further includes, in step S4, extracting sentiment information by querying the attributes of sentiment words in the sentiment dictionary.

[0025] The video sentiment analysis method based on temporal mask distillation provided by the present invention further includes, in step S5, uniformly truncating the time window by the number of blocks or the time dimension of the feature map.

[0026] The above-described one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects: This invention proposes a novel masked sentiment modeling scheme that utilizes temporal sentiment features among multiple modalities and aims to learn and identify sentiment representations based on temporal dimension sentiment scores, thereby improving the performance of existing video sentiment analysis methods.

[0027] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0029] Figure 1 This is a flowchart illustrating a video sentiment analysis method based on temporal mask distillation provided by the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. The following embodiments are used to illustrate this invention but should not be used to limit the scope of this invention.

[0031] like Figure 1 As shown, a video sentiment analysis method based on temporal mask distillation includes the following steps: S1. Create emotional dictionary features;

[0032] S2. Obtain video sentiment data: The video data to be analyzed is acquired, and the audio data and visual images are extracted separately. The audio data and visual images are mapped to a joint coding space using a multimodal encoder. The sentiment classifier is used to identify sentiment data from the joint coding space and calculate the sentiment score in the time dimension.

[0033] S21: In this embodiment, For a video clip, the video visual features and audio features Using a multimodal encoder common to video sentiment analysis Mapping to the joint coding space, we obtain video features. The frame-by-frame feature expression for the video is as follows: In the formula, For frame-by-frame features of the video, It is a multi-mode encoder. For visual images, For audio data, For the set of real numbers, The length of the video. It is the dimension of the feature. For a joint coding space set;

[0034] S22: From the sentiment classifier Identify emotions within each category; the emotion expression is: In the formula, For emotions, It is a fully connected layer. The softmax activation function is used. Global video features generated by performing max pooling operation in the time dimension;

[0035] S23: Calculate the spatiotemporal sentiment score of the video. Using the definition of class activation graphs, the calculation expression is as follows: In the formula, Let T be a vector repeated T times, and · be the Hadamard product. For video emotion collection, The emotion number is represented by the class activation graph, which shows the score distribution across the spatiotemporal dimensions for a specific category.

[0036] S3: Obtain the text information of the audio data by transcribing it; the expression for the text information is: In the formula, The text information is for audio data. For the audio data One word, The number of linguistic features.

[0037] S4: Parse the text information of the obtained audio data and search for corresponding sentiment words using common sentiment dictionaries. Then for each word Extracting language vectors and emotional vector Thus, the emotion category is obtained. Emotional Category and each level of emotion . By verifying and filtering text information, removing parts irrelevant to emotion, the emotion can be obtained.

[0038] In this embodiment, the extraction of sentiment information does not use a large pre-trained model to extract semantic features, but rather extracts it by querying the attributes of sentiment words in the sentiment dictionary.

[0039] In this embodiment, an emotion word is given. and corresponding video content The purpose is to verify their use and matching scores.

[0040] In the first stage, focus on the text. and video The emotions conveyed, namely positive and negative emotions, constitute the initial matching at the emotional level. The calculation expression for matching the emotion category with the emotion category conveyed by the video is as follows: In the formula, According to the emotion category, The category of emotion conveyed by the video. Functions with 0s and 1s;

[0041] The emotions that users can recognize are: In the formula, For fine-grained sentiment categories, The fine-grained emotional categories conveyed by the video;

[0042] The consistency expression between the fine-grained sentiment score and the fine-grained sentiment score conveyed by the video is as follows: In the formula, For fine-grained sentiment scoring, The video is scored based on the fine-grained emotions it conveys. The threshold value is used.

[0043] S5: Design a temporal and emotional complementary learning scheme: Considering the complementary and modality-specific components among intermodal features, a temporal sentiment complementarity learning scheme was designed to align intermodal features. First, a text modality is used as the query to retrieve relevant features from both visual and audio features. Then, these features are mapped to the same semantic space for alignment.

[0044] S51: First, we need to extract the visual and audio features related to the text.

[0045] For visual modalities, Language features of each word The input has A multi-head attention layer (MHA) of a certain size is used as a query to obtain visual features from the set. Retrieve relevant content from the dictionary and obtain dictionary-aware visual features. The expression for calculating visual features in dictionary-aware computing is: In the formula, The linguistic features of the video The number of linguistic features, Visual features for dictionary perception. The number of heads in the multi-head attention layer. For the first Features of the attention head output For learnable matrices, For the first The attention matrix output by each attention head. For visual feature set, , For visual images, Visual features truncated by a time window. For the learnable matrix of values ​​in the attention mechanism, For the learnable matrix of the query in the attention mechanism, For the learnable matrix of keys in the attention mechanism, As the dimension of the feature, This is a matrix transpose operation;

[0046] For acoustic modes, Language features of each word The input has A multi-head attention layer (MHA) of a certain size is used as a query to obtain acoustic feature sets. Retrieve relevant content from the dictionary to obtain the dictionary-aware acoustic features. The expression for calculating the dictionary-aware acoustic features is as follows: In the formula, For audio data, Acoustic features for dictionary perception. Acoustic features truncated by a time window. Acoustic feature set, ; It is important to note that temporal resolution, i.e., the number of blocks or temporal dimension of the feature map, is determined by using a time window. To limit attention, the time window is uniformly truncated.

[0047] S52: To address the issue of inaccurate matching results estimated in the feature space, visual linguistics and acoustic linguistics are aligned to find corresponding regions across multiple modalities, obtaining attention maps. A modality-aware mapping scheme maps visual and acoustic features to the same semantic space, yielding mapped visual and audio features. These mapped features are then pushed into emotional-level language embeddings to obtain complementary associations. Features are extracted between visual and audio features, preserving the inherent aspects of the modality. Formally, a third... The contrasting losses between emotional complementarity and intrinsic learning within each window are: In the formula, For visual contrast loss, For the mean, For from Time window, To compare distances, For the mapped visual features, As a linguistic feature, For a video's time index, The mapped audio features, For the time index of another video, for Not here Internal temperature Contrast distance ; For distance features, Distance features;

[0048] The audio contrast loss for emotional complementarity and intrinsic learning is:

[0049] The total loss is: In the formula, For visual contrast loss, For audio contrast loss, For the total loss, · represents the Hadamard accumulation.

[0050] S6: Construct a mask reconstruction task based on temporal sentiment features, aiming at the sentiment score in the spatiotemporal dimension, learn to identify sentiment representations, and obtain predicted video sentiment.

[0051] S61: To leverage the power of sentiment semantics, using the attention graph generated in step S5, and employing a cross-modal attention graph between language tags and video content tags, the formula for calculating the sentiment attention graph is as follows: In the formula, This represents the resolution of the corresponding modality, i.e., the visual features. The audio features are Emotional Attention Map Indicates and The likelihood of emotional cues corresponding to locational features;

[0052] S62: To locate the sentiment patch, sort the attention map in descending order to obtain the... The corresponding index of each tag In the mask In the case of high-attention tags, the model is encouraged to mine and recover high-level sentiment cues. Due to the abstract nature of sentiment, recovering sentiment tags is challenging, and hiding too many tags can lead to crashes. Therefore, the model with the highest attention score is revealed. Labels, which provide clues for recovery, are masked in a progressive manner. That is, a fixed masking ratio is replaced by a metric calculated according to the training progress. In the early stages, a lower masking ratio is used to avoid crashes, while as training iterations increase, the masking ratio is increased to encourage the model to learn robust sentiment representations. Finally, the masking strategy can be expressed as: In the formula, Decide whether to [do something] Each marker is used to mask it. The highest attention score is marked. To obtain the first The corresponding index of each tag, It is the ratio of training time to total training time. Low scores that do not exceed the threshold are marked. for For other cases; S63: Reconstructing Temporal Sentiment Distribution Based on Time Segments. Inspired by Pranchik's sentiment theory (Plunchik Wheel), this project models the sentiment distribution for each spatiotemporal segment. Specifically, pooling functions are used to extract representations, and then, during training, the Euclidean distance between the model's predictions and ground reality is minimized. This reconstruction task helps the model avoid overfitting to segment predictions and improves the quality of extracted representations.

[0053] Video sentiment prediction. The masked autoencoder of existing video sentiment analysis methods is replaced with a pre-trained encoder and fine-tuned for improved performance.

[0054] The beneficial effects of this invention are: This invention proposes a novel masked sentiment modeling scheme, which utilizes the temporal sentiment features between multiple modalities and takes the sentiment score in the temporal dimension as the target to learn and identify sentiment representations, thereby improving the performance of existing video sentiment analysis methods.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A video sentiment analysis method based on temporal mask distillation, characterized in that, Includes the following steps: S1: Create an emotional dictionary; S2: Acquire the video data to be analyzed, extract audio data and visual images, map the audio data and visual images to the joint coding space through a multimodal encoder, identify emotions from the joint coding space through an emotion classifier, and calculate the emotion score in the time dimension based on the emotions. S3: Obtain the text information of the audio data by transcribing it; S4: Parse the obtained text information, and obtain the emotion category, sentiment level, and sentiment intensity of each text message by querying the sentiment dictionary; by verifying and filtering the emotion category, sentiment level, and sentiment intensity, remove the parts of the text information that are not related to the emotion. S5: Guided by the sentiment category, emotion category, and emotion level of the text information, relevant visual and audio features are extracted from the video data. Visual features are input into a multi-head attention layer to obtain dictionary-aware visual linguistic features; audio features are input into a multi-head attention layer to obtain dictionary-aware acoustic linguistic features; and visual and acoustic linguistic features are mapped to the same semantic space and aligned using a modality-aware mapping scheme to obtain temporal sentiment features. S6: Construct a mask reconstruction task based on temporal sentiment features, aiming at the sentiment score in the spatiotemporal dimension, learn to identify sentiment representations, and obtain predicted video sentiment.

2. The video sentiment analysis method based on temporal mask distillation according to claim 1, characterized in that, Step S2 includes: S21: By mapping the visual image and audio data of the video to the joint coding space through a multimodal encoder, the frame-by-frame features of the video are obtained. The expression for the frame-by-frame features of the video is as follows: In the formula, For frame-by-frame features of the video, It is a multi-mode encoder. For visual images, For audio data, For the set of real numbers, The length of the video. The dimension of the feature. For a joint coding space set; S22: Identify emotions using an emotion classifier; the emotion expression is: In the formula, For emotions, It is a fully connected layer. The softmax activation function is used. Global video features generated by performing max pooling operations in the time dimension; S23: Sentiment score is calculated based on the spatiotemporal dimension of the video. The sentiment score expression is: In the formula, Let T be a vector repeated T times, and · be the Hadamard product. For video emotion collection, For emotional data.

3. The video sentiment analysis method based on temporal mask distillation according to claim 1, characterized in that, In step S3, the audio data is transcribed to obtain the text information of the audio data. The expression for the text information is: In the formula, The text information is for audio data. For the audio data One word, The number of linguistic features.

4. The video sentiment analysis method based on temporal mask distillation according to claim 1, characterized in that, Step S4 includes: S41: Analyze the text information in the video, search for corresponding sentiment words through the sentiment dictionary, extract language vectors and sentiment vectors, and obtain the sentiment category, sentiment level, and sentiment intensity; S42: Calculate the matching score between sentiment words and corresponding video content usage. By calculating the emotion category and the emotion category conveyed by the video, positive and negative emotions are obtained. The calculation expression for matching the emotion category and the emotion category conveyed by the video is as follows: In the formula, According to the emotion category, The category of emotion conveyed by the video. Functions with 0s and 1s; The emotion expressions that users can recognize are: In the formula, For fine-grained sentiment categories, For the fine-grained emotional categories conveyed by the video; The consistency expression between the fine-grained sentiment score and the fine-grained sentiment score conveyed by the video is as follows: In the formula, For fine-grained sentiment scoring, The video is scored based on the fine-grained emotions it conveys. The threshold value is used.

5. The video sentiment analysis method based on temporal mask distillation according to claim 1, characterized in that, Step S5 includes: S51: Extract visual and audio features. The visual modality, by inputting the extracted visual features into a multi-head attention layer, retrieves relevant content from the visual feature set to obtain dictionary-aware visual linguistic features. The expression for calculating dictionary-aware visual linguistic features is as follows: In the formula, Visual features The number of linguistic features, This refers to the visual linguistic features of dictionary perception. The number of heads in the multi-head attention layer. For the first Features of the attention head output For learnable matrices, For the first The attention matrix output by each attention head. For visual feature set, , For visual images, Visual features truncated by a time window. For the learnable matrix of values ​​in the attention mechanism, For the learnable matrix of the query in the attention mechanism, For the learnable matrix of keys in the attention mechanism, As the dimension of the feature, This is a matrix transpose operation; Acoustic modality is obtained by inputting the extracted audio features into a multi-head attention layer, retrieving relevant content from the acoustic feature set, and obtaining dictionary-aware acoustic linguistic features. The dictionary-aware acoustic linguistic features are calculated using the following expression: In the formula: For audio data, Acoustic linguistic features of dictionary perception. Acoustic features truncated by a time window For acoustic feature set, ; S52: By aligning the obtained dictionary-aware visual linguistic features and dictionary-aware acoustic linguistic features, corresponding regions are found in multiple modalities to obtain attention maps; visual linguistic features and acoustic linguistic features are mapped to the same semantic space through a modality-aware mapping scheme to obtain mapped visual features and mapped audio features; the mapped visual features and mapped audio features are pushed to the emotional level language embedding to obtain the associated complementary parts; by extracting features between visual features and audio features, the inherent parts of the modality are preserved. The visual contrast loss due to emotional complementarity and intrinsic learning is: In the formula, For visual contrast loss, For the mean, For from Time window, To compare distances, For the mapped visual features, As a linguistic feature, For a video's time index, The mapped audio features, For the time index of another video, for Not here Internal temperature Contrast distance ; For distance features, For distance features, · represents the Hadamard product; The audio contrast loss for emotional complementarity and intrinsic learning is: The total loss is: In the formula, For visual contrast loss, For audio contrast loss, This represents the total loss.

6. The video sentiment analysis method based on temporal mask distillation according to claim 1, characterized in that, Step S6 includes: S61: Based on the cross-modal attention map between language tags and video content tags, obtain the emotion attention map, expressed as: In the formula, For emotional attention diagrams, The number of heads in the multi-head attention layer. For the first The attention matrix output by each attention head. For real numbers, The number of linguistic features, The resolution of the corresponding modality, i.e., the visual features are Audio characteristics are Emotional Attention Map Indicates and The likelihood of emotional cues corresponding to locational features; S62: Sort the attention map in descending order to obtain the corresponding index of the label. Apply high-attention labeling to the mask to encourage the model to uncover and recover high-level sentiment cues. Utilize the label with the highest attention score to provide recovery cues. Mask the labeling in a progressive manner. The expression for the masking strategy is: In the formula, Decide whether to [do something] Each marker is used to mask it. High scores exceeding the threshold are marked. To obtain the first The corresponding index of each tag, To determine the ratio of training time to total training time, Low scores that do not exceed the threshold are marked. for For other cases; S63: Reconstruct the temporal sentiment distribution based on the time score, and model the sentiment distribution of each spatiotemporal segment with the sentiment score in the time dimension as the target.

7. A video sentiment analysis method based on temporal mask distillation according to claim 4, characterized in that... In step S4, emotional information is extracted by querying the attributes of emotional words in the emotional dictionary.

8. A video sentiment analysis method based on temporal mask distillation according to claim 5, characterized in that... In step S5, the time window is uniformly truncated by the number of blocks in the feature map or the time dimension.

Images