A short video network public opinion sentiment classification method based on big data

Abstract

The application relates to natural language processing, in particular to a short video network public opinion sentiment classification method based on big data, which comprises the following steps: obtaining short text features, audio features and image features from video data; inputting the obtained features into an activation function after splicing; judging the probability of the output of the activation function and a set threshold value to determine the sentiment type of the video; when the audio features are obtained, the audio features are converted into long texts, a cutting threshold value N is set, the first N / 2 words and the last N / 2 words of the long texts are cut, the cut words are spliced together, and the audio features are extracted from the spliced words. The application can automatically classify the sentiment of short video network public opinion, help understand public emotions and guide public opinion direction.

Description

Technical Field

[0001] This invention relates to big data technology, deep learning, natural language processing, computer vision technology, and multimodal fields, and in particular to a method for classifying online public opinion sentiment based on big data in short videos. Background Technology

[0002] In social networks, sentiment analysis can monitor users' emotional tendencies in real time under trending topics, understanding whether the public's overall attitude towards a particular event, product, or brand is positive support, negative opposition, or neutrality. For example, after a brand launches a new product, by analyzing user comments and discussions on social media platforms, the brand can quickly understand consumers' emotional feedback on the product and adjust its marketing strategies or product improvement directions accordingly. Furthermore, sentiment analysis technology can promptly detect aggressive, insulting, or inflammatory remarks and behaviors, identifying potential cyberbullying, malicious defamation, and other harmful information. Once such content is detected, the platform can quickly take measures, such as blocking or deleting the relevant information, warning or punishing the publisher, protecting users' legitimate rights and interests, and maintaining a healthy online social environment.

[0003] Short video platforms generate massive amounts of data, encompassing user behavior, comments, likes, shares, and video content across multiple dimensions. Traditional data analysis methods often struggle to handle such large datasets, necessitating the use of big data technologies for storage, processing, and analysis. In short videos, besides text data, the video content itself is crucial for analysis. Videos consist of numerous frames, and computer vision technology extracts visual features by recognizing each frame. Combining this with information such as facial expressions, body movements, and scene changes helps analyze the emotions conveyed in the video. Common techniques include facial expression recognition, motion recognition, and scene analysis.

[0004] Existing technologies typically use single data for recognition, resulting in inaccurate results. If text and image data are used together for recognition, the amount of data is usually enormous, leading to excessive calculations and wasting computing resources. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention proposes a short video online public opinion sentiment classification method based on big data. This method obtains short text features, audio features, and image features from video data. The obtained features are concatenated together and then input into an activation function. The sentiment type of the video is determined by judging the probability of the output of the activation function relative to a set threshold. When obtaining audio features, the audio features are converted into long text, and a truncation threshold N is set to truncate the first N / 2 words and the last N / 2 words of the long text. The truncated words are concatenated together and the audio features are extracted from them.

[0006] Furthermore, the calculation process for the truncation threshold N includes:

[0007]

[0008] Where, min(·) represents finding the minimum value; abs(·) represents finding the absolute value; tanh(·) represents the hyperbolic tangent function; n is the number of sentences in the long text; arsinh(·) represents the inverse hyperbolic sine function; l i N represents the length of the i-th sentence; model This indicates the text length limit for embedding vector layers in the pre-trained model.

[0009] Furthermore, keyframes are extracted from the video as image samples, and image features are extracted from the image samples. When obtaining a keyframe, the similarity between the current frame and the previous keyframe is determined. If the similarity is less than a set threshold, the current frame is designated as a keyframe; otherwise, the next frame is evaluated.

[0010] Preferably, the calculation of the similarity between the current frame and the previous keyframe includes:

[0011]

[0012] Where Similarity represents the similarity between the current frame image x and the previous keyframe image y; Dict(x,y) represents the distance between the current frame image x and the previous keyframe image y in the original video sequence; SSIM(x,y) represents the structural similarity between the current frame image x and the previous keyframe image y; ln(·) represents the logarithm; M represents the number of frames per second in the current video; and MSE(x,y) represents the mean square error between the current frame image x and the previous keyframe image y.

[0013] This invention concatenates text features, audio features, and image features, then inputs them into an activation function to form a short video sentiment classification model. This multimodal processing yields several significant benefits:

[0014] 1. By integrating multiple information sources, the accuracy of emotion understanding can be improved. A short video may contain complex emotional expressions, and a single modality may not be enough to fully understand the subtle differences in emotions. Multimodal fusion can ensure that the model captures more emotional dimensions.

[0015] 2. Each modality has certain noise or incomplete features. Through multimodal fusion, the model can be supplemented and validated by information from other modalities, reducing the impact of noise and enhancing overall robustness. Cross-validation between different modalities helps to eliminate errors caused by modality specificity, thereby improving the understanding of complex emotions.

[0016] 3. Each modality has its own advantages in expressing emotions. By combining the features of the three modalities, the model can learn the synergistic effect between these modalities, thereby capturing more complex emotional expressions.

[0017] 4. Short videos often exhibit emotional fluctuations, especially in situations with clear emotional shifts. Through multimodal fusion, the model can better capture the dynamic changes in emotions.

[0018] 5. In some situations, sentiment classification in a single modality may be biased or contradictory. By fusing three modalities, the model can establish consistency among information from different modalities, thereby making better judgments. Some emotional expressions may not be easy to capture through a single modality, but by combining information from all modalities, the model can more completely understand the emotional background in the video. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating a short video online public opinion sentiment classification method based on big data, according to an embodiment of the present invention.

[0020] Figure 2 This is a schematic diagram of the deep learning short text sentiment classification box structure in a short video network public opinion sentiment classification method based on big data according to an embodiment of the present invention;

[0021] Figure 3 This is a schematic diagram of the deep learning long text sentiment classification box structure in a short video online public opinion sentiment classification method based on big data according to an embodiment of the present invention;

[0022] Figure 4 This is a schematic diagram of named entity recognition in the deep learning long text sentiment classification box structure in a short video network public opinion sentiment classification method based on big data according to an embodiment of the present invention;

[0023] Figure 5 This is a schematic diagram of the deep learning image set sentiment classification box structure in a short video network sentiment classification method based on big data according to an embodiment of the present invention;

[0024] Figure 6 This is a schematic diagram of the overall model sentiment classification box structure in a short video online public opinion sentiment classification method based on big data according to an embodiment of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] This invention proposes a short video online public opinion sentiment classification method based on big data. The method obtains short text features, audio features, and image features from video data, concatenates the obtained features, and inputs them into an activation function. The sentiment type of the video is determined by judging the probability of the output of the activation function relative to a set threshold. When obtaining audio features, the audio features are converted into long text, and a truncation threshold N is set to truncate the first N / 2 words and the last N / 2 words of the long text. The truncated words are concatenated together and the audio features are extracted from them.

[0027] This embodiment provides a method for classifying online public opinion sentiment based on big data in short videos, such as... Figure 1 This embodiment is specifically described according to steps S1 to S4.

[0028] S1. The comments and bullet screens of the obtained short videos are taken as short texts in the short videos. The preprocessed short texts are obtained by removing noise, text segmentation and removing common words. The preprocessed short texts are then used to obtain the short text features in the short videos through a deep learning short text sentiment classification model framework.

[0029] In this embodiment, the comments and bullet comments in the short video can be obtained through API, or the data can be crawled. This invention does not limit the source of the comments and bullet comments in the short video.

[0030] In an embodiment of the present invention, the preprocessed short text is used to obtain text features in the short video through a deep learning short text sentiment classification model framework, such as... Figure 2 As shown, the deep learning short text sentiment classification model framework includes:

[0031] S11. Transform the original sample into a token sequence according to the DeBERTa model vocabulary;

[0032] Specifically, the DeBERTa model (Decoding-enhanced BERT with DisentangledAttention) is an improved model based on the BERT architecture. It not only inherits the bidirectional encoder structure of BERT, but also enhances the decoding aspect. The decoupled attention mechanism of this model processes content and position information separately, thereby better modeling contextual dependencies.

[0033] S12. Input the token sequence into the embedding layer of the DeBERTa model to obtain the Token Embedding, and extract the [CLS] token embedding to achieve the aggregation of global information;

[0034] Among them, the decoupled attention mechanism based on the DeBERTa model and the enhanced masked language model (EMLM) enable the [CLS] token to better aggregate global information. In the sentiment classification task, the [CLS] token embedding can more accurately capture the sentiment tendency of the entire short text.

[0035] S13. The [CLS] token embedding is input into the Bidirectional Memory Network (Bi-MT layer) for context modeling, thereby capturing global and local context information. Then, it is input into the Global Max Pooling (GM-Pooling layer) to extract the most representative text features. Finally, it is input into the All-to-All Connected Layer (AtA-CL layer) to obtain the final short text features.

[0036] In this process, inputting the [CLS] token embedding into the bidirectional memory network can further capture global and local contextual information, enhancing the ability to model emotional expression. At the same time, combining global max pooling can extract the most representative emotional features and avoid interference from irrelevant noise. Finally, through a fully connected layer, the features extracted by DeBERTa can be further fused and optimized, thereby better adapting to the needs of emotional classification tasks.

[0037] S2. The audio in the short video is converted into text as the long text in the short video. The preprocessed long text is obtained by text cleaning, text segmentation, removal of stop words, and truncation according to the maximum length of max_seq. The preprocessed long text is then used to obtain the audio features in the short video through a deep learning long text sentiment classification framework.

[0038] In this embodiment of the invention, the method for converting audio in the short video into text can be implemented using open-source tools such as DeepSpeech (an open-source speech recognition engine provided by Mozilla that can convert audio streams into text), Whisper (a multilingual speech recognition model that can not only convert audio into text but also handle various noise conditions), or the commercial API Baidu Speech Recognition (which supports Chinese and multiple dialects). This invention does not limit the method.

[0039] In an embodiment of the present invention, the calculation process of the truncation threshold N in the preprocessed long text obtained by truncating it according to the maximum length of the truncation threshold includes:

[0040]

[0041] Where, min(·) represents finding the minimum value; abs(·) represents finding the absolute value; tanh(·) represents the hyperbolic tangent function; n is the number of sentences in the long text; arsinh(·) represents the inverse hyperbolic sine function; l i N represents the length of the i-th sentence; model This indicates the text length limit for embedding vector layers in the pre-trained model.

[0042] In this formula, a limiting factor (combining n+arsinh(n) and the tanh function) is introduced to ensure that the maximum length of the input sequence is not too large, thus helping to avoid the problem of insufficient memory during model training. Since arsinh(n) has a smooth growth characteristic, it can help the model dynamically adjust the input length according to the complexity of the text data. This allows max_seq to be reasonably controlled according to the actual amount of data, rather than being fixed at a constant value. By effectively controlling the length of the input text, it can be ensured that when the model processes texts of different lengths, it will not process excessively long sequences that lead to training instability, nor will it lose the capture of important information due to sequences that are too short.

[0043] The truncation method in the preprocessing of long text involves cutting off N / 2 of the length from the beginning and end of the long text and then concatenating them to obtain the final truncated preprocessed text. This is because in short videos, the emotions that the video publisher wants to express are usually at the beginning or end of the video, while the middle of the video is usually the narrative rhythm, and the emotions expressed are less common. Therefore, long texts truncated in this way can maximize the extraction of the emotions that the video publisher wants to express from the audio of the short video, while keeping the text length the same.

[0044] In embodiments of the present invention, the preprocessed long text is used to obtain audio features from the short video through a deep learning long text sentiment classification framework, such as... Figure 3 As shown, the deep learning long text sentiment classification model framework includes:

[0045] S21. The results obtained by Named Entity Recognition (NER) of the preprocessed text data are concatenated and fused with the original text to obtain NER-enhanced input;

[0046] The NER-enhanced input data, obtained by concatenating and fusing the results obtained through Named Entity Recognition (NER) with the original text, not only enriches the semantic information of the text by recognizing named entities in the text through NER and concatenating and fusing this entity information with the original text, but also helps the model to better focus on key entities in the text, thereby more accurately capturing sentiment-related contextual information.

[0047] In step S21, the result obtained through Named Entity Recognition (NER) is concatenated and fused with the original text to obtain the NER-enhanced input, such as... Figure 4 As shown, it includes:

[0048] S211. Convert the preprocessed long text into a token sequence according to the DeBERTa model vocabulary;

[0049] S212. Input the token sequence into the embedding layer of the DeBERTa model to obtain the token embedding, and extract the embedding (token-level embeddings) of each token, thereby preserving fine-grained information at the word level.

[0050] S213. Token-level Embeddings are fed into the regularization layer (Stochastic Dropout Layer, S-Dropout layer) to prevent overfitting and enhance the generalization effect of the model. Then, they are fed into the Bi-MT layer (Bidirectional Memory Network) for context modeling. Finally, they are fed into the CSM layer (Conditional Sequence Model) to capture the dependencies between labels and obtain the category label sequence of named entities.

[0051] Specifically, by introducing the S-Dropout layer, some neurons are randomly dropped during training, which can effectively prevent model overfitting and enhance the model's generalization ability; by introducing the Bi-MT layer, both forward and backward contextual information can be captured simultaneously, thus gaining a more comprehensive understanding of semantic relationships in the text and better capturing long-distance dependencies in long texts, thereby improving the ability to recognize complex entities; by introducing the CSM layer, the dependencies between label sequences can be modeled, thereby improving the accuracy of the NER task and generating more coherent label sequences, thus improving the overall performance of the NER task.

[0052] S214. The category label sequence of named entities obtained through the CSM layer is concatenated and fused with the original text to obtain a new text (NER enhanced input text);

[0053] By concatenating and fusing the named entity category label sequence obtained from the CSM layer with the original text, it is possible to help downstream tasks better understand the key entities and their semantic relationships in the text, thereby improving the performance of this long text sentiment classification task.

[0054] S22. Transform the NER-enhanced input according to the DeBERTa model vocabulary to obtain a token sequence;

[0055] Among them, the decoupled attention mechanism and relative position encoding of the DeBERTa pre-trained model can better capture the contextual dependencies in long texts;

[0056] S23. Input the token sequence into the embedding layer of the DeBerta model to obtain the Token Embedding, and extract the Pooled Output to aggregate the embedding of the entire sequence.

[0057] Among them, DeBERTa's embedding layer can generate high-quality token embeddings, providing a solid foundation for subsequent feature extraction and aggregation; the Pooled Output extracted by the DeBERTa model can aggregate the embeddings of the entire sequence, thereby capturing global sentiment information;

[0058] S24. The Pooled Output is fed into the Dual-Gate Recurrent Unit (D-GRU layer) to enhance the ability to model the context and strengthen the feature representation. Finally, it is fed into the AtA-CL layer to obtain the final audio features.

[0059] The introduction of the D-GRU layer, with its dual-gate mechanism (update gate and reset gate), can better capture long-distance dependencies in long texts and enhance the ability to model contextual information. Combined with DeBERTa's PooledOutput, it further enhances the ability to express features. Finally, the fully connected layer is used to deeply fuse features in a fully connected manner, which can capture the complex relationships between features and thus improve the accuracy of sentiment classification.

[0060] S3. Extract video frames from the acquired short video as images in the short video. Obtain a preprocessed image set by extracting keyframes, adjusting size, converting color channels, and normalizing. Use a deep learning image set sentiment classification framework to obtain image features in the short video from the preprocessed image set.

[0061] In an embodiment of the present invention, the method for extracting keyframes is to calculate the similarity of adjacent keyframes based on changes in image content. If the similarity is lower than a threshold t, then the frame is selected as a keyframe. The formula for the similarity of adjacent keyframes is:

[0062]

[0063] Where Similarity represents the similarity between the current frame image x and the previous keyframe image y; Dict(x,y) represents the distance between the current frame image x and the previous keyframe image y in the original video sequence; SSIM(x,y) represents the structural similarity between the current frame image x and the previous keyframe image y; ln(·) represents the logarithm; M represents the number of frames per second in the current video; and MSE(x,y) represents the mean square error between the current frame image x and the previous keyframe image y.

[0064] This formula integrates multiple measurement methods to achieve a more comprehensive similarity assessment and balance the influence of different features. By introducing Dict(x,y) temporal distance, it can capture the temporal relationship between frames. By introducing SSIM(x,y) structural similarity, it can better capture visual structural information and increase robustness to changes in illumination and contrast. By introducing MSE(x,y) mean square error, it can capture differences between pixels and use logarithmic transformation to smooth the influence of large error values, avoiding interference from extreme values ​​on similarity. Normalization processing makes the results more stable and easier to compare.

[0065] In an embodiment of the present invention, the preprocessed image set is used to obtain image features in the short video through a deep learning image set sentiment classification framework, such as... Figure 5 As shown, the deep learning image ensemble sentiment classification model framework includes:

[0066] S31. Divide the images in the preprocessed image set into patches and perform linear embedding to obtain embedding vectors;

[0067] This embodiment uses patch partitioning to preserve local information of the image, while converting it into a high-dimensional feature representation through linear embedding;

[0068] S32. Input the embedding vectors into three pre-trained models: Vit network (Vision Transformer), VGG network (VisualGeometry Group), and ResNet network (Residual Network) to obtain image features from different angles. Then, fuse the image features of the three models to obtain an image feature after ensemble learning. The image features obtained through ensemble learning are used to learn features in a diverse way and fully explore the potential of different models.

[0069] This embodiment achieves multi-feature extraction based on three different perspectives (Vit captures global context information through the Transformer architecture, VGG captures local texture and detail information through deep convolutional networks, and ResNet captures multi-level feature representations through residual connections), thereby fully exploring the potential of different models and making up for the shortcomings of a single model, and improving the expressive power of features.

[0070] S33. The image features learned through ensemble learning will be used as input to the Bi-MT layer for temporal information modeling. By introducing the Bi-MT layer, not only can the temporal information in the feature sequence be captured, but also the forward and backward contextual information can be captured, thereby gaining a more comprehensive understanding of the dependencies in the feature sequence.

[0071] S34. Then, the features containing temporal information are fed into the TriCNN layer (Triple Convolutional Neural Network) to obtain the fusion of local features and global information;

[0072] This embodiment introduces a TriCNN layer, which not only extracts local features of the image and captures detailed information through convolution operations, but also fuses local features with global information to generate a more comprehensive feature representation. Furthermore, the TriCNN's multi-convolutional structure can capture features at different levels, enhancing the expressive power of features.

[0073] S4. The short text features, audio features, and image features obtained from the short video are concatenated and input into the activation function. The result of short video network public opinion sentiment classification prediction is achieved by setting a threshold.

[0074] In an embodiment of the present invention, the short text features, audio features, and image features obtained from the short video are concatenated and input into an activation function. A threshold is set to achieve the result of short video online public opinion sentiment classification and prediction. Figure 6 As shown, the process includes: concatenating short text features, audio features, and image features obtained from a short video to obtain a short video feature containing all features; inputting the short video feature into an activation function to finally obtain the final predicted result T of the model of this invention (T is the value obtained by max(X,Y,Z), where X is the probability of negative sentiment, Y is the probability of neutral sentiment, and Z is the probability of positive sentiment). If the value of T is equal to the value of X, the final predicted result is negative sentiment; if the value of T is equal to the value of Y, the final predicted result is neutral sentiment; if the value of T is equal to the value of Z, the final predicted result is positive sentiment.

[0075] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A short video online public opinion sentiment classification method based on big data, which extracts short text features, audio features, and image features from video data, concatenates the extracted features, and inputs them into an activation function. The sentiment type of the video is determined by comparing the probability output by the activation function with a set threshold. The method is characterized by... When acquiring audio features, the audio features are converted into long text, and a truncation threshold N is set. The first N / 2 words and the last N / 2 words of the long text are truncated. The truncated words are concatenated together, and the audio features are extracted from them. Specifically, this includes: Using the bullet comments and bullet screen messages appearing in the video as short text data samples, the DeBerta model is used to extract short text features from the video data, namely: S11 and DeBerta models convert the original samples into token sequences; S12. Input the token sequence into the embedding layer to obtain its embedding representation, and extract the hidden state based on the embedding representation; S13. Input the hidden state into the bidirectional memory network layer to extract contextual features; S14. The extracted context features are sequentially passed through a global max pooling layer and a fully connected layer to obtain short text features; After concatenating the truncated words together, named entity recognition is performed. The recognition result is concatenated with the truncated words to form the final long text data. Then, the DeBerta model is used to extract audio features from the final long text data. The DeBerta model is used to extract audio features from the final long text data, namely: The S21 and DeBerta models convert the final long text data into a token sequence. S22. Input the token sequence into the embedding layer to obtain its embedding representation, and then extract the Pooled Output; S23. Input the extracted Pooled Output into the dual-gated loop unit for data augmentation; S24. The enhanced data is processed through a fully connected layer to obtain audio features; The process of obtaining image features includes: S31. Preprocess the image samples, divide the preprocessed image into patches and perform linear embedding to obtain the embedding vector; S32. Input the embedding vectors into the pre-trained ViT network, VGG network, and ResNet network respectively, and concatenate the features obtained from each network to obtain the integrated features. S33. The integrated features are input into the bidirectional memory network layer to model the temporal information, and then the data including the temporal information is passed through the TriCNN network to obtain the fusion of local features and global information; The outputs of S34 and TriCNN networks are processed sequentially through regularization layers and fully connected layers to obtain image features.

2. The method for classifying online public opinion sentiment based on big data in short videos according to claim 1, characterized in that, The calculation process for the truncation threshold N includes: in, This indicates finding the minimum value; This indicates taking the absolute value; This represents the hyperbolic tangent function; n is the number of sentences in the long text; Represents the inverse hyperbolic sine function; This indicates the length of the i-th sentence; This indicates the text length limit for embedding vector layers in the pre-trained model.

3. The short video online public opinion sentiment classification method based on big data according to claim 1, characterized in that, Keyframes are extracted from the video as image samples, and image features are extracted from the image samples. When obtaining keyframes, the similarity between the current frame and the previous keyframe is judged. If the similarity is less than a set threshold, the current frame is used as a keyframe; otherwise, the next frame is judged.

4. The method for classifying online public opinion sentiment based on big data in short videos according to claim 3, characterized in that, The calculation of the similarity between the current frame and the previous keyframe includes: in, This indicates the similarity between the current frame image x and the previous keyframe image y; This represents the distance between the current frame image x and the previous keyframe image y in the original video sequence; This represents the structural similarity between the current frame image x and the previous keyframe image y; This indicates calculating the logarithm; M represents the number of frames per second in the current video. This represents the mean square error between the current frame image x and the previous keyframe image y.

Images