A cross-modal video positive energy evaluation method and system fusing video pictures and scene text information and a computer storage medium

Abstract

The application discloses a cross-modal video positive energy evaluation method and system fusing video pictures and scene text information and a computer storage medium, and relates to the field of video sensitive content analysis. The application solves the problem that the existing regression-based method only considers visual features in a video and ignores scene text information in the video. The application provides the following scheme: a video segment is acquired, a pre-trained R3D model is used to select features of the video segment, and a plurality of feature vectors are obtained; global average pooling operation is performed on the obtained plurality of feature vectors, and video picture features are obtained through full connection; scene text features in the video picture features are extracted, that is, repeated scene text features or sentences are deleted; a text editor is used to extract the scene text features, and mean pooling operation is performed on mark embedding output by a BERT component to obtain a feature vector of each sentence; the obtained video picture features and the obtained scene text features are input into a feature fusion module, visual encoders and scene text encoders are used to jointly encode two kinds of modal information, and cross-modal fusion tokens are aggregated; output of the feature fusion module is taken as input of an MLP module, and a positive energy score of the video is obtained through processing of the MLP module. The application is also suitable for the field of video picture information and scene text information extraction.

Description

Technical Field

[0001] This invention relates to the field of video sensitive content analysis. Background Technology

[0002] In recent years, user-generated content (UGC) videos have become the primary content format on platforms like Douyin, Kuaishou, and Xiaohongshu, attracting tens of millions and billions of users respectively. With video shooting equipment becoming increasingly advanced yet affordable, the cost of producing short videos has significantly decreased. This has led to a large number of short video users being both consumers and creators, resulting in a dramatic increase in the number of short videos online. Consequently, short videos have become a major source of information for humanity. However, the quality of these short videos often varies greatly. This is mainly reflected in the subjective positive energy of the video content. Considering that different users create videos for different purposes, this has led to many videos containing vulgar, negative, or unhealthy content.

[0003] To address this problem, most current work draws inspiration from video classification tasks in deep learning. However, since a short video often belongs to multiple categories, relying solely on classification to evaluate video content has significant limitations. Therefore, a few works have proposed using regression-based methods, assigning scores to each video based on its positive energy, thereby more accurately assessing whether the content of short videos aligns with mainstream social values. However, existing regression-based methods only consider visual features in the video, neglecting the textual information within the scene. Yet, in many cases, semantic cues within the video play a crucial role in the final subjective evaluation of the video's content quality. Summary of the Invention

[0004] The purpose of this invention is to address the problem that existing regression-based methods only consider visual features in videos while ignoring textual information within the video context.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] Option 1: A cross-modal video positive energy evaluation method that integrates video footage and scene text information, the method comprising the following steps:

[0007] S1: Acquire a video segment, wherein the number of frames in the video segment is a preset number of frames;

[0008] S2: Use a pre-trained R3D model to select features from the video segment to obtain multiple feature vectors;

[0009] S3: Perform global average pooling (GAP) on the multiple feature vectors obtained in S2, and obtain video image features through fully connected (FC).

[0010] S4: Use an OCR tool to extract scene text features from the video image features described in S3, and perform deduplication processing on the extracted scene text features by setting the frame interval value, that is, delete duplicate scene text features or sentences.

[0011] S5: Use a sub-network of the pre-trained SBERT model as a text encoder. The text editor is used to extract scene text features from S4 and perform mean pooling on the token embeddings output by the BERT component to obtain the feature vector of each sentence.

[0012] S6: Input the video image features obtained in S4 and the scene text features obtained in S5 into the feature fusion module at the same time, and use the visual encoder and the scene text encoder respectively, and aggregate the cross-modal fusion token to jointly encode the two modal information;

[0013] S7: Use the output of the feature fusion module in S6 as the input of the MLP model, and obtain the positive energy score of the video through the processing of the MLP model.

[0014] Furthermore, a preferred embodiment is provided in which the pre-trained R3D model in S2 is pre-trained using the Kinetic dataset.

[0015] Furthermore, in a preferred embodiment, in S4, the value of the frame spacing is 10.

[0016] Furthermore, a preferred embodiment is provided in which the SBERT model in S5 has two BERT models, and the two BERT models are Siamese networks.

[0017] Furthermore, a preferred embodiment is provided in which the feature fusion module in S7 includes two TransformerEncode structures.

[0018] Furthermore, a preferred embodiment is provided in which the number of layers of both Transformer Encoders is set to 24.

[0019] Furthermore, a preferred embodiment is provided, wherein in S6, the following is used: and When representing aggregation Input image tokens and scene text tokens for the visual encoder and scene text encoder;

[0020] The input fusion token for the aggregation of the l-th visual text and scene text is represented as: The workflow of the visual Transformer layer in the aggregation phase has been updated as follows:

[0021]

[0022] in, These are the output image features corresponding to the fusion token. Y represents the output feature of the next layer corresponding to the image token, Y represents the intermediate feature obtained after combining the output feature of the multi-head attention part with the input feature, MHSA represents the multi-head self-attention layer, MLP represents the multi-layer perceptron layer, and LN represents the layer normalization.

[0023] Furthermore, a preferred implementation is provided in which the workflow of the scene text Transformer layer during the aggregation phase is updated as follows:

[0024]

[0025] in, It is the scene text feature output corresponding to the fusion token. This is the output feature of the next layer corresponding to the scene text token.

[0026] Option 2: A cross-modal video positive energy system that integrates video footage and scene text information, wherein the cross-modal video positive energy system includes:

[0027] Video feature extraction module: used to extract features from video segments using a pre-trained R3D model; performs global average pooling (GAP) operation on the obtained video segments, and obtains video frame features through a fully connected (FC) model;

[0028] Text feature extraction module: used to extract scene text features from the video frame using OCR tools, and to perform deduplication processing on the extracted scene text features by using the frame interval value, that is, to delete duplicate scene text features or sentences; using a sub-network of the pre-trained SBERT model as a text encoder, the text editor is used to extract scene text features, and at the same time, the mean pooling operation is performed on the label embedding output by the BERT component to obtain the feature vector of each sentence.

[0029] Feature fusion module: It is used to simultaneously input the obtained video image features and the obtained scene text features into the feature fusion module, and use the visual encoder and scene text encoder respectively and aggregate cross-modal fusion tokens to jointly encode the two modal information; the output of the feature fusion module is used as the input of the MLP module, and the positive energy score of the video is obtained through the processing of the MLP module.

[0030] Option 3: A computer-readable storage medium storing a computer program configured to, when invoked by a processor, implement the steps of the cross-modal video positive energy evaluation method that integrates video footage and scene text information as described in any of the above options.

[0031] The advantages of this invention are:

[0032] To improve the performance of positive energy evaluation in videos, this invention designs a cross-modal video positive energy evaluation method that integrates video image information and scene text information. Unlike previous single-modal methods that only use video images, this invention utilizes a 3D convolutional neural network to extract video image features, then uses an OCR tool to extract scene text from the video image and a Sentence-BERT model to extract text features. A feature fusion module then fuses these features, and finally, the output of the fusion module is processed by an MLP module to obtain the result. Experiments show that this invention significantly outperforms single-modal techniques for video positive energy analysis. Based on the model's output score, the positive energy of the video content can be analyzed; a higher score indicates more positive content, and a lower score indicates more negative content.

[0033] This invention is also applicable to the field of extracting video image information and scene text information. Attached Figure Description

[0034] Figure 1 This is a flowchart of a cross-modal video positive energy system that integrates video footage and scene text information, as described in Embodiment Nine.

[0035] In the diagram, Video represents a video clip, Vision token represents an image token, Scene text token represents a scene text token, Fusion token represents a fusion token, and Element-wise summation represents element-wise summation.

[0036] The Video Feature Extraction Module, Text Feature Extraction Module, Feature Fusion Module, Vision transformer layer, Scene text transformer layer, and Sentence-BERT are all text encoders.

[0037] Figure 2This is a schematic diagram of the SBERT model for a cross-modal video positive energy evaluation method that integrates video footage and scene text information, as described in Implementation Method 4.

[0038] cosine-sim(u, v) is the cosine similarity calculation, pooling is the pooling operation, BERT is a pre-trained large-scale language model, Sentence A is sentence A, and Sentence B is sentence B.

[0039] Figure 3 A schematic diagram illustrating the performance of the single-mode method provided by this invention for positive energy evaluation.

[0040] Figure 4 A comparison diagram of the cross-modal fusion method provided by this invention with other feature fusion methods. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0042] Implementation Method 1, see [link] Figure 1 This embodiment describes a cross-modal video positive energy evaluation method that integrates video footage and scene text information. The method includes the following steps:

[0043] S1: Acquire a video segment, wherein the number of frames in the video segment is a preset number of frames;

[0044] S2: Use a pre-trained R3D model to select features from the video segment to obtain multiple feature vectors;

[0045] S3: Perform global average pooling (GAP) on the multiple feature vectors obtained in S2, and obtain video image features through fully connected (FC).

[0046] S4: Use an OCR tool to extract scene text features from the video image features described in S3, and perform deduplication processing on the extracted scene text features by setting the frame interval value, that is, delete duplicate scene text features or sentences.

[0047] S5: Use a sub-network of the pre-trained SBERT model as a text encoder. The text editor is used to extract scene text features from S4 and perform mean pooling on the token embeddings output by the BERT component to obtain the feature vector of each sentence.

[0048] S6: Input the video image features obtained in S4 and the scene text features obtained in S5 into the feature fusion module at the same time, and use the visual encoder and the scene text encoder respectively, and aggregate the cross-modal fusion token to jointly encode the two modal information;

[0049] S7: Use the output of the feature fusion module in S6 as the input of the MLP module, and obtain the positive energy score of the video through the processing of the MLP module.

[0050] In a specific application scenario, this application employs a residual 3D convolutional neural network (R3D model) as the feature extractor for video. The R3D model can be considered a 3D version of ResNet. This model is an extension of ResNet and 2D CNN, its main feature being the use of 3D convolution to capture the spatiotemporal features of video data, and the use of residual connections to accelerate model training and improve model performance. The input to the R3D network is a series of video frames, which, after a series of 3D convolutions, batch normalization, and non-linear activation operations, yield a multidimensional tensor representing the video features. We use a pre-trained R3D model on the Kinetic dataset as the feature extractor. Furthermore, we use the output of the last pooling layer of the R3D network as the extracted features. Then, global average pooling (GAP) is performed on the obtained feature map, and the pooling result is input into the feature fusion module through a fully connected (FC) layer. Then, an OCR tool is used to extract scene text from the video frame, and the Sentence-BERT (SBERT) model is used to extract text features. A feature fusion module then fuses the features, and finally, the output of the fusion module is passed through an MLP module to obtain the final result.

[0051] in Figure 3 The paper demonstrates the performance of a single-modal method for positive energy evaluation. Feature extraction from video images uses ResNet50, C3D, and R3D models, while temporal modeling of these features employs MLP, GRU, LSTM, and Transformer models. Performance metrics are shown across six data categories, with Transformer demonstrating significantly higher scores than MLP, GRU, and LSTM, exceeding the average across all data categories.

[0052] See Figure 4 This embodiment is described, wherein Figure 4 The results show that the present invention, with a value of 0.935, is significantly better than the technology using a single encoder for video positive energy analysis. Based on the output score of the model, the positive energy of the video content can be analyzed. The higher the score, the more positive the video content, and the lower the score, the more negative the video content.

[0053] Compared with existing feature fusion methods, the dual encoder structure described in this embodiment, which uses a visual encoder and a scene text encoder in conjunction with a cross-modal fusion token to jointly encode two modalities, is superior to other feature fusion methods.

[0054] Implementation Method 2: This implementation method further defines the cross-modal video positive energy evaluation method that integrates video footage and scene text information provided in Implementation Method 1. In S2, the pre-trained R3D model is trained using the Kinetic dataset.

[0055] Implementation Method 3: This implementation method further defines the cross-modal video positive energy evaluation method that integrates video footage and scene text information provided in Implementation Method 1. In S4, the frame spacing for extracting text using an OCR tool is set to 10.

[0056] In this implementation, we use Baidu's OCR tool to extract text information from the video footage. Before applying the OCR tool, we first extract video frames with appropriate frame spacing. To ensure that as many subtitles as possible are captured, we set the frame spacing to 10. Then, we apply the OCR tool and return all recognition results. Finally, we perform deduplication processing on the extracted text, that is, we delete duplicate text or sentences.

[0057] Implementation Method 4: This implementation method further defines the cross-modal video positive energy evaluation method that integrates video footage and scene text information provided in Implementation Method 1. The SBERT model in S5 has two BERT models, which are Siamese networks.

[0058] See Figure 2 In this embodiment, after obtaining the text information from the video, the next step is to extract the text's feature vectors. We choose the Sentence-BERT (SBERT) model as the text encoder. In the cross-modal video positive energy evaluation method that integrates video footage and scene text information described in this embodiment, we use a sub-network of the pre-trained SBERT as the text encoder, and simultaneously perform mean pooling on the label embeddings output by the BERT components to obtain the feature vectors for each sentence.

[0059] Implementation Method 5: This implementation method further defines the cross-modal video positive energy evaluation method that integrates video footage and scene text information provided in Implementation Method 1. The feature fusion module in S7 includes two Transformer Encode structures.

[0060] Implementation Method Six: This implementation method further defines the cross-modal video positive energy evaluation method that integrates video footage and scene text information provided in Implementation Method One. The number of layers in both Transformer Encoders is set to 24.

[0061] In the data loading part of the model training implementation (in Implementation Methods 5 and 6), a random sampling strategy is adopted for video frames. Each time, 96 video frames are randomly selected from the entire video. These 96 frames are then center-cropped, resulting in 96 112*112 video frames as the network input. The R3D model is then used to extract features from the 96 video frames in six steps. Simultaneously, OCR is used to extract text from the original video (excluding the extracted 96 frames).

[0062] The two Transformer Encoder structures in the feature fusion module are identical, both with 24 layers and 16 heads for the multi-head attention mechanism. The output corresponding to the fusion token in the Encoder is used as input to the final fully connected layer to obtain the positive energy score. During model training, we freeze the parameters of the content feature extraction module and only train the feature aggregation module.

[0063] The MAE loss function and AdamW optimizer were used, and Linear Warmup was selected as the warmup strategy. The number of warm-up steps was set to 15 steps, which is 15 epochs. The total number of training epochs was 100, and the initial learning rate was set to 1e-3. That is, the learning rate first increased uniformly from 6.67e-5 to the initial learning rate of 1e-3 in the first 15 epochs, and then decreased uniformly to 1.18e-5.

[0064] Implementation Method Seven: This implementation method further defines the cross-modal video positive energy evaluation method that integrates video footage and scene text information provided in Implementation Method One. In step S6, the method uses... and When representing aggregation Input image tokens and scene text tokens for the visual encoder and scene text encoder;

[0065] The input fusion token for the aggregation of the l-th visual text and scene text is represented as: The workflow of the visual Transformer layer in the aggregation phase has been updated as follows:

[0066]

[0067] in, These are the output image features corresponding to the fusion token. Y represents the output feature of the next layer corresponding to the image token, Y represents the intermediate feature obtained after combining the output feature of the multi-head attention part with the input feature, MHSA represents the multi-head self-attention layer, MLP represents the multi-layer perceptron layer, and LN represents the layer normalization.

[0068] Implementation Method Eight: This implementation method further defines the cross-modal video positive energy evaluation method that integrates video footage and scene text information provided in Implementation Method One. The workflow of the scene text Transformer layer in the aggregation phase is updated as follows:

[0069]

[0070] in, It is the scene text feature output corresponding to the fusion token. This is the output feature of the next layer corresponding to the scene text token.

[0071] In embodiments seven and eight, the semantic relevance between scene text and visual appearance varies depending on the context, as scene text in the image may provide valuable information, while in most cases the image does not contain any scene text information. Therefore, effectively aggregating these two different modalities into a unified visual representation is challenging.

[0072] To handle the fusion tasks of scene text perception and scene text absence, vision needs to learn the final features corresponding to the image modality for matching. Therefore, during the training phase, this work uses different tokens, namely image tokens or fusion tokens, to obtain the final features based on whether the OCR tool's recognition result is empty. In the case of scene text absence, the vision model in this work degenerates into a pure vision encoder model and outputs the image features of the [VISION] token as the final features. In the case of scene text perception, this work uses a scene text encoder to learn the semantic features of the scene text. The vision tower in this work is simply added... A layer of visual and scene text aggregation is used to perform mid-level fusion in the image modality and output fused features as the final features from an additional fusion token [FUS].

[0073] The visual scene text aggregation layer consists of a visual Transformer layer and a scene text Transformer layer from two encoders. To exchange relevant information between the visual and scene text, both layers are equipped with a new token, a shared special fusion token [FUS]. This work uses... and Indicates the aggregation phase The input image token and scene text token for the visual encoder and scene text encoder, respectively. The input fusion token for the l-th visual and scene text aggregation is represented as... The workflow of the visual Transformer layer in the aggregation phase has been updated as follows:

[0074]

[0075] in This corresponds to the output image features of the fusion token. The workflow of the scene text Transformer layer in the aggregation phase is similar:

[0076]

[0077] in This corresponds to the scene text feature output of the fusion token. The input fusion features of the next layer are calculated by summing their element-wise, defined as... In this way, visual features V and scene text features S are learned through independent Transformer layers. A special fusion token [FUS] acts as a bridge between the two encoders because it is shared in both. Due to the visual and scene text aggregation layer, the learning of image features and scene text features influences each other indirectly through the fusion token. For a similar bottleneck attention structure for video classification, video patches and audio are fused by averaging the predictions from both modalities. Unlike updating the shared token twice, this work directly adds the predicted fusion tokens from the visual and scene text Transformer layers to form the fusion token during the aggregation process. To further address the issue of missing scene text, this work proposes an additional image-text contrast loss to enhance visual representations and uses it in conjunction with the fusion-text contrast loss. Thus, both image-text pairs and fusion-text pairs contain visual appearance information and share information only in relevant parts of the scene text, aiming to facilitate cross-modal learning of scene text perception.

[0078] Finally, the output of the fusion module is used as the input to the MLP module, which consists of two fully connected layers. The output of the MLP is the predicted positive energy score of the video.

[0079] Implementation Method Nine: This implementation method proposes a cross-modal video positive energy system that integrates video footage and scene text information. The system includes:

[0080] Video feature extraction module: used to extract features from video segments using a pre-trained R3D model; performs global average pooling (GAP) operation on the obtained video segments, and obtains video frame features through a fully connected (FC) model;

[0081] Text feature extraction module: used to extract scene text features from the video frame using OCR tools, and to perform deduplication processing on the extracted scene text features by using the frame interval value, that is, to delete duplicate scene text features or sentences; using a sub-network of the pre-trained SBERT model as a text encoder, the text editor is used to extract scene text features, and at the same time, the mean pooling operation is performed on the label embedding output by the BERT component to obtain the feature vector of each sentence.

[0082] Feature fusion module: It is used to simultaneously input the obtained video image features and the obtained scene text features into the feature fusion module, and use the visual encoder and scene text encoder respectively and aggregate cross-modal fusion tokens to jointly encode the two modal information; the output of the feature fusion module is used as the input of the MLP module, and the positive energy score of the video is obtained through the processing of the MLP module.

[0083] The system adopted in this embodiment is implemented using a cross-modal video positive energy evaluation method that integrates video footage and scene text information, as described in embodiments one through seven.

[0084] Implementation Method 10: A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, the computer program being configured to implement, when called by a processor, the steps of the cross-modal video positive energy evaluation method that integrates video images and scene text information as described in any of the above schemes.

[0085] In a specific application scenario, the specific hardware and software environment used in this implementation method is as follows:

[0086] Hardware environment: 1. CPU: 6-core Intel(R) Core(TM) i5-9600K CPU @3.70GHz

[0087] 2. Memory: 16GB 2400MHz DDR4

[0088] 3. Graphics card: GeForce RTX 3070

[0089] On the software side, the operating system is Ubuntu 20.04.1 LTS. The R3D neural network part mainly relies on PyTorch 1.13.1 for implementation, and Anaconda is used to manage the required third-party libraries.

[0090] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present disclosure.

[0091] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0092] Those skilled in the art will understand that the above description is merely a preferred embodiment of the present invention, and the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. This is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0093] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.

Claims

1. A cross-modal video positive energy evaluation method that integrates video footage and scene text information, characterized in that, The method includes the following steps: S1: Acquire a video segment, wherein the number of frames in the video segment is a preset number of frames; S2: Use a pre-trained R3D model to select features from the video segment to obtain multiple feature vectors; S3: Perform global average pooling (GAP) on the multiple feature vectors obtained in S2, and obtain video image features through fully connected (FC). S4: Use an OCR tool to extract scene text features from the video image features described in S3, and perform deduplication processing on the extracted scene text features by setting the frame interval value, that is, delete duplicate scene text features or sentences. S5: Use a sub-network in the pre-trained SBERT model as a text encoder. The text encoder is used to extract scene text features from S4 and perform mean pooling on the token embeddings output by the BERT components to obtain the feature vector of each sentence. S6: Input the video image features obtained in S4 and the scene text features obtained in S5 into the feature fusion module at the same time, and use the visual encoder and the scene text encoder respectively, and aggregate the cross-modal fusion token to jointly encode the two modal information; S7: Use the output of the feature fusion module in S6 as the input of the MLP module, and obtain the positive energy score of the video through the processing of the MLP module; In S4, the value of the frame spacing is 10; The feature fusion module described in S7 includes two Transformer Encode structures; In S6, the use of and When representing aggregation Input image tokens and scene text tokens for the visual encoder and scene text encoder; The input fusion token for the aggregation of the l-th visual text and scene text is represented as: ， The workflow of the visual Transformer layer in the aggregation phase has been updated as follows: in, is the output image feature corresponding to the fusion token, and Y is the intermediate feature obtained after combining the output feature of the multi-head attention part with the input feature; The workflow of the scene text Transformer layer during the aggregation phase has been updated as follows: in, This is the output feature of the next layer corresponding to the scene text token. Output the scene text features corresponding to the fusion token.

2. The cross-modal video positive energy evaluation method that integrates video footage and scene text information according to claim 1, characterized in that, The pre-trained R3D model mentioned in S2 is trained using the Kinetic dataset.

3. The cross-modal video positive energy evaluation method that integrates video footage and scene text information according to claim 1, characterized in that, The SBERT model described in S5 has two BERT models, which are Siamese networks.

4. The cross-modal video positive energy evaluation method that integrates video footage and scene text information according to claim 1, characterized in that, The number of layers in both Transformer Encoders is set to 24.

5. A cross-modal video positive energy system that integrates video footage and scene text information, characterized in that, The system is implemented based on the method of claim 1, and the system includes... Video feature extraction module: used to extract features from video segments using a pre-trained R3D model; performs global average pooling (GAP) operation on the obtained video segments, and obtains video frame features through a fully connected (FC) model; Text feature extraction module: used to extract scene text features from the video frame using OCR tools, and to perform deduplication processing on the extracted scene text features by using the frame interval value, that is, to delete duplicate scene text features or sentences; a sub-network of the pre-trained SBERT model is used as a text encoder, which is used to extract scene text features, and mean pooling is performed on the label embedding output by the BERT component to obtain the feature vector of each sentence. Feature fusion module: It is used to simultaneously input the obtained video image features and the obtained scene text features into the feature fusion module, and use the visual encoder and scene text encoder respectively and aggregate cross-modal fusion tokens to jointly encode the two modal information; the output of the feature fusion module is used as the input of the MLP module, and the positive energy score of the video is obtained through the processing of the MLP module.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program configured to, when invoked by a processor, implement the steps of the cross-modal video positive energy evaluation method according to any one of claims 1-4, which integrates video footage and scene text information.

Images