A video description method, device and storage medium for multi-feature fusion

By employing a multi-feature fusion method, video features are extracted using ResNet101 and Faster-RCNN networks, and natural language descriptions are generated through cross-multi-head attention blocks. This solves the problem of feature redundancy in video descriptions and achieves efficient video description and storage.

CN114596523BActive Publication Date: 2024-11-08GUILIN UNIV OF ELECTRONIC TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210167890.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-02-23
Publication Date
2024-11-08
Estimated Expiration
2042-02-23

AI Technical Summary

Technical Problem

Existing technologies have failed to effectively eliminate redundant information between different feature sets during video description and have failed to distinguish the most dissimilar information among multiple original feature sets.

Method used

A multi-feature fusion method is adopted, which extracts motion features and global features of the video through ResNet101 and Faster-RCNN networks, and performs feature fusion training through cross-multi-head attention blocks to generate natural language descriptions.

Benefits of technology

It achieves efficient fusion of video features, generating natural and efficient video description statements that are easy to store and process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114596523B_ABST
    Figure CN114596523B_ABST
Patent Text Reader

Abstract

The present invention provides a video description method, device and storage medium for multi-feature fusion, which import a data set, wherein the data set includes multiple data groups, each of which includes a video and description information; feature extraction is performed on the videos in each data group to obtain motion features and global features, the motion features and the global features are feature spliced ​​to obtain video features corresponding to each video, object features corresponding to the description information in each data group are extracted, and contribution values ​​are calculated, and the sum of regional features of all video frames in the corresponding video is calculated according to each contribution value; the features corresponding to each video are respectively input into a training model for fusion training to obtain fusion features; the description information corresponding to each video is respectively converted into text features; the text features corresponding to each video and the fusion features corresponding to each video are respectively input into a preset transformer language model to output video description sentences.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention mainly relates to the field of video data processing technology, and specifically to a video description method, device and storage medium for multi-feature fusion. Background Art

[0002] With the development of Internet multimedia and other technologies, massive amounts of video data will inevitably be generated. How to efficiently process and store this information has become an urgent problem to be solved. Video description is an advanced form of video analysis and a sub-branch of video understanding. That is, for a given video, useful visual semantic information is extracted by computer, and then a natural language description of the video content is generated based on this visual semantic information and a predefined language model. In layman's terms, for a short video, the video description model performs preprocessing, analysis, synthesis and other operations on it, and then outputs one (or more) sentences that describe (or summarize) the video content, and the sentences are consistent with a certain human language in terms of vocabulary and grammar. The description task not only requires the application of multiple technologies in the two major research fields of computer vision and natural language processing, but also requires the study of the representation and transformation of two different modal information, video and natural language. Today's video description task is divided into two steps: (1) the encoder extracts visual features from the video; (2) the decoder generates description text based on the extracted visual features. However, these methods do not effectively eliminate the redundant information generated by the correlation between different feature sets, and do not distinguish multiple original feature sets to obtain the most differentiated information. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to address the deficiencies of the existing technology and provide a video description method, device and storage medium with multi-feature fusion.

[0004] The present invention solves the above technical problems with the following technical solution: A multi-feature fusion video description method comprises the following steps:

[0005] Importing a data set, the data set including a plurality of data groups, each of the data groups including a video and description information;

[0006] Extracting features from the videos in each of the data groups to obtain motion features and global features, and performing feature splicing on the motion features and the global features to obtain video features corresponding to each video;

[0007] Extract the object features corresponding to the description information in each data group respectively, calculate the contribution value of each object feature, and calculate the regional feature sum of all video frames in the corresponding video based on each contribution value;

[0008] The object features, video features and region features corresponding to each of the videos are respectively fused with the input training model to obtain fused features;

[0009] Converting the description information corresponding to each of the videos into text features respectively;

[0010] The text features corresponding to each of the videos and the fusion features corresponding to each of the videos are respectively input into a preset transformer language model, and a video description sentence corresponding to each video is output.

[0011] Another technical solution of the present invention to solve the above technical problem is as follows: a video description device for multi-feature fusion, comprising:

[0012] An import module, configured to import a data set, wherein the data set includes a plurality of data groups, each of which includes a video and description information;

[0013] A preprocessing module is used to extract features from the videos in each of the data groups to obtain motion features and global features, and to perform feature splicing on the motion features and the global features to obtain video features corresponding to each video; extract object features corresponding to the description information in each data group, and calculate the contribution value of each object feature; and calculate the sum of regional features of all video frames in the corresponding video based on each contribution value;

[0014] A multi-feature fusion module is used to fuse the object features, video features and region features corresponding to each of the videos with the input training model to obtain fused features;

[0015] The description generation module is used to convert the description information corresponding to each video into text features; input the text features corresponding to each video and the fusion features corresponding to each video into a preset transformer language model, and output a video description sentence.

[0016] Another technical solution of the present invention to solve the above technical problems is as follows: A video description device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the multi-feature fusion video description method as described above is implemented.

[0017] Another technical solution of the present invention to solve the above technical problem is as follows: a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the video description method of multi-feature fusion as described above is implemented.

[0018] The beneficial effects of the present invention are: multiple features extracted from the video, namely global and local features, are fused together, and the description information corresponding to the video is converted into text features, that is, processed into low-dimensional features, and input into the transformer language model together with the fused features, thereby generating natural video description sentences, which facilitates efficient video processing and storage. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 A schematic diagram of a flow chart of a video description method provided by an embodiment of the present invention;

[0021] Figure 2 This is a module block diagram of a video description device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0023] Example 1:

[0024] Figure 1 A schematic diagram of a flow chart of a video description method provided by an embodiment of the present invention.

[0025] like Figure 1 As shown, a video description method of multi-feature fusion includes the following steps:

[0026] Importing a data set, the data set including a plurality of data groups, each of the data groups including a video and description information;

[0027] Extracting features from the videos in each of the data groups to obtain motion features and global features, and performing feature splicing on the motion features and the global features to obtain video features corresponding to each video;

[0028] Extract the object features corresponding to the description information in each data group respectively, calculate the contribution value of each object feature, and calculate the regional feature sum of all video frames in the corresponding video based on each contribution value;

[0029] The object features, video features and region features corresponding to each of the videos are respectively fused with the input training model to obtain fused features;

[0030] Converting the description information corresponding to each of the videos into text features respectively;

[0031] The text features corresponding to each of the videos and the fusion features corresponding to each of the videos are respectively input into a preset transformer language model, and a video description sentence corresponding to each video is output.

[0032] In the above embodiment, multiple features extracted from the video, namely global and local features, are fused together, and the description information corresponding to the video is converted into text features, i.e. processed into low-dimensional features, and input into the transformer language model together with the fused features, thereby generating natural video description sentences, which facilitates efficient video processing and storage.

[0033] Preferably, the feature extraction is performed on each video in the data group to obtain motion features and global features, and the motion features and the global features are feature spliced ​​to obtain video features corresponding to each video, specifically:

[0034] Performing feature extraction on the videos in the data set using a ResNet101 network to obtain motion features and global features;

[0035] The motion feature and the global feature are spliced ​​together using a splicing formula to obtain the video feature. The splicing formula is F v =cat(F i , F m ), where F v is the video feature, F i is the global feature, F m is a motion feature, and cat is a splicing symbol.

[0036] In the above process, the ResNet101 network and the Faster-RCNN fast regional convolutional network can quickly extract motion features and video features, and splice the two into video features to obtain local and global features.

[0037] Preferably, the object features corresponding to the description information in each data group are extracted respectively, and the contribution value of each object feature is calculated, and the sum of the regional features of all video frames in the corresponding video is calculated according to each contribution value, specifically:

[0038] Extracting features from the description information in each of the data groups using the Faster-RCNN fast regional convolutional network to obtain object features;

[0039] The contribution value of each object feature is calculated according to the contribution value formula, which is: Among them, F o is the contribution value of the video feature, M is all the object features in a video frame, a j is the weight of the object feature, which is calculated based on the proportion of the object feature in the video image and the distance between the object feature and the video center. j is the word vector of the object feature, and w is the network weight generated during the feature extraction process;

[0040] The sum of regional features of all video frames in the corresponding video is calculated according to the regional feature formula and the contribution value. The regional feature formula is: Among them, F r For regional characteristics and (χ1, y1) and (χ2, y2) are the coordinates of the upper left corner and the upper right corner of the video frame object box, respectively, and W and H are the width and height of the video frame object box.

[0041] In the above process, the regional features of each video frame are represented by a 5D vector.

[0042] Preferably, the object features, video features and region features corresponding to each of the videos are fused with the input training model to obtain fused features, specifically:

[0043] The training model includes a cross-multi-head attention block, wherein the cross-multi-head attention block includes a processing module and an output module, and the processing module is used to perform fusion training through a fusion calculation formula, wherein the fusion calculation formula is:

[0044] H v =MultiHead(F o ,F v 'F v ),

[0045] H r =MultiHead(F o ,F r 'F r ),

[0046] H o =MultiHead(F o ,F o 'F o ),

[0047] Among them, MultiHead is the cross-head attention block symbol, F v is the video feature, F v ' is the video feature label, Fo is the object feature, F o ' is the object feature label, F r is the regional characteristics and F r ' is the regional feature label,

[0048] The output module is used to output the fusion result, which is:

[0049]

[0050] in, is the output after encoding the video features, is the output after encoding the object features, is the output after encoding the regional features, where F OUTPUT =MultiHead(F,H,H),F OUTPUT for or or F=cat(F v ,F o ,F r ), H=cat(H v ,H o ,H r ), cat is the concatenation symbol, W1, W2, and W3 are the weights generated by the training model during the fusion training process, and BN is the batch normalization symbol.

[0051] It should be understood that the final output of the cross-head attention block is the encoded video features Encoded object features And the encoded regional features and

[0052] In the above process, it is necessary to introduce various label features. For example, object labels can be used to guide the selection of regional features and video features in the video. Faster-RCNN can be used to extract object labels in the video, and then the word embedding model can be used to calculate the object feature F. o . Local features and global features are fused together through the attention module to enhance the interaction of different features.

[0053] Example 2:

[0054] Figure 2 This is a module block diagram of a video description device provided by an embodiment of the present invention.

[0055] like Figure 2 As shown, a multi-feature fusion video description device includes:

[0056] An import module, configured to import a data set, wherein the data set includes a plurality of data groups, each of which includes a video and description information;

[0057] A preprocessing module is used to extract features from the videos in each of the data groups to obtain motion features and global features, and to perform feature splicing on the motion features and the global features to obtain video features corresponding to each video; extract object features corresponding to the description information in each data group, and calculate the contribution value of each object feature; and calculate the sum of regional features of all video frames in the corresponding video based on each contribution value;

[0058] A multi-feature fusion module is used to fuse the object features, video features and region features corresponding to each of the videos with the input training model to obtain fused features;

[0059] The description generation module is used to convert the description information corresponding to each video into text features; input the text features corresponding to each video and the fusion features corresponding to each video into a preset transformer language model, and output a video description sentence.

[0060] Preferably, in the preprocessing module, feature extraction is performed on each video in the data group to obtain motion features and global features, and the motion features and the global features are feature spliced ​​to obtain video features corresponding to each video, specifically:

[0061] Performing feature extraction on the videos in the data set using a ResNet101 network to obtain motion features and global features;

[0062] The motion feature and the global feature are spliced ​​together using a splicing formula to obtain the video feature. The splicing formula is F v =cat(F i , F m ), where F v is the video feature, F i is the global feature, F m is a motion feature, and cat is a splicing symbol.

[0063] Preferably, in the preprocessing module, object features corresponding to the description information in each data group are extracted respectively, and the contribution value of each object feature is calculated. The sum of the regional features of all video frames in the corresponding video is calculated based on each contribution value, specifically:

[0064] Extracting features from the description information in each of the data groups using the Faster-RCNN fast regional convolutional network to obtain object features;

[0065] The contribution value of each object feature is calculated according to the contribution value formula, which is: Among them, F o is the contribution value of the video feature, M is all the object features in a video frame, a j is the weight of the object feature, which is calculated based on the proportion of the object feature in the video image and the distance between the object feature and the video center. j is the word vector of the object feature, and w is the network weight generated during the feature extraction process;

[0066] The sum of regional features of all video frames in the corresponding video is calculated according to the regional feature formula and the contribution value. The regional feature formula is: Among them, F r For regional characteristics and (χ1, y1) and (χ2, y2) are the coordinates of the upper left corner and the upper right corner of the video frame object box, respectively, and W and H are the width and height of the video frame object box.

[0067] Preferably, in the multi-feature fusion module, the object features, video features, and region features corresponding to each video are fused with the input training model to obtain fused features, specifically:

[0068] The training model includes a cross-multi-head attention block, wherein the cross-multi-head attention block includes a processing layer and an output layer, and the processing layer is used to perform fusion training through a fusion calculation formula, wherein the fusion calculation formula is

[0069] H v =MultiHead(F o ,F v 'F v ),

[0070] H r =MultiHead(F o ,F r 'F r ),

[0071] H o =MultiHead(F o ,F o 'F o ),

[0072] Among them, F v is the video feature, F v ' is the video feature label, F o is the object feature, F o ' is the object feature label, F r is the regional characteristics and F r ' is the regional feature label,

[0073] The output layer is used to output the fusion result, which is

[0074]

[0075] in, is the output after encoding the video features, is the output after encoding the object features, is the output after encoding the regional features, where F OUTPUT =MultiHead(F,H,H),F OUTPUT for or or F=cat(F v ,F o ,F r ), H=cat(H v ,H o ,H r ), cat is the concatenation symbol, W1, W2, and W3 are the weights generated by the training model during the fusion training process, and BN is the batch normalization symbol.

[0076] Example 3:

[0077] A video description device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the video description method of multi-feature fusion as described above is implemented.

[0078] Example 4:

[0079] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the video description method of multi-feature fusion is implemented.

[0080] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0081] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0082] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is merely a logical functional division. In actual implementation, other division methods may be used, such as combining or integrating multiple units or components into another system, or ignoring or not implementing certain features.

[0083] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected based on actual needs to achieve the objectives of the embodiments of the present invention.

[0084] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0085] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.

[0086] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A video description method based on multi-feature fusion, It is characterized in that The steps include: Importing a data set, the data set comprising a plurality of data groups, each of the data groups comprising a video and description information; Extracting features from the videos in each of the data groups respectively to obtain motion features and global features, and performing feature splicing on the motion features and the global features to obtain video features corresponding to each video; Extract the object features corresponding to the description information in each data group respectively, calculate the contribution value of each object feature, and calculate the sum of the regional features of all video frames in the corresponding video according to each contribution value; The object features, video features and region features corresponding to each of the videos are respectively fused and trained with the input training model to obtain fused features; Convert the description information corresponding to each of the videos into text features respectively; The text features corresponding to each of the videos and the fusion features corresponding to each of the videos are respectively input into a preset transformer language model, and a video description sentence corresponding to each video is output.

2. The video description method of multi-feature fusion according to claim 1, It is characterized in that The feature extraction of the videos in each of the data groups is performed to obtain motion features and global features, and the motion features and the global features are feature spliced ​​to obtain video features corresponding to each video, specifically: Extracting features from the videos in the data set using a ResNet101 network to obtain motion features and global features; The motion feature and the global feature are spliced ​​together by a splicing formula to obtain a video feature. The splicing formula is: v =cat(F i , F m ), where F v is the video feature, F i is the global feature, F m is a motion feature, and cat is a splicing symbol.

3. The video description method of multi-feature fusion according to claim 1, It is characterized in that The object features corresponding to the description information in each data group are extracted respectively, and the contribution value of each object feature is calculated, and the regional features and of all video frames in the corresponding video are calculated according to each contribution value, specifically: Extracting features from the description information in each of the data groups using the Faster-RCNN fast regional convolutional network to obtain object features; The contribution value of each object feature is calculated according to the contribution value formula, and the contribution value formula is: Among them, F o is the contribution value of the video feature, M is all the object features in a video frame, a j is the weight of the object feature, which is calculated based on the proportion of the object feature in the video screen and the distance between the object feature and the center of the video. j is the word vector of the object feature, and w is the network weight generated during the feature extraction process; The regional features and the regional features of all video frames in the corresponding video are calculated according to the regional feature formula and the contribution value. The regional feature formula is: Among them, F r is the regional characteristics and (v 1 ,y 1 ) and (v 2 ,y 2 ) are the upper left corner coordinates and upper right corner coordinates of the video frame object box, W and H are the width and height of the video frame object box.

4. The video description method of multi-feature fusion according to claim 3, It is characterized in that The object features, video features and regional features corresponding to each of the videos are respectively fused and trained with the input training model to obtain fused features, specifically: The training model includes a cross-multi-head attention block, wherein the cross-multi-head attention block includes a processing module and an output module, and the processing module is used to perform fusion training through a fusion calculation formula, wherein the fusion calculation formula is: H v =MultiHead(F o ,F v’ F v ), H r =MultiHead(F o ,F r’ F r ), H o =MultiHead(F o ,F o’ F o ), Among them, MultiHead is the cross-head attention block symbol, F v is the video feature, F v’ is the video feature label, F o is the object feature, F o’ is the object feature label, F r is the regional characteristics and, F r’ is the regional feature label, The output module is used to output the fusion result, and the fusion result is: in, is the output after encoding the video features, is the output after encoding the object features, is the output after encoding the regional features, where F OUTPUT =MultiHead(F,H,H),F OUTPUT for or or F = cat(F v ,F o ,F r ), H = cat(H v ,H o ,H r ), cat is the splicing symbol, W 1 , W 2 , W 3 It is the weight generated by the training model during the fusion training process, and BN is the batch normalization symbol.

5. A video description device with multi-feature fusion, It is characterized in that include: An import module, used to import a data set, wherein the data set includes a plurality of data groups, each of which includes a video and description information; A preprocessing module is used to extract features from the videos in each of the data groups to obtain motion features and global features, and to perform feature splicing on the motion features and the global features to obtain video features corresponding to each video; Extract the object features corresponding to the description information in each data group respectively, calculate the contribution value of each object feature, and calculate the sum of the regional features of all video frames in the corresponding video according to each contribution value; A multi-feature fusion module is used to fuse the object features, video features and regional features corresponding to each of the videos with the input training model to obtain fused features; A description generation module, used to convert the description information corresponding to each of the videos into text features; The text features corresponding to each of the videos and the fusion features corresponding to each of the videos are respectively input into a preset transformer language model, and a video description sentence is output.

6. The video description device with multi-feature fusion according to claim 5, It is characterized in that In the preprocessing module, feature extraction is performed on the videos in each of the data groups to obtain motion features and global features, and the motion features and the global features are feature spliced ​​to obtain video features corresponding to each video, specifically: Extracting features from the videos in the data set using a ResNet101 network to obtain motion features and global features; The motion feature and the global feature are spliced ​​together by a splicing formula to obtain a video feature. The splicing formula is: v =cat(F i , F m ), where F v is the video feature, F i is the global feature, F m is a motion feature, and cat is a splicing symbol.

7. The video description device with multi-feature fusion according to claim 5, It is characterized in that In the preprocessing module, the object features corresponding to the description information in each data group are extracted respectively, and the contribution value of each object feature is calculated. The regional feature sum of all video frames in the corresponding video is calculated according to each contribution value, specifically: Extracting features from the description information in each of the data groups using the Faster-RCNN fast regional convolutional network to obtain object features; The contribution value of each object feature is calculated according to the contribution value formula, and the contribution value formula is: Among them, F o is the contribution value of the video feature, M is all the object features in a video frame, a j is the weight of the object feature, which is calculated based on the proportion of the object feature in the video screen and the distance between the object feature and the center of the video. j is the word vector of the object feature, and w is the network weight generated during the feature extraction process; The regional features and the regional features of all video frames in the corresponding video are calculated according to the regional feature formula and the contribution value. The regional feature formula is: Among them, F r is the regional characteristics and (v 1 ,y 1 ) and (v 2 ,y 2 ) are the upper left corner coordinates and upper right corner coordinates of the video frame object box, W and H are the width and height of the video frame object box.

8. The video description device with multi-feature fusion according to claim 7, It is characterized in that In the multi-feature fusion module, the object features, video features and regional features corresponding to each of the videos are fused with the input training model to obtain fused features, specifically: The training model includes a cross-head attention block, wherein the cross-head attention block includes a processing layer and an output layer, and the processing layer is used to perform fusion training through a fusion calculation formula, wherein the fusion calculation formula is H v =MultiHead(F o ,F v’ F v ), H r =MultiHead(F o ,F r’ F r ), H o =MultiHead(F o ,F o’ F o ), Among them, F v is the video feature, F v’ is the video feature label, F o is the object feature, F o’ is the object feature label, F r is the regional characteristics and, F r’ is the regional feature label, The output layer is used to output the fusion result, and the fusion result is in, is the output after encoding the video features, is the output after encoding the object features, is the output after encoding the regional features, where F OUTPUT =MultiHead(F,H,H),F OUTPUT for or or F = cat(F v ,F o ,F r ), H = cat(H v ,H o ,H r ), cat is the splicing symbol, W 1 , W 2 , W 3 It is the weight generated by the training model during the fusion training process, and BN is the batch normalization symbol.

9. A video description device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the multi-feature fusion video description method according to any one of claims 1 to 4 is implemented.

10. A computer-readable storage medium storing a computer program. It is characterized in that When the computer program is executed by a processor, the multi-feature fusion video description method according to any one of claims 1 to 4 is implemented.