Video description method, device and storage medium

Abstract

The present invention provides a video description method, device and storage medium, belonging to the field of image processing technology, the method comprises: S1: importing multiple video data, and performing a first fusion analysis on each video data to obtain a first fusion feature; S2: constructing a training model, training the training model according to multiple first fusion features to obtain a target video description model; S3: performing a second fusion analysis on each video data to obtain a second fusion feature; S4: performing video description on multiple second fusion features through the target video description model to obtain a video description result. The present invention can generate descriptions without manual supervision, reducing labor costs, and has a higher description accuracy than traditional video description models at the picture feature level, increases the closeness of feature relationships, and is conducive to improving the accuracy of the video description model.

Description

Technical Field

[0001] The present invention mainly relates to the technical field of image processing, and in particular to a video description method, device and storage medium. Background Art

[0002] The challenges currently faced by video description mainly include the accuracy of machine translation and the cost of a large number of manual annotations. The commonly used video description method is to extract features from the video, input the features into the decoding model to output the description, and then compare the description predicted by the model with the manual description, so as to train the video description model to generate a better description. In this process, the extracted features are not well utilized, and the connection between the features is not very close, resulting in inaccurate descriptions. Secondly, traditional models require manual supervision, which will lead to a large increase in labor costs. In addition, the training process of traditional models is trained at the text feature level, while video understanding is mainly interpreted at the image feature level, so training at the text level will reduce the accuracy of the model description. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a video description method, device and storage medium in view of the deficiencies in the prior art.

[0004] The technical solution of the present invention to solve the above technical problem is as follows: A video description method comprises the following steps:

[0005] S1: importing a video data set, the video data set including a plurality of video data, and performing a first fusion analysis on each of the video data to obtain a first fusion feature corresponding to each of the video data;

[0006] S2: constructing a training model, and training the training model according to the plurality of the first fusion features to obtain a target video description model;

[0007] S3: performing a second fusion analysis on each of the video data to obtain a second fusion feature corresponding to each of the video data;

[0008] S4: Performing video description on the plurality of the second fusion features through the target video description model to obtain a video description result.

[0009] Another technical solution of the present invention to solve the above technical problem is as follows: a video description device, comprising:

[0010] A first fusion analysis module is used to import a video data set, the video data set includes a plurality of video data, and perform a first fusion analysis on each of the video data to obtain a first fusion feature corresponding to each of the video data;

[0011] A model training module, used for constructing a training model, and training the training model according to the plurality of the first fusion features to obtain a target video description model;

[0012] A second fusion analysis module is used to perform a second fusion analysis on each of the video data to obtain a second fusion feature corresponding to each of the video data;

[0013] The video description result obtaining module is used to perform video description on the plurality of the second fusion features through the target video description model to obtain a video description result.

[0014] Another technical solution of the present invention to solve the above technical problem is as follows: A video description device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the video description method described above is implemented.

[0015] Another technical solution of the present invention to solve the above technical problem is as follows: a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the video description method as described above is implemented.

[0016] The beneficial effects of the present invention are as follows: a first fusion feature is obtained by respectively performing a first fusion analysis on each video data, a training model is constructed, the training model is trained according to multiple first fusion features to obtain a target video description model, a second fusion feature is obtained by respectively performing a second fusion analysis on each video data, and a video description result is obtained by describing the video of multiple second fusion features through the target video description model. The description can be trained and generated without manual supervision, thereby reducing labor costs. The description accuracy at the picture feature level is higher than that of traditional video description models, the closeness of feature relationships is increased, and the accuracy of the video description model is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of a flow chart of a video description method provided by an embodiment of the present invention;

[0018] Figure 2 A module block diagram of a video description device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0020] Figure 1 A schematic diagram of a flow chart of a video description method provided by an embodiment of the present invention.

[0021] like Figure 1 As shown, a video description method comprises the following steps:

[0022] S1: importing a video data set, the video data set including a plurality of video data, and performing a first fusion analysis on each of the video data to obtain a first fusion feature corresponding to each of the video data;

[0023] S2: constructing a training model, and training the training model according to the plurality of the first fusion features to obtain a target video description model;

[0024] S3: performing a second fusion analysis on each of the video data to obtain a second fusion feature corresponding to each of the video data;

[0025] S4: Performing video description on the plurality of the second fusion features through the target video description model to obtain a video description result.

[0026] It should be understood that the definition V={v1, v2, ..., vi} represents a video data set, and vi represents a video (ie, the video data).

[0027] In the above embodiment, a first fusion feature is obtained by performing a first fusion analysis on each video data, a training model is constructed, the training model is trained according to multiple first fusion features to obtain a target video description model, a second fusion feature is obtained by performing a second fusion analysis on each video data, and a video description result is obtained by performing a video description of multiple second fusion features using the target video description model. This method does not require manual supervision to train and generate descriptions, reduces labor costs, and has higher description accuracy at the image feature level than traditional video description models, increases the closeness of feature relationships, and is conducive to improving the accuracy of the video description model.

[0028] Optionally, as an embodiment of the present invention, in step S1, the process of performing a first fusion analysis on each of the video data to obtain a first fusion feature corresponding to each of the video data includes:

[0029] Based on a convolutional neural network, performing a first feature extraction on each of the video data through the convolutional neural network to obtain a first 2D visual feature corresponding to each of the video data and a first 3D motion feature corresponding to each of the video data;

[0030] Perform first gated fusion on each of the first 2D visual features and the first 3D motion features corresponding to each of the video data to obtain a first fused feature corresponding to each of the video data.

[0031] It should be understood that the Convolutional Neural Networks (CNN) is a type of feedforward neural network that includes convolution calculations and has a deep structure, and is one of the representative algorithms of deep learning. Convolutional neural networks have representation learning capabilities and can perform shift-invariant classification on input information according to their hierarchical structure, so they are also called "Shift-Invariant Artificial Neural Networks (SIANN)".

[0032] It should be understood that the video (i.e., the video data) in the data set (i.e., the video data set) is input into the convolutional neural network (i.e., the first convolutional neural network) to extract 2D visual features (i.e., the first 2D visual features) and 3D motion features (i.e., the first 3D motion features), and R = {r1, r2, ..., rm} is defined to represent the extracted 2D visual feature set, rm represents the 2D features of the corresponding video (i.e., the first 2D visual features), and F = {f1, f2, ..., fm} is defined to represent the extracted 3D motion feature set, fi represents the 3D features of the corresponding video (i.e., the first 3D motion features), and then the extracted 2D features (i.e., the first 2D visual features) and 3D features (i.e., the first 3D motion features) are gated and fused to obtain the fused features xi (i.e., the first fused features).

[0033] In the above embodiment, the first fusion feature is obtained by performing the first fusion analysis on each video data, which increases the closeness of the feature relationship and is beneficial to improving the accuracy of the video description model.

[0034] Optionally, as an embodiment of the present invention, the process of performing first gated fusion on each of the first 2D visual features and the first 3D motion features corresponding to each of the video data to obtain a first fused feature corresponding to each of the video data includes:

[0035] The fusion features of each of the first 2D visual features and the first 3D motion features corresponding to each of the video data are calculated by the first formula to obtain the first fusion features corresponding to each of the video data. The first formula is:

[0036] xi=σ(W*rm+b)*fm+fm,

[0037] Among them, xi is the first fusion feature, σ is the nonlinear activation function, W and b are both learning parameters, rm is the 2D first visual feature, and fm is the first 3D motion feature.

[0038] It should be understood that xi refers to the first fusion feature corresponding to the mth video data, where i=m, rm refers to the 2D first visual feature corresponding to the mth video data, and fm refers to the first 3D motion feature corresponding to the mth video data.

[0039] In the above embodiment, the first fusion feature is obtained by respectively calculating the fusion features of each first 2D visual feature and the first 3D motion feature through the first formula, which increases the closeness of the feature relationship and is beneficial to improving the accuracy of the video description model.

[0040] Optionally, as an embodiment of the present invention, the process of step S2 includes:

[0041] Constructing a training network, wherein the training network includes a visual feature reconstruction network and an original video description model;

[0042] Based on an LSTM long short-term memory network, each of the first fusion features is decoded by the LSTM long short-term memory network to obtain hidden state information corresponding to each of the first fusion features;

[0043] Reconstructing features of each of the hidden state information respectively through the visual feature reconstruction network to obtain second 2D visual features corresponding to each of the first fusion features and second 3D motion features corresponding to each of the first fusion features;

[0044] respectively calculating the similarity loss of each of the second 2D visual features and the first 2D visual features corresponding to each of the video data, to obtain a 2D feature similarity loss value corresponding to each of the video data;

[0045] Respectively calculating the similarity loss of each of the second 3D motion features and the first 3D motion features corresponding to each of the video data, to obtain a 3D feature similarity loss value corresponding to each of the video data;

[0046] Determine whether all 2D feature similarity loss values ​​and all 3D feature similarity loss values ​​are greater than a preset judgment threshold. If so, update the parameters of the original video description model according to the multiple 2D feature similarity loss values ​​and the multiple 3D feature similarity loss values, and return to step S2, and reconstruct the features of each hidden state information through the visual feature reconstruction network again; if not, use the original video description model as the target video description model.

[0047] It should be understood that the LSTM long short-term memory network is a time recurrent neural network, which is specially designed to solve the long-term dependency problem of general RNN (recurrent neural network). All RNNs have a chain form of repeated neural network modules. In the standard RNN, this repeated structural module has only a very simple structure, such as a tanh layer.

[0048] It should be understood that the feature xi (ie, the first fused feature) is input into the LSTM decoding to obtain the hidden state hi (ie, the hidden state information), and the hidden state (ie, the hidden state information) represents the text description of xi (ie, the first fused feature).

[0049] Specifically, the hidden state hi (i.e., the hidden state information) is input into the visual feature reconstruction network to generate visual features 2D features (i.e., the second 2D visual features) and 3D features (i.e., the second 3D motion features) similar to the corresponding moment, and P = {p1, p2, ..., pm} is defined to represent the reconstructed 2D visual features, pm represents the corresponding reconstructed visual features (i.e., the second 2D visual features), and K = {k1, k2, ..., km} is defined to represent the reconstructed 3D motion features, pm represents the corresponding reconstructed motion features (i.e., the second 3D motion features). The similarity loss of pm and km is calculated with the 2D features (i.e., the first 2D visual features) and the 3D features (i.e., the first 3D motion features) in step S1. If the loss (i.e., the 2D feature similarity loss value or the 3D feature similarity loss value) is greater than a set threshold (i.e., the preset judgment threshold), the loss (i.e., the 2D feature similarity loss value or the 3D feature similarity loss value) is transmitted back to the video description model (i.e., the original video description model). The model training is stopped until the loss (i.e., the 2D feature similarity loss value or the 3D feature similarity loss value) is less than the set threshold (i.e., the preset judgment threshold). At this time, the optimal video description model (i.e., the target video description model) is obtained through training.

[0050] In the above embodiment, the target video description model is obtained by training the training model with multiple first fusion features. It is possible to train and generate descriptions without manual supervision, and the video description model can be trained in a self-supervised manner. Moreover, the model trained at the image feature level has a higher description accuracy than the traditional video description model.

[0051] Optionally, as an embodiment of the present invention, the process of respectively calculating the similarity loss of each of the second 2D visual features and the first 2D visual features corresponding to each of the video data to obtain the 2D feature similarity loss value corresponding to each of the video data includes:

[0052] The similarity loss of each of the second 2D visual features and the first 2D visual features corresponding to each of the video data is calculated by the second formula to obtain a 2D feature similarity loss value corresponding to each of the video data. The second formula is:

[0053] L1=cos(rm,pm),

[0054] Among them, L1 is the 2D feature similarity loss value, rm is the first 2D visual feature, γ is the parameter, pm is the second 2D visual feature, and cos is the cosine distance function.

[0055] It should be understood that L1 can also be expressed as L1m, which refers to the 2D feature similarity loss value corresponding to the mth video data, rm can be the first 2D visual feature corresponding to the mth video data, and pm can be the second 2D visual feature corresponding to the mth video data.

[0056] In the above embodiment, the similarity loss of each second 2D visual feature and the first 2D visual feature is calculated by the second formula to obtain the 2D feature similarity loss value, which can generate descriptions for training without manual supervision, thereby reducing labor costs and improving the accuracy of descriptions.

[0057] Optionally, as an embodiment of the present invention, the process of respectively calculating the similarity loss of each of the second 3D motion features and the first 3D motion features corresponding to each of the video data to obtain a 3D feature similarity loss value corresponding to each of the video data includes:

[0058] The similarity loss of each of the second 3D motion features and the first 3D motion features corresponding to each of the video data is calculated respectively by the third formula to obtain the 3D feature similarity loss value corresponding to each of the video data. The third formula is:

[0059] L2=cos(fm,km),

[0060] Among them, L2 is the 3D feature similarity loss value, fm is the first 3D motion feature, km is the second 3D motion feature, and cos is the cosine distance function.

[0061] It should be understood that L2 can also be expressed as L2m, which refers to the 3D feature similarity loss value corresponding to the mth video data, fm can be the first 3D motion feature corresponding to the mth video data, and km can be the second 3D motion feature corresponding to the mth video data.

[0062] In the above embodiment, the similarity loss of each second 3D motion feature and the first 3D motion feature is calculated respectively by the third formula to obtain the 3D feature similarity loss value, which can generate descriptions for training without manual supervision, thereby reducing labor costs and improving the accuracy of descriptions.

[0063] Optionally, as an embodiment of the present invention, the process of step S3 includes:

[0064] Performing a second feature extraction on each of the video data by using the convolutional neural network to obtain a third 2D visual feature corresponding to each of the video data and a third 3D motion feature corresponding to each of the video data;

[0065] Perform second gated fusion on each of the third 2D visual features and the third 3D motion features corresponding to each of the video data to obtain a second fused feature corresponding to each of the video data.

[0066] It should be understood that the video of the video data set (i.e., the video data) is re-input into the convolutional neural network (i.e., the second convolutional neural network) to extract 2D visual features (i.e., the third 2D visual features) and 3D motion features (i.e., the third 3D motion features), and then these two features are gated and fused to obtain the fused feature xi (i.e., the second fused feature), and input into the video description model (i.e., the target video description model) to obtain the output text description (i.e., the video description result).

[0067] In the above embodiment, the second fusion analysis of each video data is performed separately to obtain the second fusion feature, which can generate descriptions without manual supervision, thereby reducing labor costs. Moreover, the description accuracy at the image feature level is higher than that of the traditional video description model, and the closeness of the feature relationship is increased, which is beneficial to improving the accuracy of the video description model.

[0068] Figure 2 A module block diagram of a video description device provided by an embodiment of the present invention.

[0069] Optionally, as another embodiment of the present invention, Figure 2 As shown, a video description device comprises:

[0070] A first fusion analysis module is used to import a video data set, the video data set includes a plurality of video data, and perform a first fusion analysis on each of the video data to obtain a first fusion feature corresponding to each of the video data;

[0071] A model training module, used for constructing a training model, and training the training model according to the plurality of the first fusion features to obtain a target video description model;

[0072] A second fusion analysis module is used to perform a second fusion analysis on each of the video data to obtain a second fusion feature corresponding to each of the video data;

[0073] The video description result obtaining module is used to perform video description on the plurality of the second fusion features through the target video description model to obtain a video description result.

[0074] Optionally, another embodiment of the present invention provides a video description device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the video description method described above is implemented. The device may be a computer or other device.

[0075] Optionally, another embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the video description method as described above is implemented.

[0076] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0077] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0078] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0079] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0080] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0081] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0082] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A video description method, characterized in that: The steps include: S1: importing a video data set, the video data set including a plurality of video data, and performing a first fusion analysis on each of the video data to obtain a first fusion feature corresponding to each of the video data; S2: constructing a training model, and training the training model according to the plurality of the first fusion features to obtain a target video description model; S3: performing a second fusion analysis on each of the video data to obtain a second fusion feature corresponding to each of the video data; S4: Performing video description on the plurality of the second fusion features through the target video description model to obtain a video description result; In step S1, the process of performing a first fusion analysis on each of the video data to obtain a first fusion feature corresponding to each of the video data includes: Based on a convolutional neural network, performing a first feature extraction on each of the video data through the convolutional neural network to obtain a first 2D visual feature corresponding to each of the video data and a first 3D motion feature corresponding to each of the video data; Performing first gated fusion on each of the first 2D visual features and the first 3D motion features corresponding to each of the video data to obtain a first fused feature corresponding to each of the video data; The process of performing first gated fusion on each of the first 2D visual features and the first 3D motion features corresponding to each of the video data to obtain a first fusion feature corresponding to each of the video data comprises: The fusion features of each of the first 2D visual features and the first 3D motion features corresponding to each of the video data are calculated by the first formula to obtain the first fusion features corresponding to each of the video data. The first formula is: xi=σ(W*rm+b)*fm+fm, Among them, xi is the first fusion feature, σ is the nonlinear activation function, W and b are learning parameters, rm is the 2D first visual feature, and fm is the first 3D motion feature; The process of step S3 includes: Performing a second feature extraction on each of the video data by using the convolutional neural network to obtain a third 2D visual feature corresponding to each of the video data and a third 3D motion feature corresponding to each of the video data; Perform second gated fusion on each of the third 2D visual features and the third 3D motion features corresponding to each of the video data to obtain a second fused feature corresponding to each of the video data.

2. The video description method according to claim 1, characterized in that: The process of step S2 includes: Constructing a training network, wherein the training network includes a visual feature reconstruction network and an original video description model; Based on an LSTM long short-term memory network, each of the first fusion features is decoded by the LSTM long short-term memory network to obtain hidden state information corresponding to each of the first fusion features; Reconstructing features of each of the hidden state information respectively through the visual feature reconstruction network to obtain second 2D visual features corresponding to each of the first fusion features and second 3D motion features corresponding to each of the first fusion features; respectively calculating the similarity loss of each of the second 2D visual features and the first 2D visual features corresponding to each of the video data, to obtain a 2D feature similarity loss value corresponding to each of the video data; Respectively calculating the similarity loss of each of the second 3D motion features and the first 3D motion features corresponding to each of the video data, to obtain a 3D feature similarity loss value corresponding to each of the video data; Determine whether all 2D feature similarity loss values ​​and all 3D feature similarity loss values ​​are greater than a preset judgment threshold. If so, update the parameters of the original video description model according to the multiple 2D feature similarity loss values ​​and the multiple 3D feature similarity loss values, and return to step S2, and reconstruct the features of each hidden state information through the visual feature reconstruction network again; if not, use the original video description model as the target video description model.

3. The video description method according to claim 2, characterized in that: The process of respectively calculating the similarity loss of each of the second 2D visual features and the first 2D visual features corresponding to each of the video data to obtain the 2D feature similarity loss value corresponding to each of the video data includes: The similarity loss of each of the second 2D visual features and the first 2D visual features corresponding to each of the video data is calculated by the second formula to obtain a 2D feature similarity loss value corresponding to each of the video data. The second formula is: L1=cos(rm,pm), Among them, L1 is the 2D feature similarity loss value, rm is the first 2D visual feature, γ is the parameter, pm is the second 2D visual feature, and cos is the cosine distance function.

4. The video description method according to claim 2, characterized in that: The process of respectively calculating the similarity loss of each of the second 3D motion features and the first 3D motion features corresponding to each of the video data to obtain the 3D feature similarity loss value corresponding to each of the video data includes: The similarity loss of each of the second 3D motion features and the first 3D motion features corresponding to each of the video data is calculated respectively by the third formula to obtain the 3D feature similarity loss value corresponding to each of the video data. The third formula is: L2=cos(fm,km), Among them, L2 is the 3D feature similarity loss value, fm is the first 3D motion feature, km is the second 3D motion feature, and cos is the cosine distance function.

5. A video description device, characterized in that: include: A first fusion analysis module is used to import a video data set, the video data set includes a plurality of video data, and perform a first fusion analysis on each of the video data to obtain a first fusion feature corresponding to each of the video data; A model training module, used for constructing a training model, and training the training model according to the plurality of the first fusion features to obtain a target video description model; A second fusion analysis module is used to perform a second fusion analysis on each of the video data to obtain a second fusion feature corresponding to each of the video data; A video description result obtaining module, used for performing video description on the plurality of the second fusion features through the target video description model to obtain a video description result; In the first fusion analysis module, the process of performing a first fusion analysis on each of the video data to obtain a first fusion feature corresponding to each of the video data includes: Based on a convolutional neural network, performing a first feature extraction on each of the video data through the convolutional neural network to obtain a first 2D visual feature corresponding to each of the video data and a first 3D motion feature corresponding to each of the video data; Performing first gated fusion on each of the first 2D visual features and the first 3D motion features corresponding to each of the video data to obtain a first fused feature corresponding to each of the video data; In the first fusion analysis module, the process of performing first gated fusion on each of the first 2D visual features and the first 3D motion features corresponding to each of the video data to obtain the first fusion features corresponding to each of the video data includes: The fusion features of each of the first 2D visual features and the first 3D motion features corresponding to each of the video data are calculated by the first formula to obtain the first fusion features corresponding to each of the video data. The first formula is: xi=σ(W*rm+b)*fm+fm, Among them, xi is the first fusion feature, σ is the nonlinear activation function, W and b are learning parameters, rm is the 2D first visual feature, and fm is the first 3D motion feature; The second fusion analysis module is specifically used for: Performing a second feature extraction on each of the video data by using the convolutional neural network to obtain a third 2D visual feature corresponding to each of the video data and a third 3D motion feature corresponding to each of the video data; Perform second gated fusion on each of the third 2D visual features and the third 3D motion features corresponding to each of the video data to obtain a second fused feature corresponding to each of the video data.

6. A video description system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the video description method according to any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the video description method according to any one of claims 1 to 4 is implemented.

Images