Concave decoding structure and implementation method for minority dance dense video description

Abstract

The application discloses a concave decoding structure and an implementation method for minority dance dense video description, and belongs to the technical field of deep learning video understanding. The concave decoding structure comprises a forward propagation mapping function, a word attention capture layer and a multi-layer video feature screening mechanism. In detail, the forward propagation mapping function is used to deepen the network and enhance the extraction capability of deep semantic information, thereby enriching the semantic information. Meanwhile, a text feature preservation branch is added to construct the concave decoding structure, and the historical text features are used to make up for the semantic information loss in the feature transmission process, so that the context connection is enhanced, the accuracy of the description of objects, scenes and the like appearing in a long video is improved, and a description result more consistent with the video content is obtained. The application is suitable for the fields of minority dance video description, short video content review, security monitoring and the like.

Description

Technical Field

[0001] This invention relates to the field of deep learning video understanding technology, specifically to a concave decoding structure and implementation method for dense video description of ethnic minority dances. Background Technology

[0002] The rapid development of information technology has brought both opportunities and challenges to the dissemination of ethnic minority traditional culture. In the context of the new era, the methods of protecting and inheriting ethnic minority culture should also evolve with the times. How to utilize the internet to promote ethnic minority traditional culture is a question worthy of in-depth exploration. For example, in ethnic minority dance videos, dancers will arrange different scenes and wear different costumes according to different ethnic groups. However, computer technology for describing the scenes and details of ethnic dance videos is not mature enough to achieve a full understanding of the video content. Existing dense video description methods can be divided into two categories: decoders based on convolutional neural networks and decoders based on attention mechanisms.

[0003] Most dense video description methods based on convolutional neural networks employ Long Short-Term Memory (LSTM) networks to output corresponding text descriptions. Specifically, in patent application CN110709855A, titled "Technology for Dense Video Description," LSTM is used for encoding and decoding to process sequences. Based on an S2VT bidirectional encoder, two LSTMs are used to encode visual features, and then another LSTM is used to decode historical and future visual features to generate descriptive statements. In patent application CN111368142A, titled "A Method for Dense Video Event Description Based on Generative Adversarial Networks," video content-related event features obtained from an LSTM temporal model are fused with corresponding video frames and input into a natural language decoder composed of LSTMs, making the generated natural language statements more closely aligned with the video content.

[0004] The introduction of attention mechanisms has accelerated progress in the field of video understanding. Most dense video description methods based on attention mechanisms are based on the Transformer architecture, utilizing an encoder to generate a global encoding vector from video features. This encoded video feature vector is then directly input into a decoder for text generation. One invention patent application, CN113392717A, entitled "A Video Dense Description Generation Method Based on Temporal Feature Pyramids," utilizes an attention mechanism to construct the decoder. This decoder includes three sub-modules: self-attention, multi-terminal attention, and a feedforward network. This allows the decoder to filter appropriate global semantic information from dense video features, enabling it to generate more targeted descriptive statements. Another invention patent application, CN114461851A, entitled "An Interactive Transformer Multimodal Video Dense Event Description Algorithm," proposes an interactive Transformer architecture. This architecture utilizes interactive attention to construct the decoder. After extracting word sequence features using a self-attention mechanism with a mask, these features are fused and concatenated with the video features obtained from the encoder. In the task of generating description sequences, a more accurate video description result is obtained through iteration.

[0005] Most existing dense video description methods utilize attention mechanisms in their decoding to filter appropriate contextual information for text generation. However, during training, various confounding factors, such as information loss, are difficult to avoid, leading to semantic errors and thus reducing the performance of dense video description. Furthermore, the generated descriptions often deviate from the video content, lacking logical spatial descriptions, which is detrimental to the description of ethnic minority dance videos. Therefore, how to effectively utilize semantic information to generate more logical spatial descriptions that fit the video content, and better apply them to the description of ethnic minority dance videos and short video content review, has become an urgent problem to be solved. Summary of the Invention

[0006] The purpose of this invention is to provide a concave decoding structure and implementation method for dense video description of ethnic minority dances. This structure can provide a logical description of the dancers' venue layout, clothing, etc., containing spatial positional relationships, thereby improving the accuracy of dense video description and generating descriptive statements that better fit the video content.

[0007] On one hand, this application proposes a concave decoding structure for dense video description of ethnic minority dances, including a forward propagation mapping function, a word attention capture layer, and a multi-layer video feature filtering mechanism. First, historical text features are used as input to the concave decoding. The first forward propagation mapping function enriches the semantic information of the historical text features. Then, the word attention capture layer establishes the correlation of the historical text features and passes them to the second forward propagation mapping function to combine words with high correlation. Then, the obtained text features and the dense video features encoded as input are used as input to the multi-layer video feature filtering mechanism to obtain the similarity between the text vector and the video feature vector. After assigning different weights to the word vectors according to the video content, the third forward propagation mapping function and the text feature preservation branch are used to fine-tune the text features, and finally, the generated descriptive text is output.

[0008] The concave decoding structure uses historical text features and dense video features As input, the final output is a dense video description text Y. t .

[0009] Furthermore, the first forward propagation mapping function specifically operates as follows: using the first forward propagation mapping function to map historical text features. The semantic information in the input text is enriched, and the historical text features of the input are preserved, resulting in text features with more information. As shown in equation (1):

[0010]

[0011] Among them, F fully (·) represents the processing procedure of the first forward propagation mapping function. This represents the fusion process; t represents the current time, which is a natural number. When t equals 0, This is the start symbol.

[0012] Furthermore, the word attention capture layer captures the text features. Establishing relevance involves: obtaining text features by acquiring the similarity between words in the global text vector and filtering appropriate contextual information. As shown in equation (2):

[0013]

[0014] Among them, A self (·) indicates the processing of the word attention capture layer.

[0015] Furthermore, the second forward propagation mapping function is used to process the text features that have been filtered through the word attention capture layer. By combining words with high relevance and adjusting the weights of each word vector in the text features, we obtain the text features. As shown in equation (3):

[0016]

[0017] Furthermore, the multi-layer video feature filtering mechanism specifically operates as follows: using dense video feature vectors... and text features As input, the original number of channels is divided into m layers. Correlation between video feature vectors and text feature vectors is obtained in different spaces, and words related to the video content are filtered out to obtain different spatial relationships in the video features, improving the logical consistency of the generated descriptive text. Then, the text features selected from each layer are fused, utilizing text features to preserve branches and reduce the problem of insufficient description of video content caused by semantic loss during transmission, resulting in a preliminary generated descriptive text vector. As shown in equation (4):

[0018]

[0019] Among them, A multi (·) indicates the processing procedure of the multi-layer video feature filtering mechanism.

[0020] Furthermore, descriptive text is generated through the joint processing of the third forward propagation mapping function and the text feature preservation branch. Specifically, the descriptive text vector is processed by the third forward propagation mapping function. Enhancement is performed, and initial text features are introduced to strengthen contextual connections, improve the fluency of the generated descriptive text, and highlight words highly relevant to the video content, resulting in the final output video descriptive text Y. t As shown in equation (5):

[0021]

[0022] The concave decoding structure alternates between the forward propagation mapping function and the attention mechanism to filter out descriptive words that are highly relevant to video features. By fusing historical text features, it prevents the loss of semantic information during the filtering process and enhances the semantic information, which is beneficial for obtaining accurate and rich video text description results in dense video description tasks.

[0023] On the other hand, this application proposes a concave decoding implementation method for dense video description of ethnic minority dances, including:

[0024] Step 1: Extract historical text features As the initial input for concave decoding, The specific form is The characteristic sequence, where, The size can be 1*C; then add the commonly used position encoding vector P. t-1 Embedded into historical text vectors From Among them, P t-1 The specific form is P t-1 ={p1,p2,p3,...pC}, with a size of 1*C; as shown in equations (6) and (7):

[0025]

[0026]

[0027] Where l represents the position of the word in the sentence, i represents the dimension of the position of the word, a natural number starting from 0, j represents whether the encoded position of the word is odd or even, and C is the dimension of the position encoding vector;

[0028] Step 2: Apply the first forward propagation mapping function to the historical text features after adding standard position encoding. The process is as follows: As shown in equations (8) to (12), the words in the historical text feature sequence are first converted into information vectors of the same size using the Layer Norm method, as shown in the following equation:

[0029]

[0030] Where a is the weight, b is the bias, and μ is the scalar in the normalization. Standard deviation;

[0031] The obtained text feature vectors are processed through a fully connected network to learn complex text feature representations. Then, an activation function is used to provide non-linearity, enhancing the expressive power of the dense video description model; as shown in the following equation:

[0032]

[0033]

[0034] Among them, F f This is the processing function for a fully connected network, which has three layers. w1 and b1 are the weights and biases between the input layer and the hidden layer, respectively, and w2 and b2 are the weights and biases between the hidden layer and the output layer, respectively.

[0035] To reduce overfitting of the model, the feature vectors processed by the fully connected network are randomly deleted, as shown in equation (11):

[0036]

[0037]

[0038] in, It is a function that randomly deletes features from a feature vector. In The probability of random deletion. Indicates a fusion operation;

[0039] Historical text features After normalization, fully connected processing, and random deletion in the first forward propagation mapping function, the historical text features of the input are preserved, resulting in enhanced text features.

[0040] Step 3: Transfer the text features As input to the word attention capture layer; firstly, The text features are obtained by processing the text using the normalization function from step 2. Then, the input is processed by the word attention capture function, as shown in equations (13) to (16):

[0041]

[0042]

[0043] in, and All are text feature vectors The results after performing different linear transformations for The dimension of A s The processing procedure for the word attention capture function. It is the softmax function;

[0044]

[0045]

[0046] Finally, text features Perform the random deletion process in step 2 to reduce overfitting, and then apply the input features of the word attention capture layer. Maintain, obtain This word attention capture layer filters appropriate contextual information and adjusts the attention of key words in the text features;

[0047] Step 4: Process the obtained text feature sequence Perform step 2 to adjust the word focus; As input to the second forward propagation mapping function, the semantically enhanced text features are obtained after processing. As shown in equation (17):

[0048]

[0049] Step 5: Enhance the semantics of the text information With video feature encoding vector As input to the multi-layer video feature filtering mechanism, a preliminary descriptive text vector is obtained. The multi-layer video feature filtering mechanism is expressed as shown in equations (18) to (20): The detailed steps are as follows:

[0050] First, analyze the text feature vectors. The text features are obtained by processing the text using the normalization function from step 2. As one of the inputs to the multi-layer video feature filtering mechanism; the original number of channels is then divided into m layers, and video feature vectors are obtained in different spaces respectively. With text feature vectors The correlation between them is determined, and then the results of each layer are merged together to filter words related to the video content; the processing procedure is shown in equation (18):

[0051]

[0052] Among them, A m This is the processing procedure for a multi-layer video feature filtering mechanism. and All are layered video feature vectors The results after performing different linear transformations yes The number of feature channels is increased; video and text features are processed through a multi-layer video feature filtering mechanism to obtain text features with different temporal logics from different dimensions. The input of the multi-layer video feature filtering mechanism is then fused to preserve text features and improve the contextual coherence of the descriptive text. To increase the weight of words that can describe key video content, historical text features enhanced by the first forward propagation mapping function in step 2 are introduced. Reduce semantic loss during feature transfer and maintain the fluency of the descriptive text; as shown in equations (19) and (20):

[0053]

[0054]

[0055] The multi-layer video feature filtering mechanism obtains the similarity between the video feature vector to be processed at the current moment and the text feature vector generated at the previous moment under different spatial conditions, adjusts the weights of the word vectors describing the current video, and initially generates a logical descriptive statement with spatial information that can express the video content.

[0056] Step 6: Process the initially generated descriptive text vector Perform semantic enhancement; first, follow step 2 to enhance text features. After processing, the semantically enhanced description text is obtained through the third forward propagation mapping function. To enhance contextual relevance, the input features for this step are preserved, while the historical text features obtained in step 1 are also incorporated. The final generated description text Y is obtained by blending it with the enhanced description text. t This aims to increase the confidence of words that accurately describe video content and improve the accuracy of generating descriptive statements for objects and scenes appearing in long videos; as shown in equations (21) and (22):

[0057]

[0058]

[0059] The dense video description text Y t It can fit the video content well and has strong expressive power. When applied to intensive video description tasks, it can improve model performance and output accurate and rich text description results.

[0060] The advantages of the above technical solutions adopted in this invention compared with the prior art are as follows:

[0061] (1) Suitable for dense video description tasks

[0062] The concave decoding structure constructed in this invention optimizes the ability to filter contextual semantic information, resulting in accurate and rich dense video description text. After video features are encoded by networks such as Transformer and Long Short-Term Memory (LSTM), this decoding can be used to generate accurate video description text, thereby improving the performance of dense description networks.

[0063] (2) Applicable to video description systems for ethnic minority dances

[0064] The concave decoding structure constructed in this invention optimizes the ability to filter contextual semantic information, resulting in accurate and rich dense video description text with spatial logic. This decoding can be applied to dense video description models to provide relatively accurate text descriptions of backgrounds, costumes, etc., in ethnic minority dance videos, thereby improving people's understanding of ethnic minority dance culture and contributing to the promotion of traditional Chinese culture.

[0065] (3) Applicable to short video content review system

[0066] This invention can generate video description text with strong expressive power, enhance the performance of dense video description models, and the text information such as targets, behaviors, and attributes in the video description can serve as prompts for short video content review systems, effectively reducing the workload of manual review of video content, timely detecting illegal, irregular, and negative content in short videos, and constructing a healthy online environment.

[0067] (4) Applicable to outdoor security monitoring systems

[0068] The concave decoding structure constructed in this invention optimizes the ability to filter contextual semantic information, resulting in accurate and rich dense video description text. The textual information such as targets, behaviors, and attributes can serve as prompts for security monitoring systems, effectively reducing the workload of manual supervision and review of security monitoring, preventing dangerous behaviors such as drowning, suicide, and fighting, and effectively improving the working efficiency of security systems. Attached Figure Description

[0069] Figure 1 This is a schematic diagram of the overall framework of the concave decoding structure;

[0070] Figure 2 This is a schematic diagram illustrating the situation of ethnic minority dance videos in Implementation Example 1;

[0071] Figure 3 This is a diagram illustrating the short video content review process in Implementation Example 2;

[0072] Figure 4 This is a schematic diagram of the outdoor security monitoring scenario in Implementation Example 3. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit the application; that is, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0074] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0075] This invention proposes a concave decoding structure and implementation method for dense video description of ethnic minority dances. By adding a forward propagation mapping function, the semantic information input to the attention mechanism is enriched, optimizing the attention mechanism's ability to filter contextual semantic information and obtaining words more representative of the video content. Simultaneously, a concave structure is constructed to use historical semantic information to assist the downstream video description text generation task, improving the accuracy of generating descriptive statements for objects and scenes appearing in long videos. Therefore, by targeting more distinguishable features of different objects, it can accurately generate object details in the video. This improves the precision of dense video description and obtains accurate video text description results.

[0076] The technical terms appearing in this invention are explained as follows:

[0077] (1) Dense video features: After being processed by coding networks such as Transformer coding and Long Short Time Memory, the dense video features will be input into the multi-layer video feature filtering mechanism of concave decoding, which is responsible for providing dense video feature coding vectors.

[0078] (2) Historical text features: the start character and all dense video description text vectors output after concave decoding at the previous moment, with the addition of standard position encoding vectors.

[0079] (3) Forward Propagation Mapping Functions: In this invention, Layer Norm normalization and a fully connected network are used to process text features and output text feature vectors with rich semantic information. There are three forward propagation mapping functions: the first is located at the initial position of the concave decoding network to enrich the semantic information of historical text features; the second is located in the middle position of the concave decoding, after the word attention capture layer, to combine words with high correlation in the text features; and the third is located at the final output position of the concave decoding, after the multi-layer video feature filtering mechanism, to fine-tune the generated text description to obtain text features that are more in line with the video content.

[0080] (4) Word attention capture layer: It can filter the context information of text features. In this invention, the word weights are adjusted accordingly by obtaining the correlation between words in the text features.

[0081] (5) Multi-layer video feature filtering mechanism: It can query text information corresponding to video content, use multiple identical video feature filtering mechanisms to calculate the similarity between dense video features and text features, and output descriptive text that fits the current dense video content.

[0082] (6) Text feature preservation branch: It can preserve the text features of the previous time step and avoid the problem of semantic information loss during feature transfer. There can be two text feature preservation branches. One is to use the text features obtained after the forward propagation mapping function as the branch and finally fuse them with the output of the multi-layer video feature filtering mechanism. The other is to use the historical text features as the branch and fuse them with the final output of decoding as the final generated dense video description text.

[0083] (7) Concave decoding: including forward propagation mapping function, word attention capture layer and multi-layer video feature filtering mechanism, which together with text feature branches form a "concave" shaped decoding.

[0084] The constraints in this invention can be referenced as follows:

[0085] Unit layer number constraint:

[0086] (1) In the multi-layer video feature filtering mechanism, the number of layers can be set to m, where m∈{4,6,8}.

[0087] (2) In the forward propagation mapping function, the number of layers in the fully connected network can all be N, preferably N=3.

[0088] Feature size constraints:

[0089] (1) In concave decoding, the features are basically composed of a feature matrix (Ba, H, C) consisting of the number of training samples Ba, the number of video frames H, and the number of channels C.

[0090] (2) The size of the input dense video feature can be (25, H, 512).

[0091] (3) The size of the input historical text feature can be (25, H, 512).

[0092] (4) The output dense video description text feature size can be (25, H, 2086), where 2086 is the total number of words.

[0093] (5) In the multi-layer video feature selection mechanism, when the number of layers m = 8, the dimensions of the dense video features and text features are both (25, H, m, 64), which are used as input features for the 8 video feature selection mechanisms. The 8 video feature selection mechanisms have the same structure. The output descriptive text feature dimensions can be (25, H, 512).

[0094] (6) In the forward propagation mapping function, the feature size of the output of the fully connected network from layer 1 to layer 3 can be (512, 2048, 512), and the feature size can also be any size such as (512, 1024, 512) or (512, 2096, 512).

[0095] Structural constraints:

[0096] (1) The text feature preservation branches in the concave decoding structure each have S branches, and different functional modules can be used on the input to perform different processing on the features.

[0097] Example 1:

[0098] Scenes of ethnic minority dances

[0099] Applying this example to a scene depicting ethnic minority dances, the present invention's solution filters descriptive words that fit the video content, generating accurate and rich text descriptions. This text information can effectively identify spatially logical details such as the venue setup, costumes, and dance styles of ethnic dances, enabling a full understanding of the video content and contributing to the promotion of traditional Chinese culture. Figure 2 As shown: The input of the concave decoding structure is Figure 2 The video features a minority ethnic dance. The concave decoding structure describes the entire video and outputs a woman in ethnic minority costume dancing in front of a sofa in the living room.

[0100] Example 2:

[0101] Short video content review status

[0102] This example is applied to a short video content review system. The solution of this invention filters descriptive words that fit the video content, generating accurate and rich text descriptions. This text information can effectively prevent illegal, rule-breaking, and other negative content from appearing in short videos, contributing to a healthy online environment. Figure 3 As shown: The input of the concave decoding structure is Figure 3 The concave decoding structure describes the entire short video content and outputs a man taking a cigarette out of a box, putting it in his mouth, covering his hands with his hands, lighting the cigarette with a lighter, and sitting in a chair smoking.

[0103] Example 3:

[0104] Outdoor security monitoring scenarios

[0105] This example is applied to outdoor security monitoring scenarios. The solution of this invention filters descriptive words that fit the video content, generating accurate and rich text descriptions. This text information can effectively prevent outdoor accidents and improve the efficiency of reviewing surveillance videos. The outdoor security monitoring scenario is as follows: Figure 4 As shown: The input of the concave decoding structure is Figure 4 The outdoor surveillance video is described by the concave decoding structure, and the output is a person carrying a bucket and a child walking to the river. Then the person leaves with the bucket, and the boy walks to the river and steps into the river to play.

[0106] The foregoing description of specific exemplary embodiments of the invention is for illustrative and explanatory purposes. These descriptions are not intended to limit the invention to the precise forms disclosed, and it will be apparent that many changes and variations can be made in accordance with the foregoing teachings. The exemplary embodiments were chosen and described in order to explain the specific principles of the invention and its practical application, thereby enabling those skilled in the art to implement and utilize various different exemplary embodiments of the invention, as well as various different choices and variations. The scope of the invention is intended to be defined by the claims and their equivalents.

Claims

1. A concave decoding structure for dense video description of ethnic minority dances, characterized in that, This includes a forward propagation mapping function, a word attention capture layer, and a multi-layer video feature selection mechanism; First, historical text features are used as input for concave decoding. The first forward propagation mapping function is used to enrich the semantic information of the historical text features. Then, the relevance of the historical text features is established through a word attention capture layer. The features are then passed to the second forward propagation mapping function to combine words with high relevance. The resulting text features and the dense video features of the encoded input are then used as input for a multi-layer video feature filtering mechanism to obtain the similarity between the text vector and the video feature vector. After assigning different weights to the word vectors according to the video content, the third forward propagation mapping function and the text feature preservation branch are used to fine-tune the text features. Finally, the generated descriptive text is output. The first forward propagation mapping function specifically operates as follows: It applies the first forward propagation mapping function to the historical text features. The semantic information in the input text is enriched, and the historical text features of the input are preserved to obtain text features. As shown in equation (1): （1） in, This represents the processing procedure of the first forward propagation mapping function. Indicates the fusion process; Represents the current moment, which is a natural number. When equal to 0, This is the start symbol.

2. The concave decoding structure for dense video description of ethnic minority dances according to claim 1, characterized in that, The word attention capture layer captures the text features. Establishing relevance involves: obtaining text features by acquiring the similarity between words in the global text vector and filtering contextual information. As shown in equation (2): （2） in, This describes the processing steps of the word attention capture layer.

3. The concave decoding structure for dense video description of ethnic minority dances according to claim 2, characterized in that, Using the second forward propagation mapping function, the text features that have been filtered through the word attention capture layer are processed. By combining words with high relevance and adjusting the weights of each word vector in the text features, we obtain the text features. As shown in equation (3): （3）。 4. The concave decoding structure for dense video description of ethnic minority dances according to claim 1, characterized in that, The multi-layer video feature filtering mechanism specifically operates as follows: using dense video feature vectors and text features As input, the original number of channels is divided into The system first extracts the correlation between video feature vectors and text feature vectors in different spatial dimensions, filters words related to the video content, and obtains different spatial relationships in the video features. Then, it fuses the text features selected from each layer, uses text feature preservation branches, and obtains a preliminary descriptive text vector. As shown in equation (4): （4） in, This describes the processing steps of a multi-layer video feature filtering mechanism.

5. The concave decoding structure for dense video description of ethnic minority dances according to claim 4, characterized in that, The text vector is described by the third forward propagation mapping function. Enhancement is performed, and initial text features are introduced to strengthen contextual relationships, resulting in the final output video description text. As shown in equation (5): （5）。 6. A method for implementing concave decoding for dense video description of ethnic minority dances, characterized in that, include: Step 1: Extract historical text features As the initial input for concave decoding, The specific form is The feature sequence; then the position encoding vector. Embedded into historical text vectors From ,in, The specific form is As shown in equations (6) and (7): （6） （7） in, Indicates the position of a word in a sentence. The dimension representing the position of the word, a natural number starting from 0. This indicates whether the word's encoding position is odd or even. The dimension of the position encoding vector; Step 2: Apply the first forward propagation mapping function to the historical text features after adding standard position encoding. The process is as follows: As shown in equations (8) to (12), the words in the historical text feature sequence are first converted into information vectors of the same size using the Layer Norm method, as shown in the following equation: （8） in, As weight, For bias, For scalars in normalization, Standard deviation; The obtained text feature vectors are processed through a fully connected network, as shown in the following formula: （9） （10） in, This is the processing function for a fully connected network. and These represent the weights and biases between the input layer and the hidden layer. and These represent the weights and biases between the hidden layer and the output layer, respectively. The feature vectors processed by the fully connected network are randomly deleted, as shown in equation (11): （11） （12） in, It is a function that randomly deletes features from a feature vector. In The probability of random deletion. Indicates a fusion operation; Step 3: Transfer the text features As input to the word attention capture layer; firstly, The text features are obtained by processing the text using the normalization function from step 2. Then, the input is processed in the word attention capture function, as shown in equations (13) to (16): （13） （14） in, , and All are text feature vectors The results after performing different linear transformations for dimensionality The processing procedure for the word attention capture function. It is the softmax function; （15） （16） Finally, text features Perform the random deletion process in step 2, and then apply the input features of the word attention capture layer. Maintain, obtain ; Step 4: Process the obtained text feature sequence Perform all operations in step 2; As input to the second forward propagation mapping function, the semantically enhanced text features are obtained after processing. As shown in equation (17): （17） Step 5: Enhance the semantics of the text information With video feature encoding vector As input to the multi-layer video feature filtering mechanism, a preliminary descriptive text vector is obtained. ; Step 6: Process the initially generated descriptive text vector Perform semantic enhancement; first, perform all operations in step 2 on the text features. After processing, the semantically enhanced description text is obtained through the third forward propagation mapping function. .

7. The concave decoding implementation method for dense video description of ethnic minority dances according to claim 6, characterized in that, The implementation steps of the multi-layer video feature filtering mechanism are as follows: First, analyze the text feature vectors. The text features are obtained by processing the text using the normalization function from step 2. As one of the inputs to the multi-layer video feature filtering mechanism; the original number of channels is then divided into Layers, acquiring video feature vectors in different spaces. With text feature vectors The correlation between them is determined, and then the results of each layer are merged together to filter words related to the video content; the processing procedure is shown in equation (18): （18） in, This is the processing procedure for a multi-layer video feature filtering mechanism. and All are layered video feature vectors The results after performing different linear transformations yes The number of feature channels; Introducing historical text features enhanced by the first forward propagation mapping function in step 2. To maintain the fluency of the descriptive text, as shown in equations (19) and (20): （19） （20）。 8. The concave decoding implementation method for dense video description of ethnic minority dances according to claim 6, characterized in that, In step 6, historical text features are... The final generated description text is obtained by blending it with the enhanced description text. As shown in equations (21) and (22): （21） （22）。

Images