Training methods for title generation models, as well as title generation methods, devices, and media.

Abstract

This specification discloses a training method, a title generation method, an apparatus, a storage medium, and an electronic device for a title generation model. The method includes: acquiring training text of a sample video segment, the training text including sample segment text of the sample video segment and prefix text of the sample segment text, the prefix text being text annotated with keywords in the masked sample segment text; inputting the training text into a title generation model, encoding the training text using an attention mechanism to obtain semantic features of the training text; predicting based on the semantic features of the training text using the title generation model, outputting predicted keywords in the sample segment text and predicted titles of the sample video segment; and training the title generation model based on a first difference information between the predicted keywords and the annotated keywords, and a second difference information between the predicted title and the annotated title of the sample video segment.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a training method for a title generation model, a title generation method, an apparatus, a storage medium, and an electronic device. Background Technology

[0002] With the development of computer technology, more and more users are learning by watching live streams. For example, users can learn about finance by watching live streams hosted by financial service platforms. If a user misses a live stream, the platform also provides replay services, allowing the user to continue learning by watching the replay video.

[0003] In related technologies, since live replay videos are often long and usually contain some invalid information, technicians often divide the live replay videos into multiple video segments and manually set titles for each video segment. These titles are used to attract users to watch the corresponding video segments, thereby saving users' time.

[0004] However, this method of manually setting titles is inefficient, and a more intelligent title generation method is urgently needed. Summary of the Invention

[0005] This specification provides a training method for a title generation model, a title generation method, an apparatus, a storage medium, and an electronic device, which can train a title generation model to generate titles for video clips, thereby improving the efficiency of setting titles for video clips.

[0006] On the one hand, embodiments of this specification provide a training method for a title generation model, including:

[0007] The training text of the sample video segment is obtained. The training text includes the sample segment text of the sample video segment and the prefix text of the sample segment text. The prefix text is the text in the sample segment text that is masked and marked with keywords.

[0008] The training text is input into the title generation model, and the title generation model encodes the training text based on an attention mechanism to obtain the semantic features of the training text.

[0009] The title generation model predicts the keywords in the sample segmented text and the predicted title of the sample video clip based on the semantic features of the training text.

[0010] The title generation model is trained based on the first difference information between the predicted keywords and the labeled keywords, and the second difference information between the predicted title and the labeled title of the sample video segment.

[0011] In one possible implementation, the step of encoding the training text using the title generation model based on an attention mechanism to obtain the semantic features of the training text includes:

[0012] The title generation model configures the characters in the prefix text as global attention characters and the characters in the sample segmented text as local attention characters. The global attention characters are those that perform attention operations with all characters in the training text, and the local attention characters are those that perform attention operations with some characters in the training text.

[0013] The title generation model performs attention encoding on multiple characters in the training text to obtain attention features of multiple characters in the training text; the attention features of the multiple characters are then fused to obtain the semantic features of the training text.

[0014] In one possible implementation, the attention encoding of multiple characters in the training text to obtain attention features of the multiple characters in the training text includes:

[0015] For any one of the plurality of characters, if the character belongs to the prefix text, a first attention weight is determined between the character and other characters in the training text; based on the character and the first attention weight, the attention feature of the character is determined;

[0016] If the character belongs to the sample segmented text, a second attention weight is determined between the character and a reference character in the training text, wherein the reference character is a character surrounding the character in the training text; based on the character and the second attention weight, the attention feature of the character is determined.

[0017] In one possible implementation, determining the second attention weight between the character and a reference character in the training text when the character belongs to the sample segmented text includes:

[0018] When the character belongs to the sample text, the reference character in the sliding window is determined when the character is covered, and the sliding window slides on the training text;

[0019] Determine the query matrix of the character and the key matrix of the reference character;

[0020] Based on the query matrix of the character and the key matrix of the reference character, a second attention weight is determined between the character and the reference character in the training text.

[0021] In one possible implementation, the step of predicting based on the semantic features of the training text using the title generation model, and outputting the predicted keywords in the sample segmented text and the predicted title of the sample video clip, includes:

[0022] The title generation model performs fully connected and normalized semantic features on the training text to output predicted keywords in the sample segmented text.

[0023] The title generation model performs multiple rounds of iterative decoding based on the semantic features of the training text to output the predicted title of the sample video segment.

[0024] In one possible implementation, the step of using the title generation model to perform multiple rounds of iterative decoding based on the semantic features of the training text to output the predicted title of the sample video segment includes:

[0025] In the first round of iteration, the title generation model iterates based on the semantic features and start character of the training text, and outputs the first character of the predicted title of the sample video segment;

[0026] In other iterations, the title generation model outputs the corresponding character for the current iteration based on the semantic features of the training text and the characters output in the previous iteration.

[0027] In one possible implementation, training the title generation model based on the first difference information between the predicted keywords and the labeled keywords, and the second difference information between the predicted title and the labeled title of the sample video segment, includes:

[0028] A joint loss function is constructed based on the first difference information and the second difference information;

[0029] The title generation model is trained based on the joint loss function.

[0030] In one possible implementation, the method for obtaining the labeled keywords of the sample segmented text includes:

[0031] Keyword recognition is performed on the sample segment text of the sample video clip to obtain the labeled keywords of the sample segment text.

[0032] On the one hand, embodiments of this specification provide a title generation method, including:

[0033] Obtain the segmented text of the target video segment, wherein the target video segment is obtained by segmenting the video;

[0034] The split text is input into the title generation model, and the title generation model encodes the split text based on an attention mechanism to obtain the semantic features of the split text.

[0035] The title generation model predicts the title of the target video segment based on the semantic features of the segmented text.

[0036] The title generation model is trained based on the training text of the sample video clips, the labeled titles of the sample video clips, and the labeled keywords of the sample segmented text in the training text. The training text also includes the prefix text of the sample segmented text, which is the text of the labeled keywords in the sample segmented text that is masked.

[0037] In one possible implementation, after encoding the segmented text using the title generation model based on an attention mechanism to obtain the semantic features of the segmented text, the method further includes:

[0038] The title generation model predicts keywords from the split text based on its semantic features and outputs the keywords in the split text.

[0039] In one possible implementation, before inputting the segmented text into a title generation model and encoding the segmented text using an attention mechanism to obtain the semantic features of the segmented text, the method further includes:

[0040] Obtain reference keywords from the split text;

[0041] The step of inputting the segmented text into the title generation model, and encoding the segmented text using the title generation model based on an attention mechanism to obtain the semantic features of the segmented text includes:

[0042] The split text is input into the title generation model. Through the title generation model, characters in the reference keywords in the split text are configured as global attention characters, and other characters in the split text are configured as local attention characters. The global attention characters represent characters that perform attention operations with all characters in the split text, and the local attention characters represent characters that perform attention operations with some characters in the split text.

[0043] The title generation model is used to encode attention features of multiple characters in the split text to obtain attention features of multiple characters in the split text; the attention features of multiple characters in the split text are then fused to obtain semantic features of the split text.

[0044] In one possible implementation, the acquisition of the segmented text of the target video segment includes any of the following:

[0045] Speech recognition is performed on the audio of the target video segment to obtain the segmented text of the target video segment;

[0046] The portion of the video subtitles corresponding to the target video segment is determined as the segmented text of the target video segment.

[0047] On one hand, embodiments of this specification provide a training apparatus for a title generation model, comprising:

[0048] The training text acquisition module is used to acquire the training text of the sample video segment. The training text includes the sample segment text of the sample video segment and the prefix text of the sample segment text. The prefix text is the text marked with keywords in the sample segment text that is masked.

[0049] The training text encoding module is used to input the training text into the title generation model, and through the title generation model, encode the training text based on an attention mechanism to obtain the semantic features of the training text;

[0050] The training text prediction module is used to predict the semantic features of the training text based on the title generation model, and output the predicted keywords in the sample segmented text and the predicted title of the sample video segment.

[0051] The training module is used to train the title generation model based on the first difference information between the predicted keywords and the labeled keywords, and the second difference information between the predicted title and the labeled title of the sample video segment.

[0052] In one possible implementation, the training text encoding module is used to configure characters in the prefix text as global attention characters and characters in the sample segmented text as local attention characters through the title generation model. The global attention characters represent characters that perform attention operations with all characters in the training text, and the local attention characters represent characters that perform attention operations with some characters in the training text. The title generation model is used to encode attention features of multiple characters in the training text to obtain attention features of multiple characters in the training text. The attention features of the multiple characters are then fused to obtain the semantic features of the training text.

[0053] In one possible implementation, the training text encoding module is configured to, for any character among the plurality of characters, if the character belongs to the prefix text, determine a first attention weight between the character and other characters in the training text; determine the attention feature of the character based on the character and the first attention weight; if the character belongs to the sample segmented text, determine a second attention weight between the character and a reference character in the training text, wherein the reference character is a character surrounding the character in the training text; and determine the attention feature of the character based on the character and the second attention weight.

[0054] In one possible implementation, the training text encoding module is configured to: determine, when the character belongs to the sample segmented text, a reference character in a sliding window that covers the character, the sliding window sliding on the training text; determine the query matrix of the character and the key matrix of the reference character; and determine a second attention weight between the character and the reference character in the training text based on the query matrix of the character and the key matrix of the reference character.

[0055] In one possible implementation, the training text prediction module is used to perform fully connected and normalized semantic features of the training text through the title generation model, and output predicted keywords in the sample segmented text; and to perform multi-round iterative decoding based on the semantic features of the training text through the title generation model, and output the predicted title of the sample video segment.

[0056] In one possible implementation, the training text prediction module is used to output the first character of the predicted title of the sample video segment by iterating based on the semantic features of the training text and the start character through the title generation model during the first iteration; and to output the character corresponding to the current iteration by using the title generation model based on the semantic features of the training text and the character output in the previous iteration during other iterations.

[0057] In one possible implementation, the training module is configured to construct a joint loss function based on the first difference information and the second difference information; and to train the title generation model based on the joint loss function.

[0058] In one possible implementation, the method for obtaining the labeled keywords of the sample segmented text includes:

[0059] Keyword recognition is performed on the sample segment text of the sample video clip to obtain the labeled keywords of the sample segment text.

[0060] On one hand, embodiments of this specification provide a title generation apparatus, including:

[0061] The segmented text acquisition module is used to acquire the segmented text of a target video segment, wherein the target video segment is obtained by segmenting the video.

[0062] The text encoding module is used to input the text into the title generation model, and then encode the text based on the attention mechanism through the title generation model to obtain the semantic features of the text.

[0063] The title prediction module is used to predict the title of the target video segment based on the semantic features of the segmented text using the title generation model.

[0064] The title generation model is trained based on the training text of the sample video clips, the labeled titles of the sample video clips, and the labeled keywords of the sample segmented text in the training text. The training text also includes the prefix text of the sample segmented text, which is the text of the labeled keywords in the sample segmented text that is masked.

[0065] In one possible implementation, the device further includes:

[0066] The keyword prediction module is used to predict keywords in the split text based on the semantic features of the title generation model.

[0067] In one possible implementation, the device further includes:

[0068] The reference keyword acquisition module is used to acquire reference keywords in the split text;

[0069] The segmented text encoding module is further used to input the segmented text into the title generation model. Through the title generation model, characters in the reference keywords in the segmented text are configured as global attention characters, and other characters in the segmented text are configured as local attention characters. The global attention characters represent characters that perform attention operations with all characters in the segmented text, and the local attention characters represent characters that perform attention operations with some characters in the segmented text. Through the title generation model, attention encoding is performed on multiple characters in the segmented text to obtain the attention features of multiple characters in the segmented text. The attention features of multiple characters in the segmented text are fused to obtain the semantic features of the segmented text.

[0070] In one possible implementation, the text splitting acquisition module is configured to perform any of the following:

[0071] Speech recognition is performed on the audio of the target video segment to obtain the segmented text of the target video segment;

[0072] The portion of the video subtitles corresponding to the target video segment is determined as the segmented text of the target video segment.

[0073] On one hand, embodiments of this specification provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the methods described above.

[0074] On one hand, embodiments of this specification provide an electronic device, including: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the method described above.

[0075] On one hand, embodiments of this specification provide a computer program product containing instructions that, when run on a computer or processor, cause the computer or processor to perform the methods described above.

[0076] The technical solution provided in the embodiments of this specification trains a title generation model using training text from sample video clips. This training text includes sample segment text of the sample video clips and prefix text of the segment text. The prefix text is the text containing the labeled keywords in the sample segment text that is masked. This method of designing the training text allows the title generation model to simultaneously predict both keywords and titles. During training, the training text is encoded using an attention mechanism to obtain its semantic features. This attention mechanism enables the title generation model to focus on the parts of the training text that are helpful in predicting titles, improving the accuracy of the semantic features. Based on the semantic features of the training text, predicted keywords in the sample segment text and predicted titles of the sample video clips are obtained. The title generation model is trained based on a first difference between the predicted keywords and the labeled keywords, and a second difference between the predicted title and the labeled title. This allows the title generation model to generate titles efficiently and accurately. Attached Figure Description

[0077] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0078] Figure 1A schematic diagram illustrating the implementation environment of a training method for a title generation model provided in the embodiments of this specification;

[0079] Figure 2 A flowchart illustrating a training method for a title generation model provided in an embodiment of this specification;

[0080] Figure 3 A flowchart illustrating a title generation method provided in an embodiment of this specification;

[0081] Figure 4 A flowchart illustrating another method for training a title generation model provided in the embodiments of this specification;

[0082] Figure 5 This is a schematic diagram of the structure of the title generation model provided in the embodiments of this specification;

[0083] Figure 6 A flowchart illustrating another title generation method provided in the embodiments of this specification;

[0084] Figure 7 A schematic diagram of the structure of a training device for a title generation model provided in an embodiment of this specification;

[0085] Figure 8 This is a schematic diagram of the structure of a title generation device provided in an embodiment of this specification;

[0086] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification. Detailed Implementation

[0087] To make the features and advantages of this specification more apparent and understandable, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this specification without creative effort are within the scope of protection of this specification.

[0088] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0089] Artificial intelligence (AI) is the theory, methods, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0090] Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence. Its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by demonstration.

[0091] Video splitting: Video splitting is a secondary processing of traditional television media programs to meet the needs of internet video and new media short video content platforms. It involves breaking down a complete program into multiple videos according to a certain logical thinking or specific needs.

[0092] Transformer: Replaces the LSTM (Long Short-Term Memory) network with a full Attention structure to achieve a Seq2Seq deep learning model structure that can model global context and be trained in parallel.

[0093] Attention: In a general sense, it can be considered as a vector of importance weights used to estimate how strongly a target element is related to other elements, and the sum of their values ​​is used as an approximation of the target. It has a wide range of applications in various fields of deep learning.

[0094] BERT (Transformer-based bidirectional encoder representation): A pre-trained language model based on Transformers bidirectional encoders.

[0095] Longformer: An upgraded Transformer that uses a sparse self-attention mechanism to efficiently process long texts.

[0096] PALM (Pathways Language Model): A language model that combines autoencoders and autoregressors, which delegates autoencoder-based understanding to the encoder in a Transformer and autoregressive generation to the decoder in a Transformer.

[0097] Encoder: A network structure used to map an input sequence into an abstract, continuous representation that needs to ensure that the representation contains all the learned information about the input.

[0098] Decoder: A model structure that uses the continuous representations provided by the encoder to generate a single output step by step, while also taking into account the previous output.

[0099] Token: The basic word form in text, which generally refers to a character in Chinese.

[0100] Normalization: Mapping sequences of values ​​with different ranges to the interval (0, 1) to facilitate data processing. In some cases, normalized values ​​can be directly expressed as probabilities.

[0101] Dropout is a method for optimizing deep artificial neural networks. During the learning process, it reduces the interdependence between nodes by randomly setting some weights or outputs of the hidden layers to zero, thereby regularizing the neural network and reducing its structural risks. For example, in model training, given a vector (1, 2, 3, 4), after inputting this vector into a dropout layer, the dropout layer can randomly convert one of the numbers in the vector (1, 2, 3, 4) to 0. For example, converting 2 to 0 would change the vector to (1, 0, 3, 4).

[0102] Learning rate: Used to control the learning progress of the model. The learning rate guides the model in adjusting network weights using the gradient of the loss function during gradient descent. If the learning rate is too large, the loss function may directly skip the global optimum, resulting in excessive loss. If the learning rate is too small, the loss function changes very slowly, greatly increasing the convergence complexity of the network and making it easy to get trapped in local minima or saddle points.

[0103] Embedded coding, mathematically speaking, represents a correspondence, that is, mapping data in space X to space Y using a function F. This function F is injective, and the mapping result preserves the structure. An injective function means that the mapped data uniquely corresponds to the original data, and preserving the structure means that the order of the original data remains the same. For example, if there are data X1 and X2 before mapping, after mapping we get Y1 corresponding to X1 and Y2 corresponding to X2. If the original data X1 > X2, then correspondingly, the mapped data Y1 > Y2. For words, this means mapping words to another space to facilitate subsequent machine learning and processing.

[0104] Attention weights represent the importance of a piece of data during training or prediction. Importance indicates the magnitude of the influence of input data on output data. Data with high importance corresponds to higher attention weights, while data with low importance corresponds to lower attention weights. The importance of data varies in different scenarios, and training the model to assign attention weights is essentially the process of determining data importance.

[0105] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in the embodiments of this specification are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0106] The implementation environment of the technical solutions provided in the embodiments of this specification will be described below.

[0107] Figure 1 This is a schematic diagram illustrating the implementation environment of a training method for a title generation model provided in the embodiments of this specification. See also... Figure 1 The implementation environment includes terminal 110 and server 120.

[0108] Terminal 110 is connected to server 120 via a wireless or wired network. Optionally, terminal 110 may be a smartphone, tablet, laptop, desktop computer, smartwatch, etc., but is not limited to these. Terminal 110 has an application installed and running that supports title generation.

[0109] Server 120 is an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms. Server 120 provides background services for applications running on terminal 110. For example, server 120 provides corresponding services for applications running on terminal 110. In the embodiments of this specification, server 120 provides background services for applications running on the terminal that support face recognition, such as training a title generation model.

[0110] Those skilled in the art will understand that the number of terminals 110 and servers 120 can be more or less. For example, there may be only one terminal 110 and one server 120, or there may be dozens or hundreds of terminals 110 and servers 120, or even more. In this case, the above-described implementation environment may also include other terminals and servers. The embodiments in this specification do not limit the number of terminals or the type of devices.

[0111] After introducing the implementation environment of the embodiments of this specification, the application scenarios of the embodiments of this specification will be described below in conjunction with the above implementation environment. In the following description, the terminal is also the terminal 110 in the above implementation environment, and the server is also the server 120 in the above implementation environment. The technical solutions provided by the embodiments of this specification can be applied to scenarios of generating titles for various video clips. For example, the technical solutions provided by the embodiments of this specification can be applied to scenarios of generating titles for financial video clips, or to scenarios of generating titles for sports video clips, or to scenarios of generating titles for food video clips.

[0112] Taking the technical solution provided in the embodiments of this specification as an example in the scenario of generating titles for financial video clips, the terminal uploads the segmented text of the financial video clip to the server, and the server obtains the segmented text. The server inputs the segmented text of the financial video clip into a trained title generation model. Through the title generation model, the segmented text is encoded based on an attention mechanism to obtain the semantic features of the segmented text. The server uses the title generation model to predict based on the semantic features of the segmented text and outputs the title of the financial video clip. The title generation model is trained based on the training text of the sample video clip, the labeled title of the sample video clip, and the labeled keywords of the sample segmented text in the training text. The training text also includes the prefix text of the sample segmented text, which is the text of the labeled keywords masked in the sample segmented text.

[0113] It should be noted that the above description uses the technical solution provided in the embodiments of this specification as an example in the scenario of generating titles for financial video clips. For other scenarios, the implementation process belongs to the same inventive concept as described above, and the implementation process will not be repeated. In addition, the technical solution provided in the embodiments of this specification can also be applied to the scenario of generating article titles, and the embodiments of this specification do not limit this application.

[0114] After introducing the implementation environment and application scenarios of the embodiments in this specification, the training method of the title generation model provided in the embodiments of this specification is described below. (See also...) Figure 2 The method, with the server as the executing entity, includes the following steps.

[0115] 202. The server obtains the training text of the sample video segment. The training text includes the sample segment text of the sample video segment and the prefix text of the sample segment text. The prefix text is the text in the sample segment text that is masked and marked with keywords.

[0116] The sample video clips are video segments used when training the title generation model. These clips can be extracted from sample videos or are independent video segments; this specification does not limit this. Correspondingly, the training text is the video clip used to train the title generation model. The training text includes two parts: sample segment text and a prefix text. The sample segment text corresponds to the sample video clip and describes its content. In some embodiments, the sample segment text is obtained by performing text recognition on the audio of the sample video clip. The prefix text is specially set during the training of the title generation model. This prefix text includes labeled keywords from the sample segment text, but these keywords are masked. Masking means that the title generation model cannot directly obtain these labeled keywords and needs to predict them. Labeled keywords refer to keywords marked by technicians in the sample segment text, serving as the standard for the title generation model to generate keywords. In some embodiments, this training text is also referred to as long text. It should be noted that the training text used when training this title generation model is equivalent to the segmented text input when using this title generation model.

[0117] 204. The server inputs the training text into the title generation model. The title generation model encodes the training text based on the attention mechanism to obtain the semantic features of the training text.

[0118] This title generation model generates titles based on input text. An attention mechanism allows the model to focus on information within the text that is helpful for title generation, improving the relevance of the generated title to the text and thus increasing its accuracy. Encoding the training text involves converting it into abstract semantic features, which the title generation model can then process. These semantic features represent the semantics of the training text.

[0119] 206. The server uses the title generation model to make predictions based on the semantic features of the training text, and outputs the predicted keywords in the sample segment text and the predicted title of the sample video segment.

[0120] The predicted keyword is the keyword predicted by the title generation model. Since the keyword is related to the title, the purpose of predicting the keyword in the sample text is to help the subsequent title prediction process.

[0121] 208. The server trains the title generation model based on the first difference information between the predicted keyword and the labeled keyword, and the second difference information between the predicted title and the labeled title of the sample video segment.

[0122] The purpose of training the title generation model based on the first and second difference information is to minimize the first and second difference information, that is, to make the predicted keywords output by the title generation model as close as possible to the labeled keywords, and the predicted title output by the title generation model as close as possible to the labeled title, so that the title generation model can generate a title based on the input text.

[0123] The technical solution provided in the embodiments of this specification trains a title generation model using training text from sample video clips. This training text includes sample segment text of the sample video clips and prefix text of the segment text. The prefix text is the text containing the labeled keywords in the sample segment text that is masked. This method of designing the training text allows the title generation model to simultaneously predict both keywords and titles. During training, the training text is encoded using an attention mechanism to obtain its semantic features. This attention mechanism enables the title generation model to focus on the parts of the training text that are helpful in predicting titles, improving the accuracy of the semantic features. Based on the semantic features of the training text, predicted keywords in the sample segment text and predicted titles of the sample video clips are obtained. The title generation model is trained based on a first difference between the predicted keywords and the labeled keywords, and a second difference between the predicted title and the labeled title. This allows the title generation model to generate titles efficiently and accurately.

[0124] Steps 202-208 above introduce the training method of this title generation model. The following explains how to generate titles using this model; see [link to documentation]. Figure 3 Taking the server as the executing entity as an example, the method includes the following steps.

[0125] 302. The server obtains the segmented text of the target video segment, which is obtained by splitting the video into segments.

[0126] Video segmentation refers to breaking a complete video into multiple video segments. This segmentation facilitates further processing of the video. The target video segment is the segment for which a title is to be generated, and the segmented text represents the content of that target video segment.

[0127] 304. The server inputs the split text into the title generation model. The title generation model encodes the split text based on the attention mechanism to obtain the semantic features of the split text.

[0128] The title generation model is trained based on the training text of the sample video clip, the labeled title of the sample video clip, and the labeled keywords of the sample segmented text in the training text. The training text also includes the prefix text of the sample segmented text, which is the text of the labeled keyword in the sample segmented text that is masked. This is related to step 204 above and belongs to the same inventive concept. By encoding the segmented text using this title generation model, an abstract expression of the segmented text can be achieved, and the resulting semantic features can reflect the semantics of the segmented text.

[0129] 306. The server uses the title generation model to predict the title of the target video segment based on the semantic features of the split text.

[0130] The title generation model is a trained model. Using this model to predict the title based on the semantic features of the segmented text can produce a relatively accurate title, which can represent the content of the target video segment.

[0131] The technical solution provided in the embodiments of this specification inputs the segmented text of a target video clip into a title generation model. This model encodes the segmented text using an attention mechanism to obtain its semantic features. These semantic features accurately reflect the semantics of the segmented text, and thus accurately reflect the semantics of the target video clip. By using this title generation model to predict the title of the target video clip based on the semantic features of the segmented text, the title can be obtained. The title generation process eliminates the need for manual operation by technicians, improving the efficiency of title generation.

[0132] Steps 202-208 above are a brief introduction to the training method of the title generation model provided in the embodiments of this specification. To more clearly illustrate the training method of the title generation model provided in the embodiments of this specification, the technical solutions provided in the embodiments of this specification will be described below with reference to some examples. See [link to documentation]. Figure 4 The method includes the following steps.

[0133] 402. The server obtains the training text of the sample video segment. The training text includes the sample segment text of the sample video segment and the prefix text of the sample segment text. The prefix text is the text in the sample segment text that is masked and marked with keywords.

[0134] The sample video clips are video segments used when training the title generation model. These clips can be extracted from sample videos or are independent video segments; this specification does not limit this. Correspondingly, the training text is the video clip used to train the title generation model. The training text includes two parts: sample segment text and a prefix text. The sample segment text corresponds to the sample video clip and describes its content. In some embodiments, the sample segment text is obtained by performing text recognition on the audio of the sample video clip. The prefix text is specially set during the training of the title generation model. This prefix text includes labeled keywords from the sample segment text, but these keywords are masked. Masking means that the title generation model cannot directly obtain these labeled keywords and needs to predict them. Labeled keywords refer to keywords marked by technicians in the sample segment text, serving as the standard for the title generation model to generate keywords. In some embodiments, this training text is also referred to as long text. In some embodiments, the prefix text of the sample segment text is filled in by a technician when training the title generation model, or generated by the server based on keywords in the sample segment text; this specification does not limit this. For example, the segment text of a sample video clip is "Let's focus on the latest developments of AAA today," where "AAA" is a labeled keyword. The prefix text of this sample segment text could be "[Mask][Mask][Mask] is a YYY," where "[Mask][Mask][Mask]" is the masked labeled keyword "AAA." Masking the labeled keyword in this prefix text is to train the keyword generation ability of the title generation model. The keyword generation task is introduced because keyword generation forces the title generation model to extract the core information of the text, thereby improving the accuracy of title generation. This is because generating keywords helps generate titles, and this masking method can improve the accuracy of title generation.

[0135] In one possible implementation, the server retrieves the training text of the sample video segment from a training text library that stores training texts of multiple candidate video segments, to which the sample video segment belongs.

[0136] In this implementation, the server can obtain training text from the corresponding maintained training text library, and then train the title generation model based on the training text. The efficiency of obtaining training text is relatively high, thereby improving the efficiency of training the title generation model.

[0137] For example, the server queries the corresponding maintained training text library to obtain the training text of multiple candidate video segments. The server then extracts the training text of the sample video segment from the training text of these multiple candidate video segments.

[0138] In one possible implementation, the server obtains the training text of the sample video clips uploaded by the terminal.

[0139] In this implementation, the server can obtain training text from the terminal, and the training text configured by the technician through the terminal can be directly uploaded to the server for model training, which is highly efficient.

[0140] For example, the server retrieves the training text of multiple candidate video clips uploaded by the terminal. The server then extracts the training text of the sample video clip from the training text of these candidate video clips. For instance, the terminal displays a training text upload interface for selecting and uploading training text. When the terminal selects the training text of the multiple candidate video clips through this interface, it uploads the training text of those candidate video clips to the server in response to the operation on that interface. The server retrieves the training text of the multiple candidate video clips and extracts the training text of the sample video clip from the training text of these candidate video clips.

[0141] It should be noted that the server can obtain the training text of the sample video segment through any of the above methods, and the embodiments in this specification do not limit this.

[0142] To provide a clearer explanation of the above implementation methods, the method for obtaining the labeled keywords in the sample text is described below.

[0143] In one possible implementation, the server performs keyword recognition on the sample segment text of the sample video clip to obtain the labeled keywords of the sample segment text.

[0144] In this implementation, the server can quickly obtain the keywords in the sample segmented text by performing keyword recognition on the sample segmented text. These keywords are also called labeled keywords, which can be used to generate the prefix text of the sample segmented text and as annotations (Ground True) for model training.

[0145] For example, the server inputs the sample segment text of the sample video clip into a keyword recognition model, which then performs keyword recognition on the sample segment text and outputs the labeled keywords in the sample segment text. This keyword recognition model has the capability to recognize keywords in the input text; in this embodiment, it specifically identifies the labeled keywords in the sample segment text.

[0146] For example, the server inputs the sample text into a keyword recognition model, which then segments the text into words, obtaining multiple candidate words. The server uses the keyword recognition model to determine the word frequency of these candidate words. The word frequency represents the frequency of a candidate word's occurrence in the corpus; a higher word frequency indicates greater importance within the corpus. Based on the word frequency of these candidate words, the server uses the keyword recognition model to determine the keywords in the sample text.

[0147] In some embodiments, the sample segmented text includes multiple keywords, and the server can determine the multiple keywords in the sample segmented text in the manner described above. These multiple keywords are all tagging keywords of the sample segmented text.

[0148] It should be noted that the above description is based on the example of the server using a keyword recognition model to determine the keywords in the sample segmented text by word frequency. In other possible implementations, the server may also use other methods to determine the keywords in the sample segmented text, such as using keyword recognition models with other structures and principles. This specification does not limit this aspect.

[0149] 404. The server inputs the training text into the title generation model.

[0150] This title generation model is used to generate titles corresponding to the input text.

[0151] In one possible implementation, the server performs embedding encoding on the training text to obtain its embedding features. The server then inputs these embedding features into the title generation model.

[0152] In this implementation, before inputting the training text into the title generation model, the server first performs embedding encoding on the training text to achieve a discretized representation of the training text, which helps the subsequent processing of the title generation model.

[0153] For example, the server performs embedding encoding on multiple characters in the training text to obtain the character embedding features of each character in the training text. The server performs embedding encoding on the positions of multiple characters in the training text to obtain the positional embedding features of each character in the training text. The server concatenates the character embedding features and the positional embedding features of each character in the training text to obtain the embedding features of each character in the training text. These multiple character embedding features constitute the embedding features of the training text.

[0154] In some embodiments, the title generation model includes an encoder and a decoder. The encoder encodes the input text to obtain semantic features of the input text. The decoder decodes the semantic features of the input text to obtain the title corresponding to the input text. Accordingly, inputting the training text into the title generation model in step 404 above means inputting the training text into the encoder of the title generation model. Furthermore, the encoder of the title generation model is also used to predict keywords in the input text based on the semantic features of the input text. Figure 5 A structural diagram of the title generation model is provided; see [link / reference]. Figure 5 The title generation model 500 includes an encoder 501 and a decoder 502. The server inputs text 503 into the encoder 501, which encodes the text 503 to obtain its semantic features. Based on these semantic features, the encoder 501 predicts the keywords 504 within the text 503. The encoder 501 then sends these semantic features to the decoder 502, which decodes them to obtain the title 505 of the text 503. In some embodiments, the encoder 501 is an encoder of the Longformer model, and the decoder 502 is a decoder of the BERT model. Of course, with the development of science and technology, the encoder 501 and decoder 502 can also have other structures, and this specification does not limit this embodiment.

[0155] 406. The server uses the title generation model to configure the characters in the prefix text as global attention characters and the characters in the sample segmented text as local attention characters. The global attention character represents the character that performs attention operation with all characters in the training text, and the local attention character represents the character that performs attention operation with some characters in the training text.

[0156] Global attention characters are also known as global tokens, while layout attention characters are known as local tokens. Global attention characters can perform attention operations with all characters in the training text during the attention-based encoding process, while local attention characters can only perform attention operations with a portion of the training text. This allows the server to significantly reduce encoding time when encoding the training text using the title generation model without losing contextual information. Configuring characters in the prefix text as global attention characters ensures that all information in the training text serves the keyword generation task, which in turn assists the title generation task.

[0157] In some embodiments, the partial characters in the training text that undergo attention operations with the local attention characters are determined using a sliding window. The size and sliding step of the sliding window are set by a technician according to the actual situation, and this specification does not limit this. For example, during the attention operation, when the center of the sliding window moves to the position of any local attention character, the other characters in the sliding window are determined as the partial characters corresponding to that local attention character. As the sliding window moves, partial characters corresponding to different local attention characters can be determined. In some embodiments, the partial characters corresponding to the local attention characters are also referred to as the reference characters of the local attention characters.

[0158] 408. The server uses the title generation model to perform attention encoding on multiple characters in the training text, thereby obtaining the attention features of multiple characters in the training text.

[0159] The attention mechanism enables the title generation model to focus on information in the text that is helpful for title generation, improving the relevance of the generated title to the text, and thus improving the accuracy of the generated title. Encoding the training text means encoding it into abstract semantic features so that the title generation model can process it; these semantic features can represent the semantics of the training text.

[0160] In one possible implementation, for any one of the plurality of characters, if the character belongs to the prefix text, the server uses the title generation model to determine a first attention weight between the character and other characters in the training text. The server then uses the title generation model to determine the attention features of the character based on the character and the first attention weight. If the character belongs to the sample segmented text, the server uses the title generation model to determine a second attention weight between the character and a reference character in the training text, where the reference character is a character surrounding the character in the training text. The server then uses the title generation model to determine the attention features of the character based on the character and the second attention weight.

[0161] In this context, if the character belongs to the prefix text, it indicates that the character is a global attention character; if the character belongs to the sample segment text, it indicates that the character is a local attention character.

[0162] In this implementation, the server can use the title generation model to encode attention for different types of characters in the training text, thereby obtaining attention features for different types of characters. This greatly reduces the time required to determine attention features without losing contextual information.

[0163] To provide a clearer explanation of the above embodiments, the following description will be divided into several parts.

[0164] In the first part, when the character belongs to the prefix text, the server uses the title generation model to determine the first attention weight between the character and other characters in the training text.

[0165] In one possible implementation, the server uses the title generation model to perform a linear transformation on the embedding features of multiple characters in the training text, obtaining the query matrix, key matrix, and value matrix of each character in the training text. If the character belongs to the prefix text, the server uses the title generation model to multiply the query matrix of that character with the key matrices of other characters in the training text, respectively, to obtain the first attention weight between that character and the other characters in the training text.

[0166] In this implementation, the server can use the title generation model to perform attention operations on the global attention character, obtain the first attention weight between the global attention character and other characters in the training text, so that the global attention character can pay attention to all characters in the training text.

[0167] For example, the server uses the encoder of the title generation model to multiply the embedding features of each character in the training text with the first, second, and third linear transformation matrices, respectively, to obtain the query matrix, key matrix, and value matrix of each character. These multiplications of the first, second, and third linear transformation matrices are obtained during the training of the title generation model. The server then uses the encoder of the title generation model to multiply the query matrix of each character with the key matrix of each character in the training text, obtaining the first attention value between that character and each character in the training text. Finally, the server uses the encoder of the title generation model to normalize the first attention value between that character and each character in the training text, obtaining the first attention weight between that character and each character in the training text.

[0168] The second part involves the server using the title generation model to determine the attention features of the character based on the character and the first attention weight.

[0169] In one possible implementation, the server uses the title generation model to multiply the value matrix of the character by the first attention weights between the character and each character in the training text, obtaining multiple attention vectors for the character, where each attention vector corresponds to a character in the training text. The server then uses the title generation model to fuse these multiple attention vectors to obtain the character's attention features.

[0170] In this implementation, the server can determine multiple attention vectors of the character through the title generation model. After fusing the multiple attention vectors of the character, the attention feature of the character is obtained. The attention feature of the character combines the information of all characters in the training text and can represent the semantics of the character more accurately.

[0171] For example, the server uses the encoder of the title generation model to multiply the value matrix of the character by the first attention weights between the character and each character in the training text, obtaining multiple attention vectors for the character. Each attention vector corresponds to a character in the training text. The server then uses the encoder of the title generation model to sum the multiple attention vectors of the character to obtain the attention features of the character.

[0172] In the third part, when the character belongs to the sample text, the server uses the title generation model to determine the second attention weight between the character and the reference character in the training text, which is the character surrounding the character in the training text.

[0173] In one possible implementation, when the character belongs to the sample segmented text, the server uses the title generation model to determine the reference character in a sliding window that slides over the training text when covering the character. The server uses the title generation model to determine the query matrix of the character and the key matrix of the reference character. Based on the query matrix and the key matrix of the reference character, the server uses the title generation model to determine a second attention weight between the character and the reference character in the training text.

[0174] In this implementation, when the character is a local attention character, the server can use the title generation model to perform local attention encoding on the character based on a sliding window, thereby improving the efficiency of encoding the character.

[0175] For example, when the character belongs to the sample text segment, the server uses the title generation model to determine the reference character in the sliding window centered on that character. This reference character is one of the other characters covered by the sliding window. The server uses the title generation model to determine the query matrix of the character and the key matrix of the reference character. The method by which the server obtains the query matrix and key matrix of the character using the title generation model belongs to the same inventive concept as described in the first part, and the implementation process will not be repeated here. The server uses the title generation model to multiply the query matrix of the character and the key matrix of the reference character to obtain the second attention weight between the character and the reference character in the training text.

[0176] For example, if the character belongs to the sample text segment, the server uses the encoder of the title generation model to determine the reference character in the sliding window centered on that character. The server uses the encoder of the title generation model to determine the query matrix of the character and the key matrix of the reference character. The server uses the encoder of the title generation model to multiply the query matrix of the character and the key matrix of the reference character to obtain the second attention value between the character and the reference character in the training text. The server uses the encoder of the title generation model to normalize the second attention value between the character and the reference character in the training text to obtain the second attention weight between the character and the reference character in the training text.

[0177] Part Four: The server uses the title generation model to determine the attention features of the character based on the character and the second attention weight.

[0178] In one possible implementation, the server uses the title generation model to multiply the value matrix of the character by a second attention weight between the character and a reference character in the training text, obtaining multiple attention vectors for the character, where each attention vector corresponds to a reference character in the training text. The server then fuses these multiple attention vectors to obtain the character's attention features.

[0179] For example, the server uses the encoder of the title generation model to multiply the value matrix of the character by the second attention weight between the character and a reference character in the training text, obtaining multiple attention vectors for the character. Each attention vector corresponds to a reference character in the training text. The server then uses the encoder of the title generation model to sum these multiple attention vectors to obtain the attention features of the character.

[0180] 410. The server uses the title generation model to fuse the attention features of the multiple characters to obtain the semantic features of the training text.

[0181] In one possible implementation, the server uses the title generation model to concatenate the attention features of the multiple characters to obtain the semantic features of the training text.

[0182] In this implementation, the attention features of multiple characters in the training text can represent the semantics of those characters. By concatenating the attention features of those characters, the semantic features of the training text can be obtained.

[0183] 412. The server uses the title generation model to make predictions based on the semantic features of the training text, and outputs the predicted keywords in the sample segment text and the predicted title of the sample video segment.

[0184] The predicted keyword is the keyword predicted by the title generation model. Since the keyword is related to the title, the purpose of predicting the keyword in the sample text is to help the subsequent title prediction process.

[0185] In one possible implementation, the server uses the title generation model to perform fully connected and normalized semantic features on the training text, outputting predicted keywords from the sample segmented text. The server then uses the title generation model to perform multiple rounds of iterative decoding based on the semantic features of the training text, outputting the predicted title of the sample video segment.

[0186] In this implementation, the server can generate both the predicted keywords in the sample text and the predicted title of the sample video segment through the title generation model. Since the keyword generation task can assist the title generation task, the title generation model can achieve better results by using the predicted keywords and predicted titles.

[0187] To illustrate the above embodiments, the following description will be divided into two parts.

[0188] The first part involves the server using the title generation model to perform fully connected and normalized semantic features on the training text, and outputting the predicted keywords in the sample text.

[0189] In one possible implementation, the server uses the encoder of the title generation model to perform fully connected and normalized semantic features on the training text, outputting a probability set of different keywords corresponding to the training text. This probability set includes multiple probabilities, each corresponding to a candidate keyword. Based on this probability set, the server determines the predicted keyword in the sample segmented text from among the multiple candidate keywords. For example, the server determines the candidate keyword corresponding to the highest probability in the probability set as the predicted keyword in the sample segmented text; or, the server determines the candidate keywords corresponding to the N highest probabilities in the probability set as the predicted keywords in the sample segmented text, where N is a positive integer.

[0190] The second part involves the server using the title generation model to perform multiple rounds of iterative decoding based on the semantic features of the training text, and outputting the predicted title of the sample video segment.

[0191] In one possible implementation, during the first iteration, the server uses the title generation model to iterate based on the semantic features of the training text and the start character, outputting the first character of the predicted title for the sample video clip. In subsequent iterations, the server uses the title generation model to output the character corresponding to the current iteration, based on the semantic features of the training text and the character output from the previous iteration.

[0192] The start character is a character of a preset type, used to indicate that the title generation model should begin decoding. For example, the start character is... <start>or <bos>'bos' indicates (Beginning of Sequence), but this specification does not limit the examples therein.

[0193] For example, in the first iteration, the server uses the decoder of the title prediction generation model to fuse the semantic features of the training text and the embedding features of the start character, obtaining a first fused feature. The server then uses the decoder of the title prediction generation model to perform a fully connected and normalized operation on this first fused feature to obtain the first character of the title. That is, the server uses the decoder of the title prediction generation model to perform a fully connected and normalized operation on the first fused feature to obtain the probability that the position of the first character of the title corresponds to multiple candidate characters. Based on the probabilities of these multiple candidate characters, the server determines the first character of the predicted title from among them; for example, the server determines the candidate character with the highest probability among the multiple candidate characters as the first character of the predicted title.

[0194] In subsequent iterations, the server fuses the semantic features of the training text with the embedding features of the characters output from the previous iteration using the decoder of the title generation model, obtaining a second fused feature. The server then uses the decoder of the title generation model to perform a fully connected and normalized operation on this second fused feature to obtain the other character in the title. In other words, the server uses the decoder of the title generation model to perform a fully connected and normalized operation on the second fused feature to obtain the probability of the position of the other character in the title corresponding to multiple candidate characters. Based on the probabilities of these multiple candidate characters, the server determines the other character of the predicted title from among them; for example, the server determines the candidate character with the highest probability among the multiple candidate characters as the other character of the predicted title.

[0195] When the server generates a terminating character using the decoder of the title generation model, the server stops the iteration process. The characters preceding the terminating character constitute the predicted title of the sample video segment. The terminating character is a character of a preset type, used to instruct the title generation model to stop decoding. For example, the terminating character is... <stop>or <eos>eos indicates (End of Sequence), but this specification does not limit the examples thereto.

[0196] The above implementation method is illustrated below with a specific example, using the training text "Semiconductors have undergone significant adjustments, the reason being XXXXXXXXX" as an example.

[0197] During the first iteration, the server checked the start character. <bos>Perform embedding encoding to obtain the start character. <bos>The embedding feature. The server will start the character <bos>The embedded features are input into the decoder of the title generation model, and the decoder of the title generation model will then input the start character. <bos>The embedded features are fused with the semantic features of the training text to obtain the first fused feature. The server uses the decoder of the title generation model to perform a fully connected operation and normalization on the first fused feature, obtaining the first character "半" of the predicted title. In the second iteration process, the server performs embedded encoding on the first character "半", obtaining the embedded features of the first character "半". The server inputs the embedded features of the first character "半" into the decoder of the title generation model, and the decoder of the title generation model fuses the embedded features of the first character "半" with the semantic features of the training text to obtain the second fused feature. The server uses the decoder of the title generation model to perform a fully connected operation and normalization on the second fused feature, obtaining the second character "导" of the predicted title. In the third iteration process, the server performs embedded encoding on the second character "导", obtaining the embedded features of the second character "导". The server inputs the embedded features of the second character "导" into the decoder of the title generation model, and the decoder of the title generation model fuses the embedded features of the second character "导" with the semantic features of the training text to obtain the second fused feature. The server uses the decoder of the title generation model to perform a fully connected operation and normalization on the second fused feature, obtaining the third character "体" of the predicted title. In the fourth iteration process, the server performs embedded encoding on the third character "体", obtaining the embedded features of the third character "体". The server inputs the embedded features of the third character "体" into the decoder of the title generation model, and the decoder of the title generation model fuses the embedded features of the third character "体" with the semantic features of the training text to obtain the second fused feature. The server uses the decoder of the title generation model to perform a fully connected operation and normalization on the second fused feature, obtaining the fourth character, the termination character, of the predicted title <eos>The server stopped iterating and obtained the predicted title "semiconductor".

[0198] 414. The server trains the title generation model based on the first difference information between the predicted keyword and the labeled keyword, and the second difference information between the predicted title and the labeled title of the sample video segment.

[0199] The purpose of training the title generation model based on the first and second difference information is to minimize these two differences, meaning that the predicted keywords and predicted titles output by the title generation model should be as close as possible to the labeled keywords, and the predicted titles should be as close as possible to the labeled titles, enabling the title generation model to generate titles based on the input text. In some embodiments, this title generation model is also referred to as a Phrase-Aware Lonformer-Bert Network (PALBN).

[0200] In one possible implementation, the server constructs a joint loss function based on the first difference information and the second difference information. The server then trains the title generation model based on this joint loss function.

[0201] For example, the server constructs a joint loss function based on the first and second difference information. The server then trains the title generation model using gradient descent based on this joint loss function.

[0202] It should be noted that the above description is based on one round of iterative training of the title generation model by the server. In other rounds of iterative training, the training method is based on the same inventive concept as described above, and will not be repeated here.

[0203] Furthermore, in the embodiments of this specification, the title generation model can be trained using a pre-training and fine-tuning approach. Steps 402-414 above can be considered as a fine-tuning process. For the pre-training process, the server can use a publicly available <title, abstract (text)> dataset. Since there are no keywords in the <title, abstract (text)> dataset, a keyword recognition model can be used to identify keywords in the abstract of the <title, abstract (text)> dataset to obtain the <title, keyword, abstract (text)> dataset for pre-training the title generation model. The pre-training process and steps 402-414 above belong to the same inventive concept and will not be described again here.

[0204] Furthermore, the above description uses the server as the execution subject as an example. In other possible implementations, the terminal can also be the execution subject to perform the above steps 402-414. This specification does not limit this embodiment.

[0205] All the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this specification, and will not be described in detail here.

[0206] The technical solution provided in the embodiments of this specification trains a title generation model using training text from sample video clips. This training text includes sample segment text of the sample video clips and prefix text of the segment text. The prefix text is the text containing the labeled keywords in the sample segment text that is masked. This method of designing the training text allows the title generation model to simultaneously predict both keywords and titles. During training, the training text is encoded using an attention mechanism to obtain its semantic features. This attention mechanism enables the title generation model to focus on the parts of the training text that are helpful in predicting titles, improving the accuracy of the semantic features. Based on the semantic features of the training text, predicted keywords in the sample segment text and predicted titles of the sample video clips are obtained. The title generation model is trained based on a first difference between the predicted keywords and the labeled keywords, and a second difference between the predicted title and the labeled title. This allows the title generation model to generate titles efficiently and accurately.

[0207] Steps 402-414 above describe the training method of the title generation model provided in the embodiments of this specification. The following describes the method for generating titles using this title generation model; see [link to documentation]. Figure 6 Taking the server as the executing entity as an example, the method includes the following steps.

[0208] 602. The server obtains the segmented text of the target video segment, which is obtained by splitting the video into segments.

[0209] Video segmentation refers to breaking a complete video into multiple video segments. This segmentation facilitates further processing of the video. The target video segment is the segment for which a title is to be generated, and the segmented text represents the content of that target video segment.

[0210] In one possible implementation, the server performs speech recognition on the audio of the target video segment to obtain the segmented text of the target video segment.

[0211] In this implementation, the server can obtain the segmented text of the target video segment by performing speech recognition on the audio of the target video segment. The segmented text can accurately reflect the content of the target video segment.

[0212] For example, the server splits a video into multiple video segments. The server then identifies a target video segment from these segments; for instance, the target video segment might be the one selected by the user. The server inputs the audio of the target video segment into a speech recognition model, performs speech recognition on the audio of the target video segment using the speech recognition model, and outputs the segmented text of the target video segment. The speech recognition model can be of any structure, and this specification does not limit its implementation in the embodiments.

[0213] In one possible implementation, the server determines the portion of the video subtitles corresponding to the target video segment as the segmented text of the target video segment.

[0214] In this implementation, the server can directly determine the subtitles corresponding to the target video segment as the segmented text of the target video segment, which is highly efficient.

[0215] For example, the server splits a video into multiple video segments. The server then identifies the target video segment from these segments; for instance, the target video segment might be the one selected by the user. The server then identifies the corresponding subtitle for the target video segment from the video's subtitles; this subtitle is the segmented text of the target video segment.

[0216] In one possible implementation, the server obtains the segmented text of the target video clip uploaded by the terminal.

[0217] In this implementation, the server can obtain the segmented text from the terminal, and the segmented text selected by the technician through the terminal can be directly uploaded to the server for model training, which is highly efficient.

[0218] It should be noted that the server can obtain the segmented text of the target video segment through any of the above methods, and the embodiments in this specification do not limit this.

[0219] 604. The server inputs the split text into the title generation model. The title generation model encodes the split text based on the attention mechanism to obtain the semantic features of the split text.

[0220] The title generation model is trained based on the training text of the sample video clip, the labeled title of the sample video clip, and the labeled keywords of the sample segmented text in the training text. The training text also includes the prefix text of the sample segmented text, which is the text of the labeled keyword in the sample segmented text that is masked. This is related to step 204 above and belongs to the same inventive concept. By encoding the segmented text using this title generation model, an abstract expression of the segmented text can be achieved, and the resulting semantic features can reflect the semantics of the segmented text.

[0221] In one possible implementation, the server performs embedding encoding on multiple characters of the split text to obtain embedding features of those characters. The server then inputs these embedding features into a title generation model, which performs a linear transformation on them to obtain the query matrix, key matrix, and value matrix for each character in the split text. Finally, the server uses the title generation model to perform calculations based on these matrixes to obtain the semantic features of the split text.

[0222] For example, the server performs embedding encoding on multiple characters of the split text to obtain the embedding features of those characters. The server inputs these embedding features into the encoder of the title generation model. The encoder multiplies the embedding features of each character in the split text with a first linear transformation matrix, a second linear transformation matrix, and a third linear transformation matrix, respectively, to obtain the query matrix, key matrix, and value matrix for each character. These multiplications are obtained during the training of the title generation model. For any given character, the server uses the encoder of the title generation model to multiply the query matrix of that character with the key matrix of each character in the split text, respectively, to obtain the attention value between that character and each character in the split text. The server then uses the encoder of the title generation model to normalize the attention value between that character and each character in the split text, obtaining the attention weight between that character and each character in the split text. The server uses the encoder of the title generation model to multiply the value matrix of the character by the attention weights between the character and each character in the segmented text, obtaining multiple attention vectors for the character. Each attention vector corresponds to a single character in the segmented text. The server then uses the encoder of the title generation model to sum these multiple attention vectors to obtain the character's attention features. Finally, the server fuses the attention features of multiple characters in the segmented text to obtain the semantic features of the segmented text.

[0223] In one possible implementation, the server obtains reference keywords from the segmented text, which are keywords selected by the user. The server inputs the segmented text into the title generation model. Through the title generation model, characters from the reference keywords in the segmented text are configured as global attention characters, and other characters in the segmented text are configured as local attention characters. Global attention characters represent characters that perform attention operations with all characters in the segmented text, and local attention characters represent characters that perform attention operations with a subset of characters in the segmented text. The server uses the title generation model to perform attention encoding on multiple characters in the segmented text, obtaining attention features for multiple characters in the segmented text. The attention features of multiple characters in the segmented text are then fused to obtain the semantic features of the segmented text.

[0224] The other characters in the split text refer to the characters in the split text other than the characters in the keywords.

[0225] In this implementation, the user can select reference keywords when generating the title. The server configures the characters in the reference keywords in the split text as global attention characters, so that the reference keywords can pay attention to all characters in the split text. The other characters in the split text are configured as local attention characters, so that the title generation model can focus its attention on the reference keywords when generating the title, thereby generating a title related to the reference keywords.

[0226] For example, when a user uses the title generation model to generate a title, if they are not satisfied with the default title generated by the model, they can specify reference keywords, and the title generation model can generate a title related to those reference keywords.

[0227] 606. The server uses the title generation model to predict the title of the target video segment based on the semantic features of the split text.

[0228] The title generation model is a trained model. Using this model to predict the title based on the semantic features of the segmented text can produce a relatively accurate title, which can represent the content of the target video segment.

[0229] In one possible implementation, the server uses the title generation model to perform multiple rounds of iterative decoding based on the semantic features of the split text, and outputs the title of the target video segment.

[0230] In one possible implementation, during the first iteration, the server uses the title generation model to iterate based on the semantic features of the segmented text and the starting character, outputting the first character of the title of the target video segment. In subsequent iterations, the server uses the title generation model to output the corresponding character for the current iteration, based on the semantic features of the segmented text and the character output from the previous iteration.

[0231] The start character is a character of a preset type, used to indicate that the title generation model should begin decoding. For example, the start character is... <start>or <bos>'bos' indicates (Beginning of Sequence), but this specification does not limit the examples therein.

[0232] For example, in the first iteration, the server uses the decoder of the title generation model to fuse the semantic features of the split text and the embedding features of the start character to obtain the first fused feature. The server then uses the decoder of the title generation model to perform a fully connected and normalized operation on this first fused feature to obtain the first character of the title. That is, the server uses the decoder of the title generation model to perform a fully connected and normalized operation on the first fused feature to obtain the probability that the position of the first character of the title corresponds to multiple candidate characters. Based on the probabilities of these multiple candidate characters, the server determines the first character of the title from among them; for example, the server determines the candidate character with the highest probability among the multiple candidate characters as the first character of the title.

[0233] In subsequent iterations, the server uses the decoder of the title generation model to fuse the semantic features of the segmented text with the embedding features of the characters output from the previous iteration, obtaining a second fused feature. The server then uses the decoder of the title generation model to perform a fully connected and normalized operation on this second fused feature to obtain the other character in the title. In other words, the server uses the decoder of the title generation model to perform a fully connected and normalized operation on this second fused feature to obtain the probability that the position of the other character in the title corresponds to multiple candidate characters. Based on the probabilities of these multiple candidate characters, the server determines the other character in the title from among them; for example, the server determines the candidate character with the highest probability among the multiple candidate characters as the other character in the title.

[0234] 608. The server uses the title generation model to predict based on the semantic features of the split text and outputs the keywords in the split text.

[0235] In one possible implementation, the server uses the encoder of the title generation model to perform fully connected and normalized semantic features on the segmented text, outputting a probability set of different keywords corresponding to the segmented text. This probability set includes multiple probabilities, each corresponding to a candidate keyword. Based on this probability set, the server determines the keyword in the sample segmented text from among the multiple candidate keywords. For example, the server determines the candidate keyword corresponding to the highest probability in the probability set as the keyword in the sample segmented text; or, the server determines the candidate keywords corresponding to the N highest probabilities in the probability set as the keyword in the sample segmented text, where N is a positive integer.

[0236] All the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this specification, and will not be described in detail here.

[0237] The technical solution provided in the embodiments of this specification inputs the segmented text of a target video clip into a title generation model. This model encodes the segmented text using an attention mechanism to obtain its semantic features. These semantic features accurately reflect the semantics of the segmented text, and thus accurately reflect the semantics of the target video clip. By using this title generation model to predict the title of the target video clip based on the semantic features of the segmented text, the title can be obtained. The title generation process eliminates the need for manual operation by technicians, improving the efficiency of title generation.

[0238] Figure 7 This is a schematic diagram of the structure of a training device for a title generation model provided in an embodiment of this specification. See also... Figure 7 The device includes: a training text acquisition module 701, a training text encoding module 702, a training text prediction module 703, and a training module 704.

[0239] The training text acquisition module 701 is used to acquire the training text of the sample video segment. The training text includes the sample segment text of the sample video segment and the prefix text of the sample segment text. The prefix text is the text in the sample segment text that is masked and marked with keywords.

[0240] The training text encoding module 702 is used to input the training text into the title generation model. The title generation model encodes the training text based on the attention mechanism to obtain the semantic features of the training text.

[0241] The training text prediction module 703 is used to predict the keywords in the sample segment text and the predicted title of the sample video segment based on the semantic features of the training text through the title generation model.

[0242] Training module 704 is used to train the title generation model based on the first difference information between the predicted keyword and the labeled keyword, and the second difference information between the predicted title and the labeled title of the sample video segment.

[0243] In one possible implementation, the training text encoding module 702 is used to configure characters in the prefix text as global attention characters and characters in the sample segmented text as local attention characters using the title generation model. The global attention characters represent characters that perform attention operations with all characters in the training text, and the local attention characters represent characters that perform attention operations with a subset of characters in the training text. The title generation model performs attention encoding on multiple characters in the training text to obtain attention features for these characters. These attention features are then fused to obtain the semantic features of the training text.

[0244] In one possible implementation, the training text encoding module 702 is configured to, for any character among the plurality of characters, determine a first attention weight between the character and other characters in the training text if the character belongs to the prefix text. Based on the character and the first attention weight, determine the attention features of the character. If the character belongs to the sample segmented text, determine a second attention weight between the character and a reference character in the training text, the reference character being characters surrounding the character in the training text. Based on the character and the second attention weight, determine the attention features of the character.

[0245] In one possible implementation, the training text encoding module 702 is configured to, when the character belongs to the sample segmented text, determine the reference character in a sliding window that slides across the training text when the character is covered. The module determines the query matrix of the character and the key matrix of the reference character. Based on the query matrix of the character and the key matrix of the reference character, it determines a second attention weight between the character and the reference character in the training text.

[0246] In one possible implementation, the training text prediction module 703 is used to perform fully connected and normalized semantic features of the training text through the title generation model, and output predicted keywords in the sample video segment text. The title generation model then performs multiple rounds of iterative decoding based on the semantic features of the training text to output the predicted title of the sample video segment.

[0247] In one possible implementation, the training text prediction module 703 is used, during the first iteration, to output the first character of the predicted title of the sample video segment based on the semantic features of the training text and the start character through the title generation model. In subsequent iterations, the title generation model outputs the character corresponding to the current iteration based on the semantic features of the training text and the character output from the previous iteration.

[0248] In one possible implementation, the training module 704 is used to construct a joint loss function based on the first difference information and the second difference information. The title generation model is then trained based on this joint loss function.

[0249] In one possible implementation, the method for obtaining the labeled keywords of the sample text includes:

[0250] Keyword recognition was performed on the sample segment text of the sample video clip to obtain the labeled keywords of the sample segment text.

[0251] It should be noted that the training device for the title generation model provided in the above embodiments is only illustrated by the division of the above functional modules when training the title generation model. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the title generation device and the title generation method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0252] The technical solution provided in the embodiments of this specification trains a title generation model using training text from sample video clips. This training text includes sample segment text of the sample video clips and prefix text of the segment text. The prefix text is the text containing the labeled keywords in the sample segment text that is masked. This method of designing the training text allows the title generation model to simultaneously predict both keywords and titles. During training, the training text is encoded using an attention mechanism to obtain its semantic features. This attention mechanism enables the title generation model to focus on the parts of the training text that are helpful in predicting titles, improving the accuracy of the semantic features. Based on the semantic features of the training text, predicted keywords in the sample segment text and predicted titles of the sample video clips are obtained. The title generation model is trained based on a first difference between the predicted keywords and the labeled keywords, and a second difference between the predicted title and the labeled title. This allows the title generation model to generate titles efficiently and accurately.

[0253] Figure 8 This is a schematic diagram of the structure of a title generation device provided in the embodiments of this specification. See also... Figure 8 The device includes: a text segmentation acquisition module 801, a text segmentation encoding module 802, and a title prediction module 803.

[0254] The segmented text acquisition module 801 is used to acquire the segmented text of the target video segment, which is obtained by segmenting the video.

[0255] The text encoding module 802 is used to input the text into the title generation model, and the title generation model encodes the text based on the attention mechanism to obtain the semantic features of the text.

[0256] The title prediction module 803 is used to predict the title of the target video segment based on the semantic features of the split text through the title generation model.

[0257] The title generation model is trained based on the training text of the sample video clip, the labeled title of the sample video clip, and the labeled keywords of the sample segment text in the training text. The training text also includes the prefix text of the sample segment text, which is the text of the labeled keyword in the sample segment text that is masked.

[0258] In one possible implementation, the device further includes:

[0259] The keyword prediction module is used to predict keywords in the split text based on the semantic features of the title generation model.

[0260] In one possible implementation, the device further includes:

[0261] The reference keyword acquisition module is used to obtain reference keywords from the split text.

[0262] The text encoding module 802 is also used to input the text into the title generation model. Through the title generation model, characters in the reference keywords of the text are configured as global attention characters, and other characters in the text are configured as local attention characters. The global attention characters represent characters that perform attention operations with all characters in the text, and the local attention characters represent characters that perform attention operations with some characters in the text. The title generation model performs attention encoding on multiple characters in the text to obtain attention features of multiple characters. The attention features of multiple characters in the text are then fused to obtain the semantic features of the text.

[0263] In one possible implementation, the text strip acquisition module 801 is configured to perform any of the following:

[0264] Speech recognition is performed on the audio of the target video segment to obtain the segmented text of the target video segment.

[0265] The portion of the video subtitles corresponding to the target video segment is identified as the segmented text of the target video segment.

[0266] It should be noted that the title generation device provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the title generation device and the title generation method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0267] The technical solution provided in the embodiments of this specification inputs the segmented text of a target video clip into a title generation model. This model encodes the segmented text using an attention mechanism to obtain its semantic features. These semantic features accurately reflect the semantics of the segmented text, and thus accurately reflect the semantics of the target video clip. By using this title generation model to predict the title of the target video clip based on the semantic features of the segmented text, the title can be obtained. The title generation process eliminates the need for manual operation by technicians, improving the efficiency of title generation.

[0268] This specification also provides a computer storage medium that can store multiple program instructions adapted for loading and execution by a processor of the scheme described in the above method embodiments, which will not be elaborated further here.

[0269] This specification also provides a computer program product that stores at least one instruction, which is loaded by a processor and executed as described in the above method embodiments, and will not be repeated here.

[0270] Please see Figure 9 This diagram illustrates the structure of an electronic device provided in an exemplary embodiment of this specification, which can be provided as a server or a terminal. The electronic device in this specification may include one or more of the following components: a processor 99, a memory 920, an input device 930, an output device 940, and a bus 960. The processor 99, memory 920, input device 930, and output device 940 can be connected via the bus 960.

[0271] Processor 99 may include one or more processing cores. Processor 99 connects to various parts of the electronic device through various interfaces and lines, and performs various functions and processes data of electronic device 900 by running or executing instructions, programs, code sets, or instruction sets stored in memory 920, and by calling data stored in memory 920. Optionally, processor 99 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 99 may integrate one or more of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 99 and may be implemented separately through a communication chip.

[0272] The memory 920 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 920 may include non-transitory computer-readable storage medium. The memory 920 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 920 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (e.g., touch function, sound playback function, image playback function, etc.), instructions for implementing the various method embodiments described below, etc. The operating system may be the Android system, including systems deeply developed based on the Android system, the iOS system developed by Apple Inc., including systems deeply developed based on the iOS system, or other systems.

[0273] In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to establish data communication between the third-party applications and the operating system. This would allow the operating system to obtain the current scenario information of the third-party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0274] The input device 930 is used to receive input instructions or data, and includes, but is not limited to, a keyboard, mouse, camera, microphone, or touch device. The output device 940 is used to output instructions or data, and includes, but is not limited to, a display device and a speaker. In one example, the input device 930 and the output device 940 can be combined, and both the input device 930 and the output device 940 can be a touch display screen.

[0275] In addition, those skilled in the art will understand that the structure of the electronic device shown in the above figures does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the electronic device may also include radio frequency circuits, input units, sensors, audio circuits, Wireless Fidelity (WiFi) modules, power supplies, Bluetooth modules, etc., which will not be described in detail here.

[0276] exist Figure 9 In the illustrated electronic device, the processor 99 can be used to call the title generation application stored in the memory 920 to execute the method described in the above method embodiments.

[0277] The above is an illustrative embodiment of an electronic device according to this specification. It should be noted that the technical solution of this electronic device belongs to the same concept as the technical solutions of the aforementioned title generation method and title generation model training method. Details not described in detail in the technical solution of the electronic device can be found in the description of the technical solution of the aforementioned title generation method.

[0278] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium for the computer program can be a magnetic disk, optical disk, read-only memory, or random access memory, etc.

[0279] The above are merely optional embodiments of this specification and are not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification shall be included within the scope of protection of this specification.

[0280] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.< / bos> < / start> < / eos> < / bos> < / bos> < / bos> < / bos> < / eos> < / stop> < / bos> < / start>

Claims

1. A method for training a title generation model, comprising: The training text of the sample video segment is obtained. The training text includes the sample segment text of the sample video segment and the prefix text of the sample segment text. The prefix text is the text marked with keywords in the sample segment text that is masked. The sample segment text is used to describe the content of the sample video segment. The training text is input into a title generation model. The model configures characters in the prefix text as global attention characters and characters in the sample segmented text as local attention characters. Global attention characters are those that perform attention operations with all characters in the training text, and local attention characters are those that perform attention operations with some characters in the training text. The title generation model then performs attention encoding on multiple characters in the training text to obtain attention features for those characters. Finally, the attention features of these multiple characters are fused to obtain the semantic features of the training text. The title generation model predicts the keywords in the sample segmented text and the predicted title of the sample video clip based on the semantic features of the training text. The title generation model is trained based on the first difference information between the predicted keywords and the labeled keywords, and the second difference information between the predicted title and the labeled title of the sample video segment.

2. The method according to claim 1, wherein performing attention encoding on multiple characters in the training text to obtain attention features of multiple characters in the training text includes: For any one of the plurality of characters, if the character belongs to the prefix text, a first attention weight is determined between the character and other characters in the training text; Based on the character and the first attention weight, determine the attention features of the character; In the case that the character belongs to the sample segmented text, a second attention weight is determined between the character and a reference character in the training text, wherein the reference character is the character surrounding the character in the training text; Based on the character and the second attention weight, the attention features of the character are determined.

3. The method according to claim 2, wherein determining the second attention weight between the character and a reference character in the training text when the character belongs to the sample segmented text comprises: When the character belongs to the sample text, the reference character in the sliding window is determined when the character is covered, and the sliding window slides on the training text; Determine the query matrix of the character and the key matrix of the reference character; Based on the query matrix of the character and the key matrix of the reference character, a second attention weight is determined between the character and the reference character in the training text.

4. The method according to claim 1, wherein the step of predicting based on the semantic features of the training text using the title generation model, and outputting the predicted keywords in the sample segmented text and the predicted title of the sample video segment, comprises: The title generation model performs fully connected and normalized semantic features on the training text to output predicted keywords in the sample segmented text. The title generation model performs multiple rounds of iterative decoding based on the semantic features of the training text to output the predicted title of the sample video segment.

5. The method according to claim 4, wherein the step of performing multi-round iterative decoding based on the semantic features of the training text through the title generation model to output the predicted title of the sample video segment includes: In the first round of iteration, the title generation model iterates based on the semantic features and start character of the training text, and outputs the first character of the predicted title of the sample video segment; In other iterations, the title generation model outputs the corresponding character for the current iteration based on the semantic features of the training text and the characters output in the previous iteration.

6. The method according to claim 1, wherein training the title generation model based on the first difference information between the predicted keywords and the labeled keywords, and the second difference information between the predicted title and the labeled title of the sample video segment, comprises: A joint loss function is constructed based on the first difference information and the second difference information; The title generation model is trained based on the joint loss function.

7. The method according to claim 1, wherein the method for obtaining the labeled keywords of the sample segmented text includes: Keyword recognition is performed on the sample segment text of the sample video clip to obtain the labeled keywords of the sample segment text.

8. A method for generating a title, comprising: Obtain the segmented text of the target video segment, wherein the target video segment is obtained by splitting the video into segments, and the segmented text is used to describe the content of the target video segment. Obtain reference keywords from the split text; The split text is input into the title generation model. Through the title generation model, characters in the reference keywords in the split text are configured as global attention characters, and other characters in the split text are configured as local attention characters. The global attention characters represent characters that perform attention operations with all characters in the split text, and the local attention characters represent characters that perform attention operations with some characters in the split text. The title generation model is used to encode attention features of multiple characters in the split text to obtain attention features of multiple characters in the split text; the attention features of multiple characters in the split text are then fused to obtain semantic features of the split text. The title generation model predicts the title of the target video segment based on the semantic features of the segmented text. The title generation model is trained based on the training text of the sample video clips, the labeled titles of the sample video clips, and the labeled keywords of the sample segmented text in the training text. The training text also includes the prefix text of the sample segmented text, which is the text of the labeled keywords in the sample segmented text that is masked.

9. The method according to claim 8, wherein after encoding the segmented text using the title generation model based on an attention mechanism to obtain the semantic features of the segmented text, the method further includes: The title generation model predicts keywords from the split text based on its semantic features and outputs the keywords in the split text.

10. The method according to claim 8, wherein obtaining the segmented text of the target video segment includes any one of the following: Speech recognition is performed on the audio of the target video segment to obtain the segmented text of the target video segment; The portion of the video subtitles corresponding to the target video segment is determined as the segmented text of the target video segment.

11. A training device for a title generation model, comprising: The training text acquisition module is used to acquire the training text of the sample video segment. The training text includes the sample segment text of the sample video segment and the prefix text of the sample segment text. The prefix text is the text marked with keywords in the sample segment text that is masked. The sample segment text is used to describe the content of the sample video segment. A training text encoding module is used to input the training text into a title generation model. Through the title generation model, characters in the prefix text are configured as global attention characters, and characters in the sample segmented text are configured as local attention characters. The global attention characters represent characters that perform attention operations with all characters in the training text, and the local attention characters represent characters that perform attention operations with some characters in the training text. The title generation model then performs attention encoding on multiple characters in the training text to obtain attention features for these characters. Finally, the attention features of these multiple characters are fused to obtain the semantic features of the training text. The training text prediction module is used to predict the semantic features of the training text based on the title generation model, and output the predicted keywords in the sample segmented text and the predicted title of the sample video segment. The training module is used to train the title generation model based on the first difference information between the predicted keywords and the labeled keywords, and the second difference information between the predicted title and the labeled title of the sample video segment.

12. A title generation apparatus, comprising: The segmented text acquisition module is used to acquire the segmented text of a target video segment, wherein the target video segment is obtained by splitting the video into segments, and the segmented text is used to describe the content of the target video segment. The reference keyword acquisition module is used to acquire reference keywords in the split text; The text encoding module is used to input the text into a title generation model. Through the title generation model, characters in the reference keywords of the text are configured as global attention characters, and other characters in the text are configured as local attention characters. Global attention characters represent characters that perform attention operations with all characters in the text, and local attention characters represent characters that perform attention operations with some characters in the text. The title generation model then performs attention encoding on multiple characters in the text to obtain attention features of these characters. Finally, the attention features of these multiple characters are fused to obtain the semantic features of the text. The title prediction module is used to predict the title of the target video segment based on the semantic features of the segmented text using the title generation model. The title generation model is trained based on the training text of the sample video clips, the labeled titles of the sample video clips, and the labeled keywords of the sample segmented text in the training text. The training text also includes the prefix text of the sample segmented text, which is the text of the labeled keywords in the sample segmented text that is masked.

13. A computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the method as claimed in any one of claims 1 to 10.

14. An electronic device comprising: A processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and executed as described in any one of claims 1 to 10.

15. A computer program product comprising instructions that, when run on a computer or processor, causes the computer or processor to perform the method as described in any one of claims 1 to 10.

Images