Keyword recognition method and device

Abstract

The embodiments of this specification provide a keyword recognition method and device, wherein the keyword recognition method includes: obtaining a subtitle vector of a target video, and identifying a target keyword based on the subtitle vector and a preset fatigue control rule. Automatic recognition of target keywords in subtitle texts is achieved, and the recognition is more efficient. Moreover, since the fatigue control rule is set based on preset distribution information of keywords in subtitle texts, the identified target keywords can meet preset distribution conditions, so that the target keywords are more in line with actual needs, and the recognition results are more accurate in specific applications.

Description

Technical Field

[0001] The embodiments of the present specification relate to the field of video processing technology, and in particular to a keyword recognition method. Background Art

[0002] With the development of Internet technology, various types of videos are currently the most popular content carriers. Videos with subtitles make it easier for people to watch videos in various scenarios. Nowadays, the forms of video subtitles are becoming more and more diverse. A variety of video subtitles not only facilitate people's viewing, but also provide viewers with more interactivity and fun, improving the viewing experience of viewers.

[0003] At present, the style editing of subtitles still relies more on manual recognition of keywords and then style rendering of keywords. As the requirements for the video content creation cycle and quantity are becoming increasingly stringent, such manual recognition methods have a great impact on the efficiency of video creation. Therefore, an efficient and accurate keyword recognition method is urgently needed. Summary of the invention

[0004] In view of this, an embodiment of this specification provides a keyword recognition method. One or more embodiments of this specification also relate to a keyword recognition device, a computing device, a computer-readable storage medium and a computer program to solve the technical defects existing in the prior art.

[0005] According to a first aspect of an embodiment of this specification, a keyword recognition method is provided, including:

[0006] For a target video, obtaining a subtitle vector of the target video;

[0007] According to the subtitle vector, based on a preset fatigue control rule, a target keyword is identified, wherein the fatigue control rule is set based on preset distribution information of the keywords in the subtitle text.

[0008] According to a second aspect of the embodiments of this specification, a keyword recognition device is provided, including:

[0009] An acquisition module is configured to acquire a subtitle vector of a target video for the target video;

[0010] The recognition module is configured to recognize target keywords according to the subtitle vector and based on a preset fatigue control rule, wherein the fatigue control rule is set based on preset distribution information of keywords in the subtitle text.

[0011] According to a third aspect of an embodiment of this specification, a computing device is provided, including:

[0012] Memory and processor;

[0013] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the above-mentioned keyword recognition method are implemented.

[0014] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the above-mentioned keyword recognition method are implemented.

[0015] According to a fifth aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned keyword identification method.

[0016] In one embodiment of the specification, for a target video, a subtitle vector of the target video is obtained, and a target keyword is identified based on the subtitle vector and a preset fatigue control rule. Automatic identification of target keywords in subtitle text is achieved, and the identification is more efficient. Moreover, since the fatigue control rule is set based on preset distribution information of keywords in the subtitle text, the identified target keywords can meet the preset distribution conditions, so that the target keywords are more in line with actual needs, and the identification results are more accurate in specific applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a flowchart of a keyword recognition method provided by an embodiment of this specification;

[0018] Figure 2 is a flowchart of a processing process of a keyword identification method provided by an embodiment of this specification;

[0019] Figure 3 It is a flowchart of a keyword recognition method based on a keyword recognition model provided by an embodiment of this specification;

[0020] Figure 4 It is a structural schematic diagram of a keyword recognition device provided by an embodiment of this specification;

[0021] Figure 5 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0022] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0023] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0024] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0025] First, the terms involved in one or more embodiments of this specification are explained.

[0026] Cascading Style Sheets (CSS): is a language that defines text style structures such as font, color, position, etc. CSS can statically render the font size, color, font and other styles of text, and can also dynamically add animation effects to the text with various scripting languages ​​for rendering and display.

[0027] Transformer: is a natural language processing model that combines an attention mechanism and can obtain more accurate processing results based on the context.

[0028] Feedforward Neural Network (FNN): A type of neural network used for classification. Neurons are arranged in layers, and each neuron is only connected to neurons in the previous layer. It receives the output of the previous layer and outputs it to the next layer, with no feedback between layers.

[0029] In this specification, a keyword recognition method is provided. This specification also relates to a keyword recognition device, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0030] See also Figure 1 , Figure 1A flowchart of a keyword recognition method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0031] Step 102: For a target video, obtain a subtitle vector of the target video.

[0032] The target video is a video containing subtitles, which can be a video in a video database, a video on a video server, or a video in a storage medium of a video capture device, including short videos, long videos, etc. The video format of the target video can be MPEG, AVI, WMV, etc., which is not limited here.

[0033] The subtitle vector is the vector obtained after the subtitle text in the target video is vectorized, such as tools such as word2vec. The specific implementation method is: first perform word segmentation on the text, such as the jieba word segmentation tool. Secondly, sequentially mark the words in the text to obtain position labels. For example, the text "The weather is good today" is sequentially numbered, and the position label vector obtained is "(today) 1-(weather) 2-(good) 3". Then, unify the text length. For example, the text "The weather is good today" and "The weather will be better tomorrow" will have inconsistent text lengths, which will cause inconsistent vector dimensions in the subsequent vectorization process. All texts need to be of the same length, and missing positions are filled with "0". Finally, use feature encoding to encode, convert each text into a feature matrix, and then input the text vector model to obtain a text vector.

[0034] Obtaining the subtitle vector of the target video may be to obtain the subtitle text corresponding to the target video, and then vectorize the subtitle text to obtain the subtitle vector. Alternatively, after obtaining the target video, the video may be converted from audio to text to obtain the subtitle text, and then vectorize the subtitle vector to obtain the subtitle vector. The target video may also be input into a font extraction model to extract the subtitle vector in the video, which is not limited here. In addition, manual transcription may be performed to obtain the subtitle text through the target video, and then the subtitle text may be vectorized to obtain the subtitle vector.

[0035] The source of the target video may be retrieved from a video database, downloaded from a video server, or directly obtained from a storage medium where the video is shot, which is not limited here.

[0036] Specifically, for a target video, the subtitle text of the target video is vectorized to obtain a subtitle vector of the target video.

[0037] For example, for a news short video on a certain video platform, after downloading the news short video from the video server of the video platform, the subtitle text of the target video "XX product is on sale in November" is vectorized to obtain the subtitle vector of the target video {(XX) 45, (product) 123, (on) 12, (November) 99, (big) 76, (sale) 154}.

[0038] By obtaining the subtitle vector of the target video, a vector basis is provided for subsequent keyword recognition.

[0039] Step 104: According to the subtitle vector, based on a preset fatigue control rule, the target keyword is identified, wherein the fatigue control rule is set based on preset distribution information of the keyword in the subtitle text.

[0040] Keywords are words that can be style rendered in subtitle text to make the subtitle text of the video present various effects in the video. Style rendering includes: font style rendering, font size style rendering, animation effect style rendering, etc. This specification embodiment takes animation effect style rendering as an example for explanation.

[0041] The preset fatigue rules are user viewing experience rules that need to be considered when rendering subtitle text in a style. For example, if there are too many keywords in the subtitle text after style rendering, the user experience will be poor. Or, in multiple subtitle texts, each subtitle text is rendered in a keyword style, and the user experience is also poor. For these situations, it is necessary to consider the distribution of keywords in the subtitle text, and pre-set fatigue rules based on the keyword distribution information of the subtitle text. While satisfying the requirement that videos containing diverse subtitle text renderings enhance user viewing interactivity and fun, the user's normal viewing experience will not be disrupted due to unreasonable keyword rendering.

[0042] Specifically, according to the distribution of keyword vectors in the subtitle vectors and based on preset fatigue control rules, target keywords are identified.

[0043] Continuing with the above example, the keywords in the subtitle vector {(XX)45, (product)123, (on)12, (November)99, (big)76, (discount)154} are "(big)76" and "(discount)154". The keyword vectors "(big)76" and "(discount)154" appear in the subtitle vector of the above sentence. If the keywords corresponding to the above two keyword vectors are rendered in a style, the user experience will be poor. Therefore, based on the preset fatigue control rule, "big" is determined as the target keyword, and the target keyword "big" is screened out.

[0044] In one embodiment of the specification, for a target video, a subtitle vector of the target video is obtained, and a target keyword is identified based on the subtitle vector and a preset fatigue control rule. Automatic identification of target keywords in subtitle text is achieved, and the identification is more efficient. Moreover, since the fatigue control rule is set based on preset distribution information of keywords in the subtitle text, the identified target keywords can meet the preset distribution conditions, so that the target keywords are more in line with actual needs, and the identification results are more accurate in specific applications.

[0045] Optionally, step 102 of obtaining a subtitle vector of a target video includes the following specific steps:

[0046] For a target video, obtain audio data of the target video;

[0047] According to the audio data, a subtitle vector corresponding to the audio data is obtained.

[0048] The audio data of the target video is the audio data of the target video, and the format of the audio data can be WAV, FLAC, MP3, etc.

[0049] The method for obtaining the audio data of the target video can be to use a format conversion tool to convert the target video file into an audio file to obtain the audio data, or to use the audio track stripping technology of the video editing software to strip the audio track of the target video from the target video and then obtain the audio data, or to use a transcription tool to transcribe the audio of the target video to obtain the audio data, which is not limited here.

[0050] According to the audio data, a subtitle vector corresponding to the audio data is obtained. First, the audio data is converted from audio to text. The specific conversion methods include: audio input method conversion, audio-text conversion model conversion, and audio-text conversion module conversion on various voice interactive software, which are not limited here. Then, the converted text is vectorized to obtain a subtitle vector. The above step 102 has been described in detail and will not be repeated here.

[0051] Specifically, for a target video, audio data of the target video is obtained, the audio data is converted into text to obtain subtitle text, and the subtitle text is vectorized to obtain a subtitle vector.

[0052] For example, for a target video in MPEG format, the audio data of the target video in MP3 format is obtained through a format conversion tool, the audio data is input into an audio-to-text conversion model to obtain subtitle text, and the subtitle text is vectorized using the Word2vec tool to obtain a subtitle vector.

[0053] By obtaining the subtitle vector corresponding to the audio data based on the audio data of the target video, it is possible to automatically process the audio data of the target video to obtain the corresponding subtitle vector, thereby improving the processing efficiency and further improving the efficiency of keyword recognition.

[0054] Optionally, after obtaining the audio data of the target video, the following specific steps are further included:

[0055] Perform spectrum analysis on the audio data to obtain a spectrum vector of the audio data;

[0056] Correspondingly, according to the subtitle vector, based on the preset fatigue control rule, the target keyword is identified, including the following specific steps:

[0057] Concatenate the subtitle vector and the spectrum vector to obtain the target concatenated vector;

[0058] According to the target splicing vector, the target keywords are identified based on the preset fatigue control rules.

[0059] Performing spectrum analysis on audio data is the process of analyzing the spectrum of audio data to obtain spectrum vector after considering the characteristics of subtitles. It includes processing methods such as filtering, noise removal, and conversion of time vector and spectrum vector. Taking filtering as an example, the audio data is a song containing human voice and instrumental sound. The instrumental sound cannot be converted into subtitles. The characteristic of subtitles is human voice, so the instrumental sound needs to be processed. The specific processing method is to configure a filter of corresponding frequency according to the frequency of human voice. After the audio data is input into the filter, because there is a difference between the frequency of instrumental sound and the frequency of human voice, audio data with human voice characteristics is obtained, and then the audio data with human voice characteristics is vectorized to obtain a spectrum vector.

[0060] The methods for vectorizing audio data mainly include vector quantization (VQ), audio coding model processing methods, etc., which are not limited here.

[0061] The subtitle vector and the spectrum vector are spliced ​​to obtain the target splicing vector, which can be directly spliced ​​from beginning to end of the subtitle vector and the spectrum vector, or spliced ​​in an interlaced manner. For example, the subtitle vector is {12-33-4-55-9}, and the spectrum vector is {32-7-86-92-1}. After splicing from beginning to end, the target splicing vector {12-33-4-55-9-32-7-86-92-1} is obtained, and after splicing in an interlaced manner, the target splicing vector {12-32-33-7-4-86-55-92-9-1} is obtained.

[0062] Specifically, a spectral analysis is performed on the audio data, and a spectral vector of the audio data is obtained according to the subtitle features of the subtitle vector. The subtitle vector and the spectral vector are spliced ​​to obtain a target splicing vector, and according to the target splicing vector and based on preset fatigue control rules, the target keyword is identified.

[0063] For example, the audio data is subjected to spectral analysis, and the subtitle features of the subtitle vector are natural sentences. The audio data is subjected to denoising, and the spectrum of non-natural sentence features is eliminated to obtain a spectrum vector of the audio data. The subtitle vector and the spectrum vector are concatenated end to end to obtain a target concatenated vector. According to the target concatenated vector and based on preset fatigue control rules, the target keywords are identified.

[0064] By performing spectral analysis on the audio data, the spectral vector of the audio data is obtained, and the spectral vector is spliced ​​with the previously obtained subtitle vector to obtain a target splicing vector for subsequent keyword recognition, which can make keyword recognition with audio spectral characteristics more accurate.

[0065] Optionally, obtaining a subtitle vector corresponding to the audio data according to the audio data includes the following specific steps:

[0066] Obtaining subtitle text obtained by converting audio data into text, as well as timestamp information of each word in the subtitle text;

[0067] Obtain word features of each word in the subtitle text, and determine the subtitle vector corresponding to the audio data according to the word features of each word;

[0068] Correspondingly, the subtitle vector and the spectrum vector are concatenated to obtain the target concatenated vector, including the following specific steps:

[0069] Based on the timestamp information of each word, the word features and spectrum features with the same timestamp information in the subtitle vector and the spectrum vector are spliced ​​to obtain the target splicing vector.

[0070] The subtitle text contains one or more sentences, and a sentence contains one or more words. According to the time information of the audio data, the time stamp information of each word in the subtitle text is determined corresponding to the time of each word. The specific determination method can be manual labeling or audio recognition followed by automatic labeling, which is not limited here.

[0071] The word features of each word include the word's position feature, type feature, weight feature, etc.

[0072] Get the word features of each word in the subtitle text, and determine the subtitle vector corresponding to the audio data based on the word features of each word. You can query a pre-set word feature table, then annotate each word, and then encode based on the feature label to obtain the corresponding subtitle vector. You can also input the subtitle text into a text feature encoding model, embed the word features of each word through the embedding layer of the text feature encoding model, and then obtain the subtitle vector through the encoding layer.

[0073] Based on the timestamp information of each word, the word features and spectrum features with the same timestamp information in the subtitle vector and the spectrum vector are spliced. The splicing method is to splice the vector of each word in the subtitle vector and the spectrum vector of the corresponding timestamp according to the timestamp information to obtain a word splicing vector. The above operation is performed on all words in turn to obtain the target splicing vector.

[0074] Specifically, obtain the subtitle text obtained after text conversion of the audio data and the timestamp information of each word in the subtitle text, obtain the word features of each word in the subtitle text, determine the subtitle vector corresponding to the audio data according to the word features of each word, and based on the timestamp information of each word, splice the word features and spectrum features with the same timestamp information in the subtitle vector and the spectrum vector to obtain the target splicing vector.

[0075] For example, the subtitle text obtained after audio data conversion is "XX product is on sale in November", and the timestamp information corresponding to the words is {00s, 01s, 03s, 04s, 05s, 06s}. The word features of each word (XX, product, in, November, big, sale) in the subtitle text are obtained, and the subtitle vector is determined to be {(XX) 45, (product) 123, (in) 12, (November) 99, (big) 76, (sale) 1 54}, based on the timestamp information of each word {00s, 01s, 03s, 04s, 05s, 06s}, the spectrum vector is {F1, F2, F3, F4, F5, F6, F7}, and the corresponding timestamp information is {00s, 01s, 02s, 03s, 04s, 05s, 06s}. The word features and spectrum features with timestamp information in the subtitle vector and the spectrum vector are concatenated to obtain the target concatenated vector {45- F1, 123-F2, 12- F4, 99- F5, 76-F6, 154-F7}.

[0076] The subtitle vector is determined based on the word features of the words in the subtitle text, and then the subtitle vector and the spectrum vector are spliced ​​according to the timestamp information of each word to obtain the target splicing vector. The target splicing vector can accurately represent the word features and spectrum vector of the subtitle vector at the same time, which provides a vector basis for subsequent target keyword recognition and obtaining more accurate target keywords.

[0077] Optionally, step 104 of identifying target keywords according to the subtitle vector based on a preset fatigue control rule includes the following specific steps:

[0078] The subtitle vector is input into a pre-trained keyword recognition model to obtain target keywords, wherein the keyword recognition model is trained based on sample texts, the sample texts carry label information, and the label information represents preset distribution information of the keywords in the sample texts.

[0079] The pre-trained keyword recognition model is a keyword recognition model with an attention mechanism. It is obtained by pre-training the initial keyword recognition model and has encoding and fatigue control functions. The attention mechanism can combine the characteristics of rural vocabulary to obtain more accurate word recognition results. The training process can be unsupervised training or supervised training, which is not limited here.

[0080] The sample text can be obtained from an open source sample text database annotated with label information, or obtained from an open source sample text database of text without label information and then manually annotated with label information. The sample can also be obtained by extracting subtitle text from multiple videos and then annotating with label information.

[0081] The preset distribution information is the distribution information that characterizes the keywords and key sentences in the sample text. For example, in the sample text "Metro Line 2 is about to open", the two keywords "Metro" and "Line 2" are in the sample text. In addition, the position information of "Metro" and "Line 2" is 1 and 2, and the label information is {2 keywords, position: 1, 2}.

[0082] Specifically, the subtitle vector is input into a pre-trained keyword recognition model and processed by the attention mechanism to obtain the target keyword.

[0083] For example, the subtitle vector "{(XX) 45, (product) 123, (on) 12, (November) 99, (big) 76, (discount) 154}" is input into the pre-trained keyword recognition model, and after being processed by the attention mechanism, the target keyword "big" is obtained.

[0084] The target keyword is obtained by inputting the subtitle vector into a pre-trained keyword recognition model, wherein the keyword recognition model is trained based on sample text, and the sample text carries label information, and the label information represents the preset distribution information of the keywords in the sample text. The keyword recognition model can obtain the preset distribution information of the keywords based on the label information of the keywords, and obtain a more accurate target keyword recognition result.

[0085] Optionally, the keyword recognition model includes an encoding layer and a fatigue control layer;

[0086] Input the subtitle vector into the pre-trained keyword recognition model to obtain the target keyword, including the following specific steps:

[0087] Input the subtitle vector into the encoding layer to obtain the semantic representation vector;

[0088] The semantic representation vector is input into the fatigue control layer, and the cross attention mechanism is used to obtain the keyword probability distribution;

[0089] Determine the target keywords based on the keyword probability distribution.

[0090] The encoding layer can encode the word feature vector in the subtitle vector according to the word features of the words in the subtitle vector, integrate the semantic features of the subtitle vector, and obtain a semantic representation vector.

[0091] The fatigue control layer can use the cross-attention mechanism to obtain the probability distribution of keywords based on the semantic features of the subtitle vector, and determine the target keyword based on the probability distribution of keywords. This can meet the user's fatigue requirements. For example, if there are multiple keywords in a sentence, the target keyword can be selectively screened instead of rudely determining each keyword as the target keyword. Or, if there are multiple consecutive sentences containing keywords, fatigue control is required to prevent multiple sentences from containing the target keyword, which will affect the user experience.

[0092] The cross-attention mechanism is an attention mechanism that can capture the contextual connections within and between sentences. It can make the determined keyword probability distribution not only limited to the keyword, but can be combined with other words or other sentences in the sentence for analysis to obtain a more accurate keyword probability distribution, and thus determine a more accurate keyword recognition result.

[0093] The keyword distribution probability is used to characterize the probability of a keyword at a corresponding time and is determined based on the weight label of the word.

[0094] Based on the probability distribution of keywords, the weighted decoding result is obtained in combination with the Viterbi algorithm, and then the target keyword is determined according to the weighted result.

[0095] Specifically, the subtitle vector is input into the encoding layer to obtain the semantic representation vector, and the semantic representation vector is input into the fatigue control layer. The cross-attention mechanism is used to obtain the keyword probability distribution. Based on the keyword probability distribution, the weighted decoding result is obtained, and then the target keyword is determined according to the weight.

[0096] For example, the subtitle vector "{(XX) 45, (product) 123, (on) 12, (November) 99, (big) 76, (discount) 154}" is input into the encoding layer to obtain the semantic representation vector {46, 122, 20, 96, 74, 148}. The semantic representation vector {46, 122, 20, 96, 74, 148} is input into the fatigue control layer, and the cross-attention mechanism is used to obtain the keyword probability distribution {10%, 29%, 37%, 20%, 86%, 74%}. Based on the keyword probability distribution, the weighted decoding results {1.0, 2.9, 3.7, 2.0, 8.6, 7.4} are obtained, and then the target keyword is determined to be "big" corresponding to 8.6 according to the weight.

[0097] By inputting the subtitle vector into the encoding layer, the semantic representation vector is obtained;

[0098] The semantic representation vector is input into the fatigue control layer, and the cross attention mechanism is used to obtain the keyword probability distribution;

[0099] Determine the target keywords based on the keyword probability distribution.

[0100] By utilizing the attention mechanism of the fatigue control layer to obtain the key probability distribution and determine the target keywords, we can better combine the contextual connection and perform probability analysis and determine the target keywords based on the keyword probability distribution, making the identification of the target keywords more accurate.

[0101] Optionally, the training method of the keyword recognition model includes the following specific steps:

[0102] Acquire a sample text and obtain a text vector of the sample text, wherein the sample text carries label information, and the label information represents preset distribution information of keywords in the sample text;

[0103] Input the text vector into the encoding layer to obtain the sample semantic representation vector of the sample text;

[0104] The sample semantic representation vector is input into the fatigue control layer, and the cross attention mechanism is used to obtain the probability distribution of sample keywords;

[0105] According to the probability distribution of sample keywords, the sample keyword recognition result is obtained;

[0106] Calculate the loss value based on the sample keyword recognition results and label information;

[0107] According to the loss value, the model parameters of the encoding layer and the fatigue control layer are adjusted, and the step of obtaining the sample text and obtaining the text vector of the sample text is returned.

[0108] The method for calculating the loss value can be to calculate the loss value through a pre-set loss function, or to compare the label information in the recognition result with the label information in the sample text to calculate the loss value. For example, the label information obtained by the recognition result is {o,b1,b2,b3,e,o,i}, and the label information in the sample text is {o,b2,b3,b3,e,o,i}. After comparison, it is found that two label information are inconsistent, and the loss value is 2 / 6, that is, 33.3%.

[0109] Return to the step of obtaining sample text and obtaining the text vector of the sample text. That is, the model is iteratively trained and the model parameters are continuously adjusted through the loss value, so that the model can more accurately identify keywords and improve the accuracy of recognition.

[0110] By calculating the sample keyword recognition results and label information, the loss value is calculated, and then the model parameters of the encoding layer and the fatigue control layer are adjusted according to the loss value. The model is iteratively trained to enable the model to recognize keywords more accurately and improve the recognition accuracy.

[0111] Optionally, the preset distribution information includes first preset distribution information of keywords in a single sentence; the tag information includes keyword tag information, and the keyword tag information is pre-marked based on the first preset distribution information;

[0112] Input the text vector into the encoding layer to obtain the sample semantic representation vector of the sample text, including the following specific steps:

[0113] Input the text vector into the encoding layer to obtain the word-level semantic representation vector in the sample text;

[0114] The sample semantic representation vector is input into the fatigue control layer, and the cross attention mechanism is used to obtain the sample keyword probability distribution, including:

[0115] The word-level semantic representation vector is input into the fatigue control layer, and the cross-attention mechanism is used to obtain the probability distribution of sample keywords;

[0116] According to the sample keyword recognition results and label information, the loss value is calculated, including:

[0117] Calculate the loss value based on the sample keyword recognition results and keyword label information.

[0118] The first preset distribution information of the keywords is the position distribution information representing the keywords in the sentences in the sample text.

[0119] By inputting the text vector containing the first preset distribution information of the keyword into the encoding layer and obtaining the word-level semantic representation vector in the sample text, the keyword recognition model can calculate the probability distribution of the keyword more accurately, and the obtained keyword label information is more accurate, and the calculation of the loss value is more effective, so that a keyword recognition model with more accurate recognition performance can be obtained. In addition, the convergence of the model training is accelerated, thereby improving the training efficiency.

[0120] Optionally, the preset distribution information further includes second preset distribution information of key sentences in the sample text, where the key sentences are sentences containing keywords; the label information further includes key sentence label information, where the key sentence label information is pre-labeled based on the second preset distribution information;

[0121] After the text vector is fed into the encoding layer, it also includes:

[0122] Obtain sentence-level semantic representation vectors in sample texts;

[0123] Input the sentence-level semantic representation vector into the fatigue control layer to obtain the key sentence recognition result;

[0124] According to the sample keyword recognition results and keyword tag information, the loss value is calculated, including:

[0125] Calculate a first loss value based on the sample keyword recognition result and keyword tag information;

[0126] Calculate the second loss value according to the key sentence recognition result and the key sentence label information;

[0127] A total loss value is determined based on the first loss value and the second loss value.

[0128] The second preset distribution information of the key sentence is the position distribution information of the key sentence containing the keyword in the sample text among multiple sentences.

[0129] By inputting the text vector containing the first preset distribution information of keywords and the second preset distribution information of key sentences into the encoding layer, the word-level semantic representation vector and the sentence-level semantic representation vector in the sample text are obtained, which can make the keyword recognition model more accurate in the probability distribution calculation of keywords and the judgment of key sentences, and thus the obtained keyword label information and the judgment results of key sentences are more accurate, and the calculation of the total loss value is more effective, so that a keyword recognition model with more accurate recognition performance can be obtained. In addition, the convergence of model training is accelerated, thereby improving training efficiency.

[0130] Optionally, obtaining a sample text and obtaining a text vector of the sample text includes the following specific steps:

[0131] Acquire a sample video, obtain sample audio data of the sample video, perform spectrum analysis on the sample audio data, and obtain a sample spectrum vector of the sample audio data;

[0132] Obtaining a sample text obtained after converting the sample audio data into text, and timestamp information of each sample word in the sample text;

[0133] Obtaining word features of each sample word in the sample text, and determining a text vector of the sample text according to the word features of each sample word;

[0134] Based on the timestamp information of each sample word, the word features and spectrum features with the same timestamp information in the text vector and the sample spectrum vector are concatenated to obtain a sample concatenated vector;

[0135] Input the text vector into the encoding layer to obtain the sample semantic representation vector of the sample text, including:

[0136] The sample concatenation vector is input into the encoding layer to obtain the sample semantic representation vector.

[0137] By performing spectral analysis on sample audio data, a spectral vector of the sample audio data is obtained, and the spectral features of the sample spectral vector and the word features of the sample subtitle vector are spliced ​​to obtain a sample splicing vector. This allows the training samples of the model to contain more information, and a keyword recognition model with more accurate recognition performance can be obtained. In addition, targeted keyword recognition can be performed on the text to be recognized that contains both spectrum and text.

[0138] Optionally, after step 104, the keyword recognition method may further include the following steps:

[0139] Get the type information of the target keyword;

[0140] Determine the target display template corresponding to the target keyword according to the type information;

[0141] Utilize the target display template to render the target keywords in the target video.

[0142] The type information of the target keyword is the corresponding rendering effect type when the target keyword is rendered. For example, for the keyword "big", the keyword can be rendered in a larger font size style, for the keyword "yellow", the keyword can be rendered in a font color style, and for the keyword "bang", the keyword can be rendered with an explosion effect animation effect.

[0143] The target display template is a pre-set keyword rendering template that can be selected according to type information. One template can be set for multiple types of keywords, or a corresponding template can be set for each type of keyword. There is no limitation here.

[0144] The target display template is used to render the target keyword in the target video. The specific rendering method may be Cascading Style Sheet (CSS) rendering.

[0145] According to the subtitle vector and based on the preset fatigue control rules, the target keyword and the type information of the target keyword are identified. According to the type information, the target display template corresponding to the target keyword is determined. The target display template is used to render the target keyword in the target video, so that keywords of different types can obtain corresponding rendering effects and be displayed, thereby improving the user's viewing experience.

[0146] The following combination Figure 2 , taking the application of the keyword recognition method provided in this specification in the video subtitles of the action word video as an example, the keyword recognition method is further explained. Among them, Figure 2 A flowchart of a processing process of a keyword recognition method provided by an embodiment of the present specification is shown, which specifically includes the following steps.

[0147] Step 202: For a target video, obtain audio data of the target video.

[0148] For the target video containing the action words, obtain the audio data XXX.MP3 of the target video.

[0149] Step 204: Perform spectrum analysis on the audio data to obtain a spectrum vector of the audio data.

[0150] After performing spectrum analysis on the audio data XXX.MP3, targeted identification is performed on the spectrum therein, and other spectrums are filtered to obtain the spectrum vector {F1, F2, F3, F4, F5, F6, F7, F8} of the audio data.

[0151] Step 206: Obtain the subtitle text obtained after converting the audio data into text, and the timestamp information of each word in the subtitle text.

[0152] The audio data is input into the audio-to-text conversion model to obtain the subtitle text "Persistence is victory" after text conversion, as well as the timestamp information {18s, 22s, 24s, 28s} of each word in the subtitle text (persistence, persistence, is, victory).

[0153] Step 208: Obtain word features of each word in the subtitle text, and determine the subtitle vector corresponding to the audio data based on the word features of each word.

[0154] Get the word features (Token1, Token2, Token3, Token4) of each word in the subtitle text (persistence, keep going, that is, victory), input the text vector encoding model, and get the corresponding subtitle vector {Token1, Token2, Token3, Token4}.

[0155] Step 210: Based on the timestamp information of each word, the word features and spectrum features with the same timestamp information in the subtitle vector and the spectrum vector are concatenated to obtain a target concatenated vector.

[0156] Based on the timestamp information {18s, 22s, 24s, 28s} of each word, the word features (Token1, Token2, Token3, Token4) and spectrum features (F2, F4, F6, F8) with the same timestamp information {18s, 22s, 24s, 28s} in the subtitle vector {Token1, Token2, Token3, Token4} and the spectrum vector {F1, F2, F3, F4, F5, F6, F7, F8} are concatenated to obtain the target concatenated vector {Token1-F2, Token2-F4, Token3-F6, Token4-F8}.

[0157] Step 212: Input the target concatenated vector into the encoding layer to obtain a semantic representation vector.

[0158] The target concatenated vector {Token1- F2, Token2- F4, Token3- F6, Token4- F8} is input into the encoding layer to obtain the semantic representation vector {Token0, Token1, F2, Token2, F4, Token0, Token3, F6, Token4.F8}.

[0159] Step 214: Input the semantic representation vector into the fatigue control layer and use the cross attention mechanism to obtain the keyword probability distribution.

[0160] The semantic representation vector {Token0, Token1, F2, Token2, F4, Token3, F6, Token4.F8} is input into the fatigue control layer, and the cross attention mechanism is used to obtain the keyword probability distribution (99%, 10%, 15%, 30%, 95%).

[0161] Step 216: Determine the tag information of the keyword based on the keyword probability distribution.

[0162] Based on the keyword probability distribution (99%, 10%, 15%, 30%, 95%), determine the keyword label information (1, i, i, o, b2, e, o, b1, e), where 1 indicates that the sentence contains the target keyword, i indicates other types, b1 indicates the highest probability keyword, b2 indicates the second highest probability keyword, o indicates the starting position of the keyword, and e indicates the ending position of the keyword.

[0163] Step 218: Determine the target keyword based on the tag information of the keyword.

[0164] Based on the tag information of the keyword (1, i, i, o, b2, e, o, b1, e), the target keyword is determined to be "victory", which is the action word in the video subtitle.

[0165] An embodiment of the present specification obtains audio data of a target video, obtains a spectrum vector based on spectrum analysis, performs text conversion on the audio data to obtain a subtitle text, performs vectorization processing to obtain a subtitle vector, and splices the word features in the subtitle vector and the spectrum features in the spectrum vector according to the timestamp information corresponding to the words to obtain a target splicing vector, so that a subsequent input model is used for target keyword recognition, which can effectively combine the subtitle audio spectrum features and the subtitle audio text features, and the recognition result is more accurate. Finally, a semantic representation vector is obtained by inputting the encoding layer of the model, and label information is obtained according to the probability distribution through the fatigue control layer. Finally, the target keyword is determined to be an action word, and fatigue control can be performed on the recognition of the target keyword to ensure that the action word obtained can be better adapted to the user's viewing experience in the subsequent application of the video subtitles.

[0166] Figure 3 A flowchart of a keyword recognition method based on a keyword recognition model provided by an embodiment of this specification is shown.

[0167] Among them, the keyword recognition model includes an encoding layer and a fatigue control layer. The encoding layer includes the concatenation of the subtitle vector and the spectrum vector, which are then input into a six-layer bidirectional Transformer model for bidirectional encoding to obtain a word-level semantic representation vector and a sentence-level semantic representation vector. The fatigue control layer includes a cross-attention layer, a feedforward neural network, a cache, and a two-layer unidirectional Transformer model. The sentence-level semantic representation vector is input into the feedforward neural network to determine whether the sentence contains keywords. The word-level semantic representation vector is input into the cross-attention layer, combined with the context vector and the hidden layer vector, and with the assistance of the cache, it is sequentially input into the two-layer unidirectional Transformer model to obtain the keyword label of each word.

[0168] Corresponding to the above method embodiment, this specification also provides a keyword recognition device embodiment, Figure 4FIG. 2 shows a schematic diagram of a keyword recognition device provided by an embodiment of the present specification. Figure 4 As shown, the device comprises:

[0169] The acquisition module 402 is configured to acquire a subtitle vector of a target video for the target video;

[0170] The identification module 404 is configured to identify the target keyword according to the subtitle vector and based on a preset fatigue control rule, wherein the fatigue control rule is set based on preset distribution information of the keyword in the subtitle text.

[0171] Optionally, the acquisition module 402 includes:

[0172] A first acquisition submodule is configured to acquire audio data of a target video for the target video;

[0173] The subtitle vector conversion submodule is configured to obtain the subtitle vector corresponding to the audio data based on the audio data.

[0174] Optionally, the acquisition module 402 further includes:

[0175] The spectrum vector conversion submodule is configured to perform spectrum analysis on the audio data to obtain the spectrum vector of the audio data;

[0176] Correspondingly, the identification module 404 includes:

[0177] A splicing submodule, configured to splice the subtitle vector and the spectrum vector to obtain a target splicing vector;

[0178] The first recognition submodule is configured to recognize the target keyword according to the target splicing vector and based on a preset fatigue control rule.

[0179] Optionally, the subtitle vector conversion submodule is further configured to obtain the subtitle text obtained after the audio data is converted into text, and the timestamp information of each word in the subtitle text; obtain the word feature of each word in the subtitle text, and determine the subtitle vector corresponding to the audio data according to the word feature of each word;

[0180] Correspondingly, the splicing submodule is further configured to splice the word features and spectrum features with the same timestamp information in the subtitle vector and the spectrum vector based on the timestamp information of each word, so as to obtain a target splicing vector.

[0181] Optionally, the recognition module 404 is further configured to input the subtitle vector into a pre-trained keyword recognition model to obtain a target keyword, wherein the keyword recognition model is trained based on a sample text, the sample text carries label information, and the label information represents preset distribution information of the keywords in the sample text.

[0182] Optionally, the keyword recognition model includes an encoding layer and a fatigue control layer;

[0183] The recognition module 404 is further configured to input the subtitle vector into the encoding layer to obtain a semantic representation vector; input the semantic representation vector into the fatigue control layer, and use the cross-attention mechanism to obtain a keyword probability distribution; and determine the target keyword based on the keyword probability distribution.

[0184] Optionally, the device further comprises:

[0185] The training module is configured to obtain sample text and obtain a text vector of the sample text, wherein the sample text carries label information, and the label information represents preset distribution information of keywords in the sample text; input the text vector into the encoding layer to obtain a sample semantic representation vector of the sample text; input the sample semantic representation vector into the fatigue control layer, and use the cross-attention mechanism to obtain the sample keyword probability distribution; obtain the sample keyword recognition result according to the sample keyword probability distribution; calculate the loss value according to the sample keyword recognition result and the label information; adjust the model parameters of the encoding layer and the fatigue control layer according to the loss value, and return to execute the steps of obtaining the sample text and obtaining the text vector of the sample text.

[0186] Optionally, the preset distribution information includes first preset distribution information of keywords in a single sentence; the tag information includes keyword tag information, and the keyword tag information is pre-marked based on the first preset distribution information;

[0187] The training module is further configured to input the text vector into the encoding layer to obtain the word-level semantic representation vector in the sample text; input the word-level semantic representation vector into the fatigue control layer, and use the cross-attention mechanism to obtain the sample keyword probability distribution; calculate the loss value based on the sample keyword recognition results and keyword label information.

[0188] Optionally, the preset distribution information further includes second preset distribution information of key sentences in the sample text, where the key sentences are sentences containing keywords; the label information further includes key sentence label information, where the key sentence label information is pre-labeled based on the second preset distribution information;

[0189] The training module is further configured to obtain a sentence-level semantic representation vector in a sample text; input the sentence-level semantic representation vector into a fatigue control layer to obtain a key sentence recognition result; calculate a first loss value based on the sample keyword recognition result and keyword label information; calculate a second loss value based on the key sentence recognition result and the key sentence label information; and determine a total loss value based on the first loss value and the second loss value.

[0190] Optionally, the training module is further configured to obtain a sample video, obtain sample audio data of the sample video, perform spectral analysis on the sample audio data, and obtain a sample spectrum vector of the sample audio data; obtain a sample text obtained after performing text conversion on the sample audio data, and timestamp information of each sample word in the sample text; obtain word features of each sample word in the sample text, and determine a text vector of the sample text based on the word features of each sample word; based on the timestamp information of each sample word, splice the word features and spectrum features with the same timestamp information in the text vector and the sample spectrum vector to obtain a sample splicing vector; and input the sample splicing vector into the encoding layer to obtain a sample semantic representation vector.

[0191] Optionally, the device further comprises:

[0192] The display module is configured to obtain type information of the target keyword; determine a target display template corresponding to the target keyword according to the type information; and render the target keyword in a target video using the target display template.

[0193] In one embodiment of the specification, for a target video, a subtitle vector of the target video is obtained, and a target keyword is identified based on the subtitle vector and a preset fatigue control rule. Automatic identification of target keywords in subtitle text is achieved, and the identification is more efficient. Moreover, since the fatigue control rule is set based on preset distribution information of keywords in the subtitle text, the identified target keywords can meet the preset distribution conditions, so that the target keywords are more in line with actual needs, and the identification results are more accurate in specific applications.

[0194] The above is a schematic scheme of a keyword recognition device of this embodiment. It should be noted that the technical scheme of the keyword recognition device and the technical scheme of the keyword recognition method described above are of the same concept, and the details not described in detail in the technical scheme of the keyword recognition device can be found in the description of the technical scheme of the keyword recognition method described above.

[0195] Figure 5 The block diagram of a computing device according to an embodiment of the present specification is shown. The components of the computing device 500 include but are not limited to a memory 510 and a processor 520. The processor 520 is connected to the memory 510 via a bus 530, and the database 550 is used to store data.

[0196] The computing device 500 also includes an access device 540 that enables the computing device 500 to communicate via one or more networks 560. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 540 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world microwave interconnection access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0197] In one embodiment of the present specification, the above components of the computing device 500 and Figure 5 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 5 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0198] Computing device 500 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. Computing device 500 may also be a mobile or stationary server.

[0199] The processor 520 is used to execute the following computer executable instructions, which, when executed by the processor, implement the steps of the above-mentioned keyword recognition method.

[0200] In one embodiment of the specification, for a target video, a subtitle vector of the target video is obtained, and a target keyword is identified based on the subtitle vector and a preset fatigue control rule. Automatic identification of target keywords in subtitle text is achieved, and the identification is more efficient. Moreover, since the fatigue control rule is set based on preset distribution information of keywords in the subtitle text, the identified target keywords can meet the preset distribution conditions, so that the target keywords are more in line with actual needs, and the identification results are more accurate in specific applications.

[0201] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above keyword recognition method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above keyword recognition method.

[0202] An embodiment of the present specification further provides a computer-readable storage medium storing computer-executable instructions, which can implement the steps of the above-mentioned keyword recognition method when executed by a processor.

[0203] In one embodiment of the specification, for a target video, a subtitle vector of the target video is obtained, and a target keyword is identified based on the subtitle vector and a preset fatigue control rule. Automatic identification of target keywords in subtitle text is achieved, and the identification is more efficient. Moreover, since the fatigue control rule is set based on preset distribution information of keywords in the subtitle text, the identified target keywords can meet the preset distribution conditions, so that the target keywords are more in line with actual needs, and the identification results are more accurate in specific applications.

[0204] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned keyword recognition method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned keyword recognition method.

[0205] An embodiment of the present specification further provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned keyword recognition method.

[0206] In one embodiment of the specification, for a target video, a subtitle vector of the target video is obtained, and a target keyword is identified based on the subtitle vector and a preset fatigue control rule. Automatic identification of target keywords in subtitle text is achieved, and the identification is more efficient. Moreover, since the fatigue control rule is set based on preset distribution information of keywords in the subtitle text, the identified target keywords can meet the preset distribution conditions, so that the target keywords are more in line with actual needs, and the identification results are more accurate in specific applications.

[0207] The above is a schematic scheme of a computer program of this embodiment. It should be noted that the technical scheme of the computer program and the technical scheme of the above keyword identification method belong to the same concept, and the details not described in detail in the technical scheme of the computer program can be referred to the description of the technical scheme of the above keyword identification method.

[0208] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0209] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0210] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0211] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0212] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A keyword recognition method, comprising: For a target video, obtaining a subtitle vector of the target video; Input the subtitle vector into the encoding layer of a pre-trained keyword recognition model to obtain a semantic representation vector, wherein the keyword recognition model includes the encoding layer and a fatigue control layer, the fatigue control layer includes a cross attention layer, a feedforward neural network, a cache and a two-layer unidirectional Transformer model, and the semantic representation vector includes a word-level semantic representation vector and a sentence-level semantic representation vector; Input the semantic representation vector into the fatigue control layer, use the cross-attention mechanism to obtain the keyword probability distribution, and determine the target keyword based on the keyword probability distribution, wherein the cross-attention mechanism is an attention mechanism that captures the contextual connection within a sentence and between sentences, and the keyword probability distribution represents the probability size of the keyword at the corresponding time. Input the semantic representation vector into the fatigue control layer, use the cross-attention mechanism to obtain the keyword probability distribution, and determine the target keyword based on the keyword probability distribution, including: input the sentence-level semantic representation vector into the feedforward neural network to determine whether the sentence contains the keyword; input the word-level semantic representation vector into the cross-attention layer to obtain the context vector and the hidden layer vector; input the context vector and the hidden layer vector into the two-layer unidirectional Transformer model through the cache to obtain the keyword label of each word; determine the keyword probability distribution according to the keyword label, and determine the target keyword based on the keyword probability distribution.

2. According to the method of claim 1, the step of obtaining a subtitle vector of a target video comprises: For a target video, obtaining audio data of the target video; According to the audio data, a subtitle vector corresponding to the audio data is obtained.

3. The method according to claim 2, after acquiring the audio data of the target video for the target video, further comprises: Performing spectrum analysis on the audio data to obtain a spectrum vector of the audio data; Splicing the subtitle vector and the spectrum vector to obtain a target splicing vector; The step of inputting the subtitle vector into an encoding layer in a pre-trained keyword recognition model to obtain a semantic representation vector comprises: The target concatenated vector is input into the encoding layer of a pre-trained keyword recognition model to obtain a semantic representation vector.

4. The method according to claim 3, wherein obtaining the subtitle vector corresponding to the audio data according to the audio data comprises: Acquire subtitle text obtained by converting the audio data into text, and timestamp information of each word in the subtitle text; Acquire word features of each word in the subtitle text, and determine a subtitle vector corresponding to the audio data according to the word features of each word; The step of splicing the subtitle vector and the spectrum vector to obtain a target splicing vector includes: Based on the timestamp information of each word, word features and spectrum features with the same timestamp information in the subtitle vector and the spectrum vector are spliced ​​to obtain a target splicing vector.

5. The method according to claim 1, wherein: The keyword recognition model is obtained by training based on sample texts, and the sample texts carry label information, and the label information represents preset distribution information of keywords in the sample texts.

6. According to the method of claim 5, the training method of the keyword recognition model comprises: Acquire a sample text and obtain a text vector of the sample text, wherein the sample text carries label information, and the label information represents preset distribution information of keywords in the sample text; Inputting the text vector into the encoding layer to obtain a sample semantic representation vector of the sample text; Inputting the sample semantic representation vector into the fatigue control layer, and using the cross attention mechanism to obtain the sample keyword probability distribution; Obtaining a sample keyword recognition result according to the sample keyword probability distribution; Calculating a loss value according to the sample keyword recognition result and the label information; According to the loss value, the model parameters of the encoding layer and the fatigue control layer are adjusted, and the step of obtaining the sample text and obtaining the text vector of the sample text is returned to be executed.

7. The method according to claim 6, wherein the preset distribution information comprises first preset distribution information of keywords in a single sentence; the tag information comprises keyword tag information, and the keyword tag information is pre-labeled based on the first preset distribution information; The step of inputting the text vector into the encoding layer to obtain a sample semantic representation vector of the sample text includes: Inputting the text vector into the encoding layer to obtain a word-level semantic representation vector in the sample text; The step of inputting the sample semantic representation vector into the fatigue control layer and using a cross attention mechanism to obtain a sample keyword probability distribution includes: Inputting the word-level semantic representation vector into the fatigue control layer, and using a cross-attention mechanism to obtain a sample keyword probability distribution; The calculating the loss value according to the sample keyword recognition result and the label information includes: A loss value is calculated based on the sample keyword recognition result and the keyword label information.

8. The method according to claim 7, wherein the preset distribution information further comprises second preset distribution information of key sentences in the sample text, wherein the key sentences are sentences containing keywords; the label information further comprises key sentence label information, wherein the key sentence label information is pre-labeled based on the second preset distribution information; After the text vector is input into the encoding layer, the method further includes: Obtaining a sentence-level semantic representation vector in the sample text; Inputting the sentence-level semantic representation vector into the fatigue control layer to obtain a key sentence recognition result; The calculating the loss value according to the sample keyword recognition result and the keyword tag information includes: Calculating a first loss value according to the sample keyword recognition result and the keyword tag information; Calculating a second loss value according to the key sentence recognition result and the key sentence label information; A total loss value is determined according to the first loss value and the second loss value.

9. The method according to claim 8, wherein obtaining the sample text and obtaining the text vector of the sample text comprises: Acquire a sample video, obtain sample audio data of the sample video, perform spectrum analysis on the sample audio data, and obtain a sample spectrum vector of the sample audio data; Acquire a sample text obtained by converting the sample audio data into text, and timestamp information of each sample word in the sample text; Acquire word features of each sample word in the sample text, and determine a text vector of the sample text according to the word features of each sample word; Based on the timestamp information of each sample word, concatenate the word features and spectrum features having the same timestamp information in the text vector and the sample spectrum vector to obtain a sample concatenated vector; The step of inputting the text vector into the encoding layer to obtain a sample semantic representation vector of the sample text includes: The sample concatenation vector is input into the encoding layer to obtain a sample semantic representation vector.

10. The method according to any one of claims 1 to 9, after inputting the semantic representation vector into the fatigue control layer, using the cross attention mechanism to obtain the keyword probability distribution, and determining the target keyword based on the keyword probability distribution, further comprising: Obtaining type information of the target keyword; Determining a target display template corresponding to the target keyword according to the type information; The target keyword is rendered in the target video using the target display template.

11. A keyword recognition device, comprising: An acquisition module is configured to acquire a subtitle vector of a target video for the target video; The recognition module is configured to input the subtitle vector into the encoding layer of a pre-trained keyword recognition model to obtain a semantic representation vector, wherein the keyword recognition model includes the encoding layer and the fatigue control layer, the fatigue control layer includes a cross-attention layer, a feedforward neural network, a cache and a two-layer unidirectional Transformer model, and the semantic representation vector includes a word-level semantic representation vector and a sentence-level semantic representation vector; input the semantic representation vector into the fatigue control layer, use the cross-attention mechanism to obtain the keyword probability distribution, and determine the target keyword based on the keyword probability distribution, wherein the cross-attention mechanism is an attention mechanism that captures the contextual connection within and between sentences, and the keyword probability distribution table The probability size of the keyword at the corresponding time, the semantic representation vector is input into the fatigue control layer, the keyword probability distribution is obtained by using the cross attention mechanism, and the target keyword is determined based on the keyword probability distribution, including: inputting the sentence-level semantic representation vector into the feedforward neural network to determine whether the sentence contains the keyword; inputting the word-level semantic representation vector into the cross attention layer to obtain a context vector and a hidden layer vector; inputting the context vector and the hidden layer vector into the two-layer unidirectional Transformer model through the cache to obtain the keyword label of each word; determining the keyword probability distribution according to the keyword label, and determining the target keyword based on the keyword probability distribution.

12. A computing device comprising: Memory and processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the keyword recognition method according to any one of claims 1 to 10 are implemented.

13. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the steps of the keyword recognition method according to any one of claims 1 to 10.

Images