Method, device, electronic device and storage medium for dividing topic paragraphs

Abstract

The present invention provides a topic paragraph division method, device, electronic device and storage medium. First, a target text is obtained; the target text is obtained based on transcribing a target audio, and the target text carries an audio segment identifier in the target audio, and the audio segment identifier includes a speaker identifier, an audience identifier and a silence identifier; then, based on the audio segment identifier, the target text is divided into sentences and punctuated to obtain a text to be divided; finally, the vector representation of each sentence in the text to be divided is determined, and the text to be divided is divided into topic paragraphs based on the vector representation of each sentence in the text to be divided and / or the audio segment identifier. The audio segment identifier in the target audio is introduced, the text to be divided is obtained in combination with the audio segment identifier, and the text to be divided is divided into topic paragraphs, which can make full use of the information contained in the target audio, reduce the influence of the credibility of the target text on the division result, and make the division result of the topic paragraph more accurate.

Description

Technical Field

[0001] The present invention relates to the technical field of text mining, and in particular to a method, device, electronic device and storage medium for dividing topic paragraphs. Background Art

[0002] With the development of technology, text recognition has been widely used in various fields. In order to achieve text recognition, it is very important to divide the text into topic paragraphs, especially for the topic paragraph division of the text obtained by speech transcription.

[0003] At present, the method for dividing the topic paragraphs of the text obtained by speech transcription is usually to use the Long Short-Term Memory (LSTM) model in deep learning, and treat the topic paragraph division of the text obtained by speech transcription as a pure natural language processing task.

[0004] The advantage of the above scheme is that it can process texts of any length, but due to errors in the speech transcription process, the credibility of the text obtained by speech transcription is reduced, affecting the segmentation results. Summary of the invention

[0005] The present invention provides a topic paragraph division method, device, electronic device and storage medium, which are used to solve the defects in the prior art.

[0006] The present invention provides a method for dividing topic paragraphs, comprising:

[0007] Obtain a target text; the target text is obtained by transcribing the target audio, the target text carries an audio segment identifier in the target audio, the audio segment identifier includes a speaker identifier, an audience identifier, and a silence identifier;

[0008] Based on the audio segment identifier, the target text is divided into sentences and punctuated to obtain a text to be divided;

[0009] Determine a vector representation of each sentence in the text to be divided, and divide the text to be divided into topic paragraphs based on the vector representation of each sentence in the text to be divided and / or the audio segment identifier.

[0010] According to a topic paragraph division method provided by the present invention, the target text is obtained based on the following method:

[0011] Extracting audio features of the target audio, and determining, based on the audio features, an audio segment corresponding to the audio segment identifier in the target audio;

[0012] The audio segment corresponding to the speaker identifier is converted into an audio transcription text, and the audio segment identifier is associated with the audio transcription text to obtain the target text.

[0013] According to a topic paragraph division method provided by the present invention, the audio features include filter bank features and Mel cepstral coefficients of each frame of audio in the target audio;

[0014] Accordingly, determining the audio segment corresponding to the audio segment identifier in the target audio based on the audio feature includes:

[0015] Determine the attention features of each frame of audio based on the filter bank features and Mel cepstral coefficients of each frame of audio;

[0016] Performing an affine transformation on the attention features of each frame of audio to obtain a category corresponding to each frame of audio;

[0017] Based on the categories corresponding to the audio frames, an audio segment corresponding to the audio segment identifier is determined.

[0018] According to a topic paragraph division method provided by the present invention, the topic paragraph division is performed on the text to be divided based on the vector representation of each sentence in the text to be divided, including:

[0019] Aggregating the sentences in the text to be divided based on the vector representation of each sentence in the text to be divided to determine a set of sentences contained in the text to be divided;

[0020] Based on the sentence sets, the topic paragraphs contained in the text to be divided are determined.

[0021] According to a method for dividing topic paragraphs provided by the present invention, the method aggregates the sentences in the text to be divided based on the vector representation of each sentence in the text to be divided to determine the topic paragraphs contained in the text to be divided, including:

[0022] For any sentence in the text to be divided, if there is a spare sentence set before any sentence, and the spare sentence set includes adjacent sentences of any sentence, then the vector representation of the spare sentence set is calculated, and the similarity between the vector representation of any sentence and the vector representation of the spare sentence set is calculated;

[0023] Based on the similarity, the any sentence is classified to obtain a classification result of the any sentence, and based on the classification result of each sentence in the text to be divided, a set of each sentence included in the text to be divided is determined.

[0024] According to a topic paragraph division method provided by the present invention, the topic paragraph division is performed on the text to be divided based on the audio segment identifier, including:

[0025] Based on the audio segment identifier, a paragraph recognition model is used to determine the score information corresponding to each sentence in the text to be divided; the score information is used to indicate the score of each sentence in the text to be divided as a paragraph ending sentence;

[0026] Determine each topic paragraph in the text to be divided based on the score information corresponding to each sentence in the text to be divided;

[0027] The paragraph recognition model is trained based on target text samples and Internet text samples that carry paragraph end sentence tags and audio segment identification tags.

[0028] According to a topic paragraph division method provided by the present invention, based on the audio segment identifier, a paragraph recognition model is used to determine the score information corresponding to each sentence in the text to be divided, including:

[0029] Determining each recognition window based on the character recognition length of the paragraph recognition model;

[0030] Based on the audio segment identifiers in each recognition window, the paragraph recognition model is used to determine the initial score information corresponding to each sentence in the text to be divided;

[0031] Based on the initial score information corresponding to each sentence in the text to be divided in each recognition window, the score information corresponding to each sentence in the text to be divided is determined.

[0032] According to a topic paragraph division method provided by the present invention, the initial score information corresponding to each sentence in the text to be divided is determined by using the paragraph recognition model based on the audio segment identifiers in each recognition window, including:

[0033] Determine a vector representation of the audio segment identifier and a position vector representation of each punctuation point in the text to be divided;

[0034] In any window of the recognition windows, the vector representation of the audio segment identifier in the window, the position vector representation of each punctuation mark, and the text to be divided in the window are input into the encoding layer of the paragraph recognition model to obtain the vector representation of each character in the audio segment identifier in the window and the encoding feature of the window;

[0035] Inputting the encoding features in any window into the decoding layer of the paragraph recognition model to obtain the decoding information in any window;

[0036] The decoded information in any window is input to the output layer, and the initial score information corresponding to each sentence in the text to be divided in any window output by the output layer is obtained.

[0037] According to a topic paragraph division method provided by the present invention, the topic paragraph division is performed on the text to be divided based on the vector representation of each sentence in the text to be divided and the audio segment identifier, including:

[0038] Aggregating the sentences in the text to be divided based on the vector representation of each sentence in the text to be divided to determine a set of sentences contained in the text to be divided;

[0039] For any sentence set in the sentence sets, based on the audio segment identifier, using a paragraph recognition model, determining score information corresponding to each sentence in the sentence set; the score information is used to represent the score of each sentence in the sentence set being a paragraph ending sentence;

[0040] Determine each topic paragraph in the text to be divided based on the score information corresponding to each sentence in each sentence set;

[0041] The paragraph recognition model is trained based on target text samples and Internet text samples that carry paragraph end sentence tags and audio segment identification tags.

[0042] The present invention also provides a topic paragraph division device, comprising:

[0043] An acquisition module, configured to acquire a target text; the target text is obtained by transcribing a target audio, the target text carries an audio segment identifier in the target audio, the audio segment identifier includes a speaker identifier, an audience identifier, and a silence identifier;

[0044] A marking module, used for dividing the target text into sentences and marking punctuation marks based on the audio segment identifier to obtain a text to be divided;

[0045] The division module is used to determine the vector representation of each sentence in the text to be divided, and divide the text to be divided into topic paragraphs based on the vector representation of each sentence in the text to be divided and / or the audio segment identifier.

[0046] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-mentioned methods for dividing topic paragraphs are implemented.

[0047] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described methods for dividing topic paragraphs.

[0048] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned methods for dividing topic paragraphs.

[0049] The subject paragraph division method, device, electronic device and storage medium provided by the present invention first obtain a target text; the target text is obtained based on transcribing the target audio, and the target text carries the audio segment identifier in the target audio, and the audio segment identifier includes a speaker identifier, an audience identifier and a silence identifier; then based on the audio segment identifier, the target text is divided into sentences and punctuated to obtain a text to be divided; finally, the vector representation of each sentence in the text to be divided is determined, and based on the vector representation of each sentence in the text to be divided and / or the audio segment identifier, the text to be divided is divided into subject paragraphs. The audio segment identifier in the target audio is introduced, the text to be divided is obtained in combination with the audio segment identifier, and the text to be divided is divided into subject paragraphs, which can make full use of the information contained in the target audio, reduce the influence of the credibility of the target text on the division result, and make the division result of the subject paragraph more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on the drawings in the following description without paying any creative work.

[0051] Figure 1 It is a flow chart of the method for dividing the subject paragraphs provided by the present invention;

[0052] Figure 2 is a schematic diagram of an audio segment corresponding to an audio segment identifier in a target audio provided by the present invention;

[0053] Figure 3 It is a structural schematic diagram of the subject paragraph division device provided by the present invention;

[0054] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] As we all know, the development of speech recognition technology has been applied in various fields, including smart medical care, justice, education, and various C-end consumer products, such as voice recorders, translators and other intelligent terminal devices. Intelligent terminal devices can use speech recognition technology to perform real-time speech recognition of the speaker's speech content and transcribe it into text for presentation. However, how to divide the text obtained by speech transcription into topic paragraphs is a problem that needs to be solved.

[0057] At present, the method of dividing the topic paragraphs of the text obtained by speech transcription is usually to treat the topic paragraph division of the text obtained by speech transcription as a pure natural language processing task. Relying on the machine learning method, the characteristics of the topic vocabulary of some texts are obtained through the topic model (Latent Dirichlet Allocation, LDA), and all sentences are aggregated together through a certain entropy criterion to form related paragraphs. However, this method can only roughly divide the paragraphs and cannot guarantee the division effect.

[0058] Alternatively, we can use deep learning methods and LSTM models to treat existing sentences as sequence labeling tasks, perform binary classification on all sentences in the entire text, determine whether the sentence is the last sentence of a paragraph, and finally divide all text paragraphs. The advantage of this solution is that it can handle texts of any length, but it does not take into account the errors in the speech transcription process, which reduces the credibility of the text obtained by speech transcription and affects the segmentation results. At the same time, as the length of the text increases, the credibility of the topic paragraph segmentation will also decrease, resulting in poor segmentation results.

[0059] The above solutions, whether in machine learning or deep learning, are all solving an NLP problem. In actual use, especially for speech transcription, the text generally contains rich prior information, which helps to better realize the topic paragraph division of the text. For example, speaker characteristics are an extremely important biological feature, especially the pause state in the middle of the speech, on-site feedback and other information content. The pause state can include (1) the pause between a paragraph caused by thinking, taking a short break, drinking water in the middle of the speech, switching the main theme of the speech, etc. This kind of pause gap is long and the frequency of occurrence is low; (2) the pause between speakers, such as hesitation, breathing, stuttering, etc. This kind of pause gap is short and the frequency of occurrence is high. On-site feedback can include applause from the audience. Generally speaking, silent segments and audience segments are very important information and are important turning points of the topic in the text obtained by speech transcription.

[0060] To this end, an embodiment of the present invention provides a topic paragraph division method, which can achieve topic paragraph division of the text obtained by speech transcription by using the speaker segment, audience segment and silent segment contained in the above audio.

[0061] Figure 1 FIG. 1 is a flow chart of a method for dividing topic paragraphs provided in an embodiment of the present invention. Figure 1 As shown, the method includes:

[0062] S1, obtaining a target text; the target text is obtained by transcribing a target audio, the target text carries an audio segment identifier in the target audio, the audio segment identifier includes a speaker identifier, an audience identifier, and a silence identifier;

[0063] S2, based on the audio segment identifier, dividing the target text into sentences and marking punctuation marks to obtain a text to be divided;

[0064] S3, determining the vector representation of each sentence in the text to be divided, and dividing the text to be divided into topic paragraphs based on the vector representation of each sentence in the text to be divided and / or the audio segment identifier.

[0065] Specifically, the topic paragraph division method provided in the embodiment of the present invention has a topic paragraph division device as its executor, which can be configured in a server. The server can be a local server or a cloud server. The local server can specifically be a computer, a tablet computer, a smart phone, etc., which is not specifically limited in the embodiment of the present invention.

[0066] First, step S1 is performed to obtain a target text, which can be obtained by transcribing a target audio, and the target audio can be the audio of a speaker collected by a smart device in a target scene. The target scene can be a speech scene or other scenes, which are not specifically limited here. Taking a speech scene as an example, the speaker is the speaker.

[0067] In addition to the audio clips of the speaker, the target audio may also contain audio clips of the audience and silent clips, that is, the categories of audio clips in the target audio may include speaker clips, audience clips, and silent clips. Therefore, in an embodiment of the present invention, audio clip identifiers of different categories in the target audio may be used to mark the audio clips, and the audio clip identifiers are used to characterize the audio clips of each category in the target audio, that is, the audio clip identifiers correspond to the categories of the audio clips contained in the target audio. Therefore, the audio clip identifiers may include speaker identifiers, audience identifiers, and silent identifiers.

[0068] In the embodiment of the present invention, the audio segment identifier in the target audio may be carried in the target text to facilitate the subsequent division of the target text into topic paragraphs.

[0069] Then, step S2 is executed to divide the target text into sentences and mark punctuations according to the audio segment identifier to obtain the text to be divided. Here, since the target text contains at least one sentence, the target text can be divided into sentences and marked with punctuations. By dividing the target text into sentences, each sentence in the target text can be obtained, and by marking punctuations, punctuations can be added to each sentence obtained by division. In the process of sentence division and punctuation marking, the audio segment identifier can be introduced, and combined with the audio segment identifier, sentence division and punctuation marking of the target text can be achieved. The obtained text to be divided contains sentences carrying punctuation information, and the punctuation information can include commas, periods, exclamation marks, question marks, and no marks, etc.

[0070] In an embodiment of the present invention, the above process can be automatically implemented by a deep neural network model, for example, the audio segment identifier and the target text can be input into the deep neural network model to obtain the text to be divided output by the deep neural network model. The deep neural network model used can be a seq2seq deep neural network model, which can be a bidirectional LSTM model.

[0071] The deep neural network model may include an encoder layer and a decoder layer. The encoder layer is used to obtain the audio segment identifier and the target text, encode the audio segment identifier and the target text, and obtain the one-hot feature vector of the audio segment identifier and the vector representation of each word in the target text. The decoder is used to obtain the one-hot feature vector of the audio segment identifier and the vector representation of each word, decode the two, and obtain and output the text to be divided.

[0072] The deep neural network model can be trained using speech recognition text with punctuation information labels. That is, the speech recognition text can be input into the initial model of the deep neural network model to obtain the punctuation information output by the initial model, and the loss function value of the initial model is determined based on the punctuation information and punctuation information labels output by the initial model. The model parameters of the initial model are adjusted, and the above process is repeated until the loss function converges. The training is completed, and a deep neural network model with sentence segmentation and punctuation annotation functions is obtained.

[0073] Finally, step S3 is executed to divide the text to be divided into topic paragraphs. When dividing, the vector representation of each sentence in the text to be divided can be determined first, and then the text to be divided can be divided into topic paragraphs according to the vector representation of each sentence in the text to be divided. The text to be divided can also be divided into topic paragraphs directly through a neural network model. The vector representation of each sentence in the text to be divided can also be combined with the neural network model to achieve the topic paragraph division of the text to be divided. It can be understood that the topic paragraph division is to determine the topic paragraphs contained in the text to be divided, that is, to cluster each sentence in the text to be divided according to the topic content to obtain the topic paragraphs.

[0074] The vector representation of each sentence in the text to be divided can be the sentence vector corresponding to each sentence in the text to be divided, and each sentence in the text to be divided has a vector representation. The vector representation can be extracted by the sentence-bert model, which is not specifically limited here.

[0075] When the text to be divided is divided into topic paragraphs according to the vector representation of each sentence in the text to be divided, the similarity between any two adjacent sentences in the text to be divided can be calculated through the vector representation of the two adjacent sentences to determine the correlation between the two adjacent sentences, and then the two adjacent sentences can be classified according to the correlation. The above process is iterated to finally determine the paragraph to which each sentence in the text to be divided belongs, so as to achieve the topic paragraph division of the text to be divided.

[0076] Since similarity is introduced, the similarity can be used to characterize the content similarity between two adjacent sentences in the text to be divided. Therefore, after determining the paragraph to which each sentence in the text to be divided belongs, each paragraph in the text to be divided corresponds to a different topic.

[0077] When the text to be divided into topic paragraphs directly through the neural network model, the text to be divided can be input into the paragraph recognition model constructed based on the neural network model. Since the text to be divided contains audio segment identifiers, the paragraph recognition model can determine the sentences in the text to be divided that are paragraph ending sentences, thereby realizing the division of topic paragraphs.

[0078] When combining the vector representation of each sentence in the text to be divided with the neural network model to realize the topic paragraph division of the text to be divided, the vector representation of each sentence in the text to be divided can be determined first, and then the text to be divided is divided into topic paragraphs according to the vector representation of each sentence, and multiple sentence sets are determined. Then, each sentence set is input into the neural network model, and the paragraph ending sentence in each sentence set is determined by the neural network model, thereby realizing the topic paragraph division of the text to be divided.

[0079] The method for dividing topic paragraphs provided in the embodiment of the present invention first obtains the target text; the target text is obtained based on transcribing the target audio, and the target text carries the audio segment identifier in the target audio, and the audio segment identifier includes a speaker identifier, an audience identifier, and a silence identifier; then based on the audio segment identifier, the target text is divided into sentences and punctuated to obtain the text to be divided; finally, the vector representation of each sentence in the text to be divided is determined, and the text to be divided is divided into topic paragraphs based on the vector representation of each sentence in the text to be divided and / or the audio segment identifier. The audio segment identifier in the target audio is introduced, the text to be divided is obtained in combination with the audio segment identifier, and the text to be divided is divided into topic paragraphs, which can make full use of the information contained in the target audio, reduce the influence of the credibility of the target text on the division result, and make the division result of the topic paragraph more accurate.

[0080] On the basis of the above-mentioned embodiment, in the subject paragraph division method provided in the embodiment of the present invention, the target text is obtained based on the following method:

[0081] Extracting audio features of the target audio, and determining, based on the audio features, an audio segment corresponding to the audio segment identifier in the target audio;

[0082] The audio segment corresponding to the speaker identifier is converted into an audio transcription text, and the audio segment identifier is associated with the audio transcription text to obtain the target text.

[0083] Specifically, in an embodiment of the present invention, when determining the target text, the audio features of the target audio may be extracted first. The audio features may be used to identify relevant features of the target audio, such as filter bank features and Mel Frequency Cepstrum Coefficients (MFCC). It is understandable that the target audio may include multiple frames of audio, each of which corresponds to a filter bank feature and MFCC.

[0084] Then, the audio segment corresponding to the audio segment identifier in the target audio can be determined in combination with the audio features of the target audio. The process of determining the audio segment corresponding to the audio segment identifier in the target audio can be understood as the process of classifying each frame of audio in the target audio. This process can be implemented through a voice activation detection (VAD) model. The VAD model can distinguish between the speaker's voice and the audience's background noise in the target audio, and can also distinguish between speech and silence, which can avoid wasting bandwidth resources.

[0085] Here, the VAD model can be a neural network classification model with a two-layer BI-LSTM+Attention as the basic structure. The VAD model may include an input layer and a processing layer. The input layer may extract audio features in the target audio. The dimension of the audio features may be set as needed, for example, 64 dimensions. The processing layer may be combined with the audio features to determine each audio segment in the target audio. Each audio segment corresponds to an audio segment identifier.

[0086] Finally, the audio segment corresponding to the speaker identifier in the target audio is converted into an audio transcription text, and the audio segment identifier is associated with the audio transcription text to obtain the target text. In other words, the target text is the audio transcription text carrying the audio segment identifier.

[0087] In an embodiment of the present invention, when determining the target text, consideration is given to associating the audio segment identifier with the audio transcribed text to obtain the target text, so that the subsequent division of the topic paragraphs based on the audio segment identifier in the target text can ensure the accuracy of the division results.

[0088] On the basis of the above embodiment, in the subject paragraph division method provided in the embodiment of the present invention, the audio features include filter bank features and Mel cepstral coefficients of each frame of audio in the target audio;

[0089] Accordingly, determining the audio segment corresponding to the audio segment identifier in the target audio based on the audio feature includes:

[0090] Determine the attention features of each frame of audio based on the filter bank features and Mel cepstral coefficients of each frame of audio;

[0091] Performing an affine transformation on the attention features of each frame of audio to obtain a category corresponding to each frame of audio;

[0092] Based on the categories corresponding to the audio frames, an audio segment corresponding to the audio segment identifier is determined.

[0093] Specifically, since the target audio includes each frame of audio, its audio features may include filter bank features and Mel-cepstral coefficients of each frame of audio in the target audio.

[0094] Furthermore, when determining the audio segment corresponding to the audio segment identifier in the target audio, the attention features of each frame of audio can be first determined through the filter group features and Mel-cephalometric coefficients of each frame of audio; then, an affine transformation is performed on the attention features of each frame of audio to obtain the category corresponding to each frame of audio; finally, the audio segment corresponding to the audio segment identifier is determined according to the category corresponding to each frame of audio.

[0095] The above process can be implemented by a processing layer in a VAD model, and the processing layer may include a speech feature conversion layer and a decision layer. The speech feature conversion layer may jointly process the filter bank features and the Mel cepstral coefficients to obtain a joint feature. The joint processing method may be to splice the filter bank features with the Mel cepstral coefficients, and the obtained joint feature may be the feature obtained after splicing.

[0096] The speech feature transformation layer can also perform attention transformation on the joint features based on the Attention mechanism to obtain the attention features of each frame of audio. It can be understood that after passing through the speech feature transformation layer, each frame of audio has an attention feature fb voice .

[0097] The decision layer can be used to perform affine transformation on the attention features of each frame of audio, predict the category corresponding to each frame of audio in the target audio, that is, determine whether each frame of audio belongs to a silent segment, an audience segment or a speaker segment, and determine the confidence that each frame of audio belongs to each segment.

[0098] After determining the category corresponding to each frame of audio in the target audio, since each type of audio segment has a minimum speech interval, it is necessary to make a decision on the category corresponding to each frame of audio, that is, to determine whether the length of the audio segment composed of continuous frames of audio of the same category is greater than the minimum speech interval. If it is greater, the audio segment is considered to be an audio segment under this category. In order to ensure the smoothness and continuity between speech segments, the audio segments under each category can include speaker segments, audience segments, and silent segments. The corresponding audio segment identifiers can be respectively <s>、 as well as < / s>The minimum speech interval can be set as needed, for example, it can be set to t1ms.

[0099] Figure 2 A schematic diagram of an audio segment corresponding to an audio segment identifier in a target audio, such as Figure 2 As shown, in the target audio, 1 represents the speaker segment, 2 represents the listener segment, and 3 represents the silent segment.

[0100] In the embodiment of the present invention, the audio segment corresponding to the audio segment identifier is determined through the filter bank features and Mel-frequency cepstral coefficients of each frame of audio, combined with the attention mechanism, so that the information in the target audio can be fully utilized.

[0101] On the basis of the above-mentioned embodiment, the subject paragraph division method provided in the embodiment of the present invention divides the text to be divided into subject paragraphs based on the vector representation of each sentence in the text to be divided, including:

[0102] Aggregating the sentences in the text to be divided based on the vector representation of each sentence in the text to be divided to determine a set of sentences contained in the text to be divided;

[0103] Based on the sentence sets, the topic paragraphs contained in the text to be divided are determined.

[0104] Specifically, in an embodiment of the present invention, when dividing the text to be divided into topic paragraphs according to the vector representation of each sentence in the text to be divided, the sentences can be first aggregated according to the vector representation of each sentence to determine the topic paragraphs contained in the text to be divided.

[0105] The aggregation process can be achieved through a clustering algorithm or by calculating the similarity between sentences, which is not specifically limited here. For example, taking the calculation of the similarity between sentences as an example, we can start from the first sentence of the text to be divided, calculate the similarity between the first sentence and the second sentence based on the vector representation of the first sentence and the second sentence, and judge the correlation between the first sentence and the second sentence based on the similarity. Sentences with greater correlation can be in the same topic paragraph, and sentences with less correlation can be in different topic paragraphs.

[0106] Furthermore, the correlation between the third sentence and the second sentence can be determined. When the correlation between the first sentence and the second sentence is relatively large, the correlation between the third sentence and the first two sentences as a whole can be determined. This process is repeated until the correlation of all sentences is determined, that is, the set of sentences contained in the text to be divided is determined.

[0107] Then, the topic paragraphs contained in the text to be divided can be determined according to each sentence set, so as to realize the division of topic paragraphs. Here, each sentence set can be directly regarded as a paragraph theme, or the paragraph ending sentence in each sentence set can be further determined by combining the neural network model to realize the accurate division of topic paragraphs.

[0108] In the embodiment of the present invention, the aggregation method is used to determine the subject paragraphs contained in the text to be divided, which can greatly improve the division efficiency.

[0109] On the basis of the above-mentioned embodiment, the subject paragraph division method provided in the embodiment of the present invention, based on the vector representation of each sentence in the text to be divided, aggregates each sentence in the text to be divided, and determines each sentence set contained in the text to be divided, including:

[0110] For any sentence in the text to be divided, if there is a spare sentence set before any sentence, and the spare sentence set includes adjacent sentences of any sentence, then the vector representation of the spare sentence set is calculated, and the similarity between the vector representation of any sentence and the vector representation of the spare sentence set is calculated;

[0111] Based on the similarity, the any sentence is classified to obtain a classification result of the any sentence, and based on the classification result of each sentence in the text to be divided, a set of each sentence included in the text to be divided is determined.

[0112] Specifically, in an embodiment of the present invention, when aggregating each sentence in the text to be divided through the vector representation of each sentence in the text to be divided to determine a set of sentences contained in the text to be divided, for any sentence in the text to be divided, if there is a set of alternative sentences before any sentence, and the set of alternative sentences includes adjacent sentences of any sentence, then the vector representation of the set of alternative sentences is calculated, and the similarity between the vector representation of any sentence and the vector representation of the set of alternative sentences is calculated.

[0113] Here, any sentence can be any sentence except the first sentence in the text to be divided. An adjacent sentence refers to a sentence that precedes any sentence and is adjacent to any sentence. In addition to adjacent sentences, the standby sentence set also includes a standby sentence set whose vector representation has a similarity threshold with the vector representation of the adjacent sentence, which is not specifically limited in the embodiment of the present invention. Among them, the similarity threshold can be set as needed, for example, it can be set to a value such as 0.6. The vector representation of the standby sentence set refers to the average pooled vector of the vector representations of all sentences in the standby sentence set.

[0114] For example, when any of the sentences is the second sentence, the adjacent sentence is the first sentence, and the sentence set only contains the first sentence. The spare sentence set can be recorded as the first spare sentence set. At this time, the vector representation of the first spare sentence set can be calculated. Since the first spare sentence set only contains the first sentence, the vector representation of the first sentence can be directly used as the vector representation of the first spare sentence set. At this time, the similarity between the vector representation of the second sentence and the vector representation of the first spare sentence set can be calculated by the following formula:

[0115] score <v k ,v i > = cos(v k ,v i )

[0116] Among them, v k is the vector representation of the set of alternative statements that exist before any statement, v i is any sentence. 1≤i≤n, n is the total number of sentences contained in the text to be divided.

[0117] Then, the relationship between the similarity and the similarity threshold is determined. If the similarity is greater than or equal to the similarity threshold, it can be determined that the second statement has a greater correlation with the first standby statement set, and the second statement can be classified into the first standby statement set. If the similarity is less than the similarity threshold, it can be determined that the second statement has a less correlation with the first standby statement set, and the second statement can be classified into a new standby statement set, which can be recorded as the second standby statement set.

[0118] When any of the sentences is the third sentence, the adjacent sentence is the second sentence. Based on the above process, it can be determined whether the second sentence is in the first standby sentence set or the second standby sentence set. Furthermore, the similarity between the vector representation of the third sentence and the vector representation of the second standby sentence set can be calculated by the above formula, and based on the size relationship between the similarity and the similarity threshold, it can be determined whether the second sentence belongs to the second standby sentence set or a new standby sentence set. When any of the sentences is any other sentence in the text to be divided, the above process can be iteratively executed until all the sentences in the text to be divided are included in one standby sentence set, and the division of the topic paragraphs is completed. Each standby sentence set obtained is used as a sentence set in the text to be divided, and each sentence set corresponds to a topic paragraph.

[0119] In an embodiment of the present invention, by iteratively calculating similarities to divide the text into topic paragraphs, the division speed can be improved, and by calculating the similarity between the vector representation of any sentence and the vector representation of the previous set of alternative sentences, the accuracy of the division result can be guaranteed.

[0120] On the basis of the above embodiment, the subject paragraph division method provided in the embodiment of the present invention divides the text to be divided into subject paragraphs based on the audio segment identifier, including:

[0121] Based on the audio segment identifier, a paragraph recognition model is used to determine the score information corresponding to each sentence in the text to be divided; the score information is used to indicate the score of each sentence in the text to be divided as a paragraph ending sentence;

[0122] Determine each topic paragraph in the text to be divided based on the score information corresponding to each sentence in the text to be divided;

[0123] The paragraph recognition model is trained based on target text samples and Internet text samples that carry paragraph end sentence tags and audio segment identification tags.

[0124] Specifically, in an embodiment of the present invention, when the text to be divided is divided into topic paragraphs by audio segment identification, the paragraph recognition model can be used to determine the score information corresponding to each sentence in the text to be divided according to the audio segment identification. The paragraph recognition model can be constructed based on a neural network model. When the paragraph recognition model is used, the text to be divided can be input into the paragraph recognition model. The paragraph recognition model is combined with the audio segment identification carried by the text to be divided to output the score information corresponding to each sentence in the text to be divided. Each sentence in the text to be divided corresponds to a score information, and the score information can be used to characterize the score of the corresponding sentence as the end sentence of the paragraph.

[0125] The paragraph recognition model can be obtained by training the initial model with target text samples and Internet text samples that carry paragraph ending sentence labels and audio segment identification labels. The target text sample refers to the text obtained by transcribing the speech sample, and 10,000 hours of Chinese target text samples can be used here. The Internet text sample refers to the speech text existing on the Internet, and 40G of Internet text samples can be used here. The initial model is jointly trained with the target text samples and the Internet text samples. Since the data volume of the Internet text samples is large, the introduction of the Internet text samples can ensure the stability of the paragraph recognition model. The introduction of the target text samples can improve the reliability of the paragraph recognition model, so that it can be applied to the recognition of paragraph ending sentences in the text obtained by speech conversion.

[0126] It is understandable that the initial model in the embodiment of the present invention can be a model obtained by random initialization or a mac-BERT pre-trained language model, which is not specifically limited here. If the initial model is a mac-BERT pre-trained language model, the training process of the initial model can be understood as a fine-tuning process of the mac-BERT pre-trained language model.

[0127] Since the text style obtained by speech transcription is generally very different from the style of Internet text samples used in training the mac-BERT pre-trained language model, fine-tuning the mac-BERT pre-trained language model through target text samples and Internet text samples can achieve model style transfer, ensuring that the obtained paragraph recognition model will not suffer from a decrease in recognition effect due to differences in language style when processing downstream tasks such as identifying the end of a paragraph in the text obtained by speech transcription.

[0128] Finally, after determining the score information corresponding to each sentence in the text to be divided, it can be determined whether each sentence in the text to be divided is a paragraph ending sentence based on the score information. For example, the score information corresponding to each sentence in the text to be divided can be compared with the score threshold. If the score information corresponding to any sentence in the text to be divided is greater than or equal to the score threshold, the sentence in the text to be divided is determined to be a paragraph ending sentence. Otherwise, if the score information corresponding to any sentence in the text to be divided is less than the score threshold, the sentence in the text to be divided is determined to be a paragraph sentence.

[0129] Furthermore, after determining whether all sentences in the text to be divided are paragraph ending sentences, the sentences before each paragraph ending sentence to the previous paragraph ending sentence can be classified as a topic paragraph, thereby determining the topic paragraphs in the text to be divided.

[0130] In the embodiment of the present invention, when determining each topic paragraph in the text to be divided, a paragraph recognition model is introduced, which can simplify the division of the text to be divided. Moreover, the paragraph recognition model is obtained by training the target text sample and the Internet text sample as training samples, which can reduce the difficulty of collecting training samples, improve training efficiency, and ensure the reliability and accuracy of the paragraph recognition model.

[0131] On the basis of the above-mentioned embodiment, the subject paragraph division method provided in the embodiment of the present invention, based on the audio segment identifier, adopts the paragraph recognition model to determine the score information corresponding to each sentence in the text to be divided, including:

[0132] Determining each recognition window based on the character recognition length of the paragraph recognition model;

[0133] Based on the audio segment identifiers in each recognition window, the paragraph recognition model is used to determine the initial score information corresponding to each sentence in the text to be divided;

[0134] Based on the initial score information corresponding to each sentence in the text to be divided in each recognition window, the score information corresponding to each sentence in the text to be divided is determined.

[0135] Specifically, in the embodiment of the present invention, when applying the paragraph recognition model, the character recognition length of the paragraph recognition model can be determined first. Normally, the character recognition length of the paragraph recognition model can be 512 characters. Then, according to the character recognition length of the paragraph recognition model, each recognition window can be determined. Each recognition window can be obtained by the sliding action of a sliding window, and the width of each recognition window and the distance between two adjacent recognition windows can be set as required, for example, the width of each recognition window can be identical with the sliding window, which can be 512 characters, and the distance between two adjacent recognition windows can be identical with the sliding distance of the sliding window, which can be 256 characters.

[0136] Furthermore, the paragraph recognition model may be used according to the audio segment identifier in each recognition window to determine the initial score information corresponding to each sentence in the text to be divided in the recognition window;

[0137] Since two adjacent recognition windows have an overlapping area, the sentence in the overlapping area corresponds to two initial score information, and the score information corresponding to the sentence in the overlapping area can be the average of the two initial score information. The sentences in other areas except the overlapping area all correspond to one initial score information, and the initial score information can be used as the score information corresponding to the sentence.

[0138] In the embodiment of the present invention, when the paragraph recognition model is used, the character recognition length of the paragraph recognition model is taken into consideration, so that the phenomenon of missing score information can be avoided and the accuracy of the score information corresponding to each sentence can be ensured.

[0139] On the basis of the above-mentioned embodiment, the subject paragraph division method provided in the embodiment of the present invention, based on the audio segment identifier in each recognition window, adopting the paragraph recognition model, determining the initial score information corresponding to each sentence in the text to be divided, comprises:

[0140] Determine a vector representation of the audio segment identifier and a position vector representation of each punctuation point in the text to be divided;

[0141] In any window among the recognition windows, inputting the vector representation of the audio segment identifier in the any window, the position vector representation of each punctuation mark, and the text to be divided in the any window into the encoding layer of the paragraph recognition model to obtain the encoding features in the any window;

[0142] Inputting the encoding features in any window into the decoding layer of the paragraph recognition model to obtain the decoding information in any window;

[0143] The decoded information in any window is input to the output layer, and the initial score information corresponding to each sentence in the text to be divided in any window output by the output layer is obtained.

[0144] Specifically, in an embodiment of the present invention, when a paragraph recognition model is used in each recognition window, a vector representation of the audio segment identifier and a position vector representation of each punctuation mark in the text to be divided can be determined first. The vector representation of the audio segment identifier can be a one-hot vector obtained by converting the audio segment identifier, and the position vector representation of each punctuation mark in the text to be divided can also be a one-hot vector obtained by converting each punctuation mark in the text to be divided.

[0145] Then, in any window in each recognition window, the vector representation of the audio segment identifier in the window, the position vector representation of each punctuation mark in the text to be divided in the window, and the text to be divided in the window can be input into the encoding layer of the paragraph recognition model to obtain the encoding features in the window, which can include the encoding features corresponding to the vector representation of the audio segment identifier in the window, the encoding features corresponding to the position vector representation of each punctuation mark in the text to be divided in the window, and the encoding features corresponding to the vector representation of each sentence in the text to be divided in the window. The encoding layer can be a 12-layer transformer structure.

[0146] Then, the coding features in any window are input to the decoding layer of the paragraph recognition model to obtain the decoding information in any window, which may include the decoding information corresponding to each coding feature in the window. The decoding layer may be a bilstm+attention structure, and the tag bit of each sentence may be [SEP]. The decoding layer may realize the affine transformation of the vector representation of the sentence corresponding to each [SEP], so that each [SEP] has correlation.

[0147] Finally, the decoded information in any window is input to the output layer to obtain the initial score information corresponding to each sentence in the text to be divided in any window output by the output layer. The output layer can be implemented by a sigmoid activation function.

[0148] In the embodiment of the present invention, the specific structure of the paragraph recognition model and its operation flow in each recognition window are provided, which can ensure the feasibility of the paragraph recognition model.

[0149] On the basis of the above-mentioned embodiment, the subject paragraph division method provided in the embodiment of the present invention divides the text to be divided into subject paragraphs based on the vector representation of each sentence in the text to be divided and the audio segment identifier, including:

[0150] Aggregating the sentences in the text to be divided based on the vector representation of each sentence in the text to be divided to determine a set of sentences contained in the text to be divided;

[0151] For any sentence set in the sentence sets, based on the audio segment identifier, using a paragraph recognition model, determining score information corresponding to each sentence in the sentence set; the score information is used to represent the score of each sentence in the sentence set being a paragraph ending sentence;

[0152] Determine each topic paragraph in the text to be divided based on the score information corresponding to each sentence in each sentence set;

[0153] The paragraph recognition model is trained based on target text samples and Internet text samples that carry paragraph end sentence tags and audio segment identification tags.

[0154] Specifically, in an embodiment of the present invention, when the text to be divided is divided into topic paragraphs according to the vector representation of each sentence in the text to be divided and the audio segment identifier, the sentences in the text to be divided can be aggregated according to the vector representation of each sentence in the text to be divided to determine the set of sentences contained in the text to be divided. The detailed process can be referred to the above embodiment, which will not be repeated here.

[0155] Then, for any sentence set in each sentence set, the score information corresponding to each sentence in the any sentence set can be determined by using the audio segment identifier and the paragraph recognition model; the score information is used to characterize the score of each sentence in the any sentence set as a paragraph ending sentence. Based on the paragraph recognition model, the score information corresponding to each sentence in all sentence sets can be determined. Then, based on the score information corresponding to each sentence in each sentence set, the subject paragraphs in the text to be divided are determined. The detailed process can be found in the above embodiment, which will not be repeated here. It should be noted that the difference between the above embodiment and the embodiment of the present invention is that the object processed by the paragraph recognition model is different. In the above embodiment, the object processed by the paragraph recognition model is the text to be divided, while in the embodiment of the present invention, the object processed by the paragraph recognition model is each sentence set.

[0156] In the embodiment of the present invention, the aggregation method is used to determine the set of sentences contained in the text to be divided, which can greatly improve the efficiency of division. Furthermore, the introduction of the paragraph recognition model can determine whether each sentence in each sentence set is a paragraph ending sentence, which can achieve more accurate topic paragraph division.

[0157] like Figure 3 As shown, based on the above embodiment, an embodiment of the present invention provides a topic paragraph division device, including:

[0158] An acquisition module 31 is used to acquire a target text; the target text is obtained by transcribing a target audio, and the target text carries an audio segment identifier in the target audio, wherein the audio segment identifier includes a speaker identifier, an audience identifier, and a silence identifier;

[0159] A marking module 32, configured to divide the target text into sentences and mark punctuations based on the audio segment identifier to obtain a text to be divided;

[0160] The division module 33 is used to determine the vector representation of each sentence in the text to be divided, and divide the text to be divided into topic paragraphs based on the vector representation of each sentence in the text to be divided and / or the audio segment identifier.

[0161] On the basis of the above-mentioned embodiment, in the subject paragraph division device provided in the embodiment of the present invention, the acquisition module is specifically used for:

[0162] Extracting audio features of the target audio, and determining, based on the audio features, an audio segment corresponding to the audio segment identifier in the target audio;

[0163] The audio segment corresponding to the speaker identifier is converted into an audio transcription text, and the audio segment identifier is associated with the audio transcription text to obtain the target text.

[0164] On the basis of the above embodiment, in the subject paragraph division device provided in the embodiment of the present invention, the audio features include filter bank features and Mel cepstral coefficients of each frame of audio in the target audio;

[0165] Accordingly, the acquisition module is also used for:

[0166] Determine the attention features of each frame of audio based on the filter bank features and Mel cepstral coefficients of each frame of audio;

[0167] Performing an affine transformation on the attention features of each frame of audio to obtain a category corresponding to each frame of audio;

[0168] Based on the categories corresponding to the audio frames, an audio segment corresponding to the audio segment identifier is determined.

[0169] On the basis of the above-mentioned embodiment, in the subject paragraph division device provided in the embodiment of the present invention, the division module is specifically used for:

[0170] Aggregating the sentences in the text to be divided based on the vector representation of each sentence in the text to be divided to determine a set of sentences contained in the text to be divided;

[0171] Based on the sentence sets, the topic paragraphs contained in the text to be divided are determined.

[0172] On the basis of the above-mentioned embodiment, in the subject paragraph division device provided in the embodiment of the present invention, the division module is specifically used for:

[0173] For any sentence in the text to be divided, if there is a spare sentence set before any sentence, and the spare sentence set includes adjacent sentences of any sentence, then the vector representation of the spare sentence set is calculated, and the similarity between the vector representation of any sentence and the vector representation of the spare sentence set is calculated;

[0174] Based on the similarity, the any sentence is classified to obtain a classification result of the any sentence, and based on the classification result of each sentence in the text to be divided, a set of each sentence included in the text to be divided is determined.

[0175] On the basis of the above-mentioned embodiment, in the subject paragraph division device provided in the embodiment of the present invention, the division module is specifically used for:

[0176] Based on the audio segment identifier, a paragraph recognition model is used to determine the score information corresponding to each sentence in the text to be divided; the score information is used to indicate the score of each sentence in the text to be divided as a paragraph ending sentence;

[0177] Determine each topic paragraph in the text to be divided based on the score information corresponding to each sentence in the text to be divided;

[0178] The paragraph recognition model is trained based on target text samples and Internet text samples that carry paragraph end sentence tags and audio segment identification tags.

[0179] On the basis of the above-mentioned embodiment, in the subject paragraph division device provided in the embodiment of the present invention, the division module is specifically used for:

[0180] Determining each recognition window based on the character recognition length of the paragraph recognition model;

[0181] Based on the audio segment identifiers in each recognition window, the paragraph recognition model is used to determine the initial score information corresponding to each sentence in the text to be divided;

[0182] Based on the initial score information corresponding to each sentence in the text to be divided in each recognition window, the score information corresponding to each sentence in the text to be divided is determined.

[0183] On the basis of the above-mentioned embodiment, in the subject paragraph division device provided in the embodiment of the present invention, the division module is specifically used for:

[0184] Determine a vector representation of the audio segment identifier and a position vector representation of each punctuation point in the text to be divided;

[0185] In any window among the recognition windows, inputting the vector representation of the audio segment identifier in the any window, the position vector representation of each punctuation mark, and the text to be divided in the any window into the encoding layer of the paragraph recognition model to obtain the encoding features in the any window;

[0186] Inputting the encoding features in any window into the decoding layer of the paragraph recognition model to obtain the decoding information in any window;

[0187] The decoded information in any window is input to the output layer, and the initial score information corresponding to each sentence in the text to be divided in any window output by the output layer is obtained.

[0188] On the basis of the above-mentioned embodiment, in the subject paragraph division device provided in the embodiment of the present invention, the division module is further specifically used for:

[0189] Aggregating the sentences in the text to be divided based on the vector representation of each sentence in the text to be divided to determine a set of sentences contained in the text to be divided;

[0190] For any sentence set in the sentence sets, based on the audio segment identifier, using a paragraph recognition model, determining score information corresponding to each sentence in the sentence set; the score information is used to represent the score of each sentence in the sentence set being a paragraph ending sentence;

[0191] Determine each topic paragraph in the text to be divided based on the score information corresponding to each sentence in each sentence set;

[0192] The paragraph recognition model is trained based on target text samples and Internet text samples that carry paragraph end sentence tags and audio segment identification tags.

[0193] Specifically, the functions of each module in the topic paragraph division device provided in the embodiment of the present invention correspond one-to-one to the operation flow of each step in the above method embodiment, and the effects achieved are also consistent. Please refer to the above embodiment for details, and no further details will be given in the embodiment of the present invention.

[0194] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the topic paragraph division method provided in the above embodiments, the method comprising: obtaining a target text; the target text is obtained by transcribing the target audio, the target text carries an audio segment identifier in the target audio, the audio segment identifier includes a speaker identifier, an audience identifier and a silence identifier; based on the audio segment identifier, the target text is divided into sentences and punctuated to obtain a text to be divided; the vector representation of each sentence in the text to be divided is determined, and the text to be divided is divided into topic paragraphs based on the vector representation of each sentence in the text to be divided and / or the audio segment identifier.

[0195] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0196] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the topic paragraph division method provided in the above-mentioned embodiments, and the method includes: obtaining a target text; the target text is obtained based on transcribing a target audio, and the target text carries an audio segment identifier in the target audio, and the audio segment identifier includes a speaker identifier, an audience identifier, and a silence identifier; based on the audio segment identifier, the target text is divided into sentences and punctuated to obtain a text to be divided; determine the vector representation of each sentence in the text to be divided, and divide the text to be divided into topic paragraphs based on the vector representation of each sentence in the text to be divided and / or the audio segment identifier.

[0197] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the topic paragraph division method provided in the above-mentioned embodiments, the method comprising: obtaining a target text; the target text is obtained based on transcribing a target audio, the target text carries an audio segment identifier in the target audio, the audio segment identifier includes a speaker identifier, an audience identifier and a silence identifier; based on the audio segment identifier, the target text is divided into sentences and punctuated to obtain a text to be divided; determining a vector representation of each sentence in the text to be divided, and based on the vector representation of each sentence in the text to be divided and / or the audio segment identifier, dividing the text to be divided into topic paragraphs.

[0198] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0199] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0200] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for dividing topic paragraphs, characterized in that: include: Obtain a target text; the target text is obtained by transcribing the target audio, the target text carries an audio segment identifier in the target audio, the audio segment identifier includes a speaker identifier, an audience identifier, and a silence identifier; Based on the audio segment identifier, the target text is divided into sentences and punctuation marks are marked to obtain a text to be divided; Determine a vector representation of each sentence in the text to be divided, and divide the text to be divided into topic paragraphs based on the vector representation of each sentence in the text to be divided and / or the audio segment identifier; The target text is obtained based on the following method: Extracting audio features of the target audio, and determining an audio segment corresponding to the audio segment identifier in the target audio based on the audio features; the audio features include filter bank features and Mel cepstral coefficients of each frame of audio in the target audio; The audio segment corresponding to the speaker identifier is converted into an audio transcription text, and the audio segment identifier is associated with the audio transcription text to obtain the target text.

2. The topic paragraph division method according to claim 1, characterized in that: The step of determining, based on the audio feature, an audio segment corresponding to the audio segment identifier in the target audio includes: Based on the filter bank features and Mel cepstral coefficients of each frame of audio, determining the attention features of each frame of audio; the attention features are obtained by performing attention transformation on the joint features based on the Attention mechanism, and the joint features are obtained by concatenating the filter bank features and the Mel cepstral coefficients; Performing an affine transformation on the attention features of each frame of audio to obtain a category corresponding to each frame of audio; Based on the categories corresponding to the audio frames, an audio segment corresponding to the audio segment identifier is determined.

3. The method for dividing topic paragraphs according to any one of claims 1-2, characterized in that: Based on the vector representation of each sentence in the text to be divided, the text to be divided is divided into topic paragraphs, including: Aggregating the sentences in the text to be divided based on the vector representation of each sentence in the text to be divided to determine a set of sentences contained in the text to be divided; Based on the sentence sets, the topic paragraphs contained in the text to be divided are determined.

4. The topic paragraph division method according to claim 3 is characterized in that: The step of aggregating the sentences in the text to be divided based on the vector representation of the sentences in the text to be divided to determine the set of sentences contained in the text to be divided includes: For any sentence in the text to be divided, if there is a spare sentence set before any sentence, and the spare sentence set includes adjacent sentences of any sentence, then the vector representation of the spare sentence set is calculated, and the similarity between the vector representation of any sentence and the vector representation of the spare sentence set is calculated; Based on the similarity, the any sentence is classified to obtain a classification result of the any sentence, and based on the classification result of each sentence in the text to be divided, a set of each sentence included in the text to be divided is determined.

5. The method for dividing topic paragraphs according to any one of claims 1-2, characterized in that: Based on the audio segment identifier, the text to be divided is divided into topic paragraphs, including: Based on the audio segment identifier, a paragraph recognition model is used to determine the score information corresponding to each sentence in the text to be divided; the score information is used to indicate the score of each sentence in the text to be divided as a paragraph ending sentence; Determine each topic paragraph in the text to be divided based on the score information corresponding to each sentence in the text to be divided; The paragraph recognition model is trained based on target text samples and Internet text samples that carry paragraph end sentence tags and audio segment identification tags.

6. The method for dividing topic paragraphs according to claim 5, characterized in that: The step of determining score information corresponding to each sentence in the text to be divided by using a paragraph recognition model based on the audio segment identifier includes: Determining each recognition window based on the character recognition length of the paragraph recognition model; Based on the audio segment identifiers in each recognition window, the paragraph recognition model is used to determine the initial score information corresponding to each sentence in the text to be divided; Based on the initial score information corresponding to each sentence in the text to be divided in each recognition window, the score information corresponding to each sentence in the text to be divided is determined.

7. The method for dividing topic paragraphs according to claim 6, characterized in that: The step of determining the initial score information corresponding to each sentence in the text to be divided by using the paragraph recognition model based on the audio segment identifiers in each recognition window includes: Determine a vector representation of the audio segment identifier and a position vector representation of each punctuation point in the text to be divided; In any window among the recognition windows, inputting the vector representation of the audio segment identifier in the any window, the position vector representation of each punctuation mark, and the text to be divided in the any window into the encoding layer of the paragraph recognition model to obtain the encoding features in the any window; Inputting the encoding features in any window into the decoding layer of the paragraph recognition model to obtain the decoding information in any window; The decoded information in any window is input to the output layer, and the initial score information corresponding to each sentence in the text to be divided in any window output by the output layer is obtained.

8. The method for dividing topic paragraphs according to any one of claims 1-2, characterized in that: Based on the vector representation of each sentence in the text to be divided and the audio segment identifier, the text to be divided is divided into topic paragraphs, including: Aggregating the sentences in the text to be divided based on the vector representation of each sentence in the text to be divided to determine a set of sentences contained in the text to be divided; For any sentence set in the sentence sets, based on the audio segment identifier, using a paragraph recognition model, determining score information corresponding to each sentence in the sentence set; the score information is used to represent the score of each sentence in the sentence set being a paragraph ending sentence; Determine each topic paragraph in the text to be divided based on the score information corresponding to each sentence in each sentence set; The paragraph recognition model is trained based on target text samples and Internet text samples that carry paragraph end sentence tags and audio segment identification tags.

9. A device for dividing topic paragraphs, characterized in that: include: An acquisition module, configured to acquire a target text; the target text is obtained by transcribing a target audio, the target text carries an audio segment identifier in the target audio, the audio segment identifier includes a speaker identifier, an audience identifier, and a silence identifier; A marking module, used for dividing the target text into sentences and marking punctuation marks based on the audio segment identifier to obtain a text to be divided; A division module, used for determining a vector representation of each sentence in the text to be divided, and dividing the text to be divided into topic paragraphs based on the vector representation of each sentence in the text to be divided and / or the audio segment identifier; The target text is obtained based on the following method: Extracting audio features of the target audio, and determining an audio segment corresponding to the audio segment identifier in the target audio based on the audio features; the audio features include filter bank features and Mel cepstral coefficients of each frame of audio in the target audio; The audio segment corresponding to the speaker identifier is converted into an audio transcription text, and the audio segment identifier is associated with the audio transcription text to obtain the target text.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the topic paragraph division method according to any one of claims 1 to 8 are implemented.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the subject paragraph division method according to any one of claims 1 to 8 are implemented.

Images