Text alignment methods, devices, computer equipment, and storage media

Abstract

This application discloses a text alignment method applied in the field of text processing technology to improve the accuracy of text alignment. The method provided includes: acquiring the text to be aligned; segmenting the text into sentences to generate reference sentence vectors for the sentences in the text to be aligned; acquiring audio data corresponding to the text to be aligned; generating transcribed text of the audio data; generating paragraph vectors for each basic paragraph in the transcribed text; determining the order of each basic paragraph in the transcribed text based on the first paragraph information; calculating the vector similarity between the paragraph vector of each basic paragraph and the reference sentence vector in that order to obtain a similarity set; comparing the similarity value corresponding to each basic paragraph with a preset similarity threshold to obtain a comparison result; and determining the basic paragraph corresponding to the text to be aligned as the target text based on the comparison result to obtain the target aligned text.

Description

Technical Field

[0001] This application relates to the field of text processing technology, and in particular to a text alignment method, apparatus, computer device, and storage medium. Background Technology

[0002] In plain text alignment scenarios, traditional alignment methods based on statistical rules extract feature information based on statistical information such as keywords or word data, calculate the similarity between the original sentence and the target sentence based on the extracted feature information, and achieve text alignment. This method requires manually formulated rules, lacks universality, and ultimately results in poor text alignment. Summary of the Invention

[0003] This application provides a text alignment method, apparatus, computer device, and storage medium to improve the accuracy of text alignment.

[0004] A text alignment method, comprising:

[0005] Obtain the text to be aligned, segment the text to be aligned into sentences to obtain the text sentences to be aligned, and generate a reference sentence vector for the text sentences to be aligned;

[0006] Acquire audio data corresponding to the text to be aligned, perform speech recognition on the audio data, and generate transcribed text of the audio data, wherein the transcribed text includes at least one basic paragraph;

[0007] Obtain the paragraph information of the transcribed text as the first paragraph information, and generate a paragraph vector for each basic paragraph in the transcribed text based on the first paragraph information;

[0008] Based on the first paragraph information, the order of each basic paragraph in the transcribed text is determined. According to the order, the vector similarity between the paragraph vector of each basic paragraph and the vector of the reference sentence is calculated in turn to obtain a similarity set, wherein the similarity set contains the similarity value corresponding to each basic paragraph.

[0009] For each of the basic paragraphs, the similarity value corresponding to the basic paragraph is compared with a preset similarity threshold to obtain a comparison result. Based on the comparison result, the basic paragraph corresponding to the text to be aligned is determined as the target text.

[0010] Following the aforementioned sequence, each target text is summarized sequentially to obtain the target aligned text of the text to be aligned.

[0011] A text alignment device, comprising:

[0012] The text to be aligned processing module is used to acquire the text to be aligned, segment the text to be aligned into sentences, obtain the text sentences to be aligned, and generate a reference sentence vector for the text sentences to be aligned.

[0013] A text transcription module is used to acquire audio data corresponding to the text to be aligned, perform speech recognition on the audio data, and generate transcribed text of the audio data, wherein the transcribed text includes at least one basic paragraph;

[0014] The paragraph vector generation module is used to obtain the paragraph information of the transcribed text as the first paragraph information, and generate the paragraph vector of each basic paragraph in the transcribed text based on the first paragraph information.

[0015] The similarity calculation module is used to determine the order of each basic paragraph in the transcribed text based on the first paragraph information, and calculate the vector similarity between the paragraph vector of each basic paragraph and the reference sentence vector in the order to obtain a similarity set, wherein the similarity set contains the similarity value corresponding to each basic paragraph.

[0016] The target text determination module is used to compare the similarity value corresponding to each base paragraph with a preset similarity threshold to obtain a comparison result, and determine the base paragraph corresponding to the text to be aligned as the target text based on the comparison result;

[0017] The text alignment module is used to summarize each of the target texts in the order described above to obtain the target aligned text of the text to be aligned.

[0018] A computer device includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the above-described text alignment method.

[0019] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described text alignment method.

[0020] The text alignment method, apparatus, computer device, and storage medium provided in this application acquire the text to be aligned, generate sentence vectors for the text to be aligned, acquire audio data corresponding to the text to be aligned, generate transcribed text of the audio data based on speech recognition technology, generate paragraph vectors for each basic paragraph in the transcribed text, calculate the similarity between the sentence vectors and the paragraph vectors of each basic paragraph to obtain a similarity value, compare the similarity value with a preset similarity threshold to determine the alignment relationship between the current basic paragraph and the text to be aligned corresponding to the current sentence vector, if the similarity value is less than the preset similarity threshold, then the transcribed text of the current basic paragraph is used as the alignment text of the text to be aligned, so that the text to be aligned can obtain the paragraph information and sentence information of the transcribed text, thereby improving the accuracy of aligning the text to be aligned. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of an application environment for a text alignment method according to an embodiment of this application;

[0023] Figure 2 This is a flowchart of a text alignment method in one embodiment of this application;

[0024] Figure 3 This is a schematic diagram of the structure of a text alignment device in one embodiment of this application;

[0025] Figure 4 This is a schematic diagram of a computer device according to one embodiment of this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] The text alignment method provided in this application embodiment can be applied to, for example, Figure 1In this application environment, computer devices communicate with servers via a network. These computer devices can be, but are not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. Servers can be implemented using standalone servers or server clusters consisting of multiple servers.

[0028] System framework 100 may include terminal devices, a network, and a server. The network serves as the medium for providing a communication link between the terminal devices and the server. The network may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0029] Users can use terminal devices to interact with the server over the network to receive or send messages, etc.

[0030] Terminal devices 101, 102, and 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Eperts Group Audio Layer III), MP4 players (Moving Picture Eperts Group Audio Layer IV), laptops, and desktop computers, etc.

[0031] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on terminal devices 101, 102, and 103.

[0032] It should be noted that the text alignment method provided in this application embodiment is executed by the server, and correspondingly, the text alignment device is set in the server.

[0033] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown in this embodiment is merely illustrative. Depending on the implementation requirements, there can be any number of terminal devices, networks, and servers. The terminal devices in this embodiment can specifically correspond to application systems in actual production.

[0034] In one embodiment, such as Figure 2 As shown, a text alignment method is provided, which is applied to... Figure 1 Taking the server in the example, the following steps are included:

[0035] S10: Obtain the text to be aligned, segment the text to be aligned into sentences, obtain the sentences to be aligned, and generate a reference sentence vector for the sentences to be aligned.

[0036] Specifically, the text to be aligned refers to the target text to be aligned. The text to be aligned is segmented into sentences to obtain multiple sentences to be aligned. A reference sentence vector is generated for each sentence to be aligned based on a pre-trained model.

[0037] Among them, the open-source BERT-based model was used as the pre-trained model.

[0038] S20, acquire audio data corresponding to the text to be aligned, perform speech recognition on the audio data, and generate transcribed text of the audio data, wherein the transcribed text includes at least one basic paragraph.

[0039] Specifically, audio data refers to the audio file corresponding to the text to be aligned, which can be conference audio data or telephone recording data. The text to be aligned represents the speech of the audio data in text form.

[0040] Speech recognition is performed on the audio data to generate a transcribed text, which includes at least one base paragraph. A base paragraph consists of multiple text sentences to be aligned.

[0041] As an alternative implementation, the text to be aligned in each basic paragraph of the transcribed text is segmented into sentences based on punctuation marks. Sentence segmentation based on punctuation marks is an existing technology and will not be elaborated here.

[0042] S30, Obtain the paragraph information of the transcribed text as the first paragraph information, and generate the paragraph vector of each basic paragraph in the transcribed text based on the first paragraph information.

[0043] Specifically, paragraph information refers to the number of paragraphs and their order in the transcribed text. Using this paragraph information as the first paragraph information, a paragraph vector is generated for each basic paragraph. The method for generating paragraph vectors is as follows: Sentences are segmented based on each basic paragraph, resulting in multiple segmented sentences; the sentence vector for each segmented sentence is calculated separately; the sentence vectors of all segmented sentences are summed, and the sums are averaged to obtain the paragraph vector for that basic paragraph.

[0044] S40. Based on the information of the first paragraph, determine the order of each basic paragraph in the transcribed text. Calculate the vector similarity between the paragraph vector of each basic paragraph and the vector of the reference sentence in the order of the paragraphs to obtain a similarity set. The similarity set contains the similarity value corresponding to each basic paragraph.

[0045] S50, for each basic paragraph, compare the similarity value corresponding to the basic paragraph with the preset similarity threshold to obtain the comparison result, and determine the basic paragraph corresponding to the text to be aligned as the target text based on the comparison result.

[0046] In S40 to S50, based on the information from the first paragraph, the reference sentence vector is compared with the paragraph vector of the first basic paragraph in sequence.

[0047] For example, A. If the reference sentence vector being calculated corresponds to the first sentence of the text to be aligned, then the vector similarity between the reference sentence vector of the first sentence and the paragraph vector of the first basic paragraph is calculated to obtain a similarity value.

[0048] B. Calculate the vector similarity between the vector of the first sentence to be aligned (formed by the reference sentence vector of the first sentence to be aligned and the reference sentence vector of the next sentence to be aligned) and the paragraph vector of the first basic paragraph to obtain the next similarity value.

[0049] Repeat the above A and B steps to form a similarity set of all similarity values, record the maximum similarity value, and use the maximum similarity value as the preset similarity threshold.

[0050] If the similarity value obtained in the next execution is less than the maximum similarity value, then the text sentence to be aligned corresponding to the reference sentence vector corresponding to the maximum similarity value will be marked with a paragraph end mark, and the text sentence to be aligned corresponding to the reference sentence vector corresponding to the first execution will be marked with a paragraph start mark. The paragraph start mark and paragraph end mark will be used as a target text to be aligned. The order of the target text should be the same as the order of the corresponding basic paragraphs.

[0051] Select the second base paragraph of the transcribed text and repeat the above steps until the paragraph end marker of the second paragraph of the text to be aligned is determined, thus obtaining the target text corresponding to the second base paragraph.

[0052] The alignment of the remaining parts follows the same principle as described above, and will not be repeated here.

[0053] S60, in chronological order, summarize each target text to obtain the target alignment text of the text to be aligned.

[0054] Specifically, according to the order of the basic paragraphs and the correspondence between the target text and the basic paragraphs, the order of the target text is determined and summarized to obtain the target aligned text of the text to be aligned. The target aligned text, relative to the text to be aligned, includes segmentation information, that is, the paragraph information in the transcribed text is transferred to the text to be aligned, thus forming the target aligned text.

[0055] The text alignment method provided in this application involves acquiring the text to be aligned, generating sentence vectors for the text to be aligned, acquiring the corresponding audio data, generating transcribed text of the audio data using speech recognition technology, generating paragraph vectors for each basic paragraph in the transcribed text, calculating the similarity between the sentence vectors and the paragraph vectors of each basic paragraph, and comparing the similarity value with a preset similarity threshold to determine the alignment relationship between the current basic paragraph and the text to be aligned corresponding to the current sentence vector. If the similarity value is less than the preset similarity threshold, the transcribed text of the current basic paragraph is used as the alignment text for the text to be aligned, enabling the text to be aligned to obtain the paragraph and sentence information of the transcribed text, thereby improving the accuracy of text alignment.

[0056] Furthermore, as an optional implementation, in S10, the process of obtaining the text to be aligned, segmenting the text to be aligned into sentences, obtaining the sentences to be aligned, and generating a reference sentence vector for the sentences to be aligned includes:

[0057] S101, Segment the text to be aligned to obtain a set of sentences to be aligned, the set of sentences to be aligned includes at least one sentence to be aligned.

[0058] S102, the sentences to be aligned in the set of sentences to be aligned are segmented by the word segmenter to obtain segmented sentences.

[0059] S103 generates reference sentence vectors for segmented sentences based on a pre-trained model.

[0060] Specifically, the text to be aligned is segmented into sentences, and these sentences are then divided into words. At the same time, a sentence start mark is added at the beginning of each sentence and a sentence end mark is added at the end to distinguish between different sentences.

[0061] To facilitate parallel processing of the text sentences to be aligned, the maximum length of each text sentence to be aligned is set to L.

[0062] Here, L is generally set based on the reference sentence vector of the text sentence to be aligned. For example, if the length of the reference sentence vector of the text sentence to be aligned is 512, then the length L is set to 512.

[0063] When the length of the text sentence to be aligned is less than L, it is padded with 0 at the end; when the length of the text sentence to be aligned is greater than L, only the first L words are extracted.

[0064] The segmented sentences are input into the pre-trained model to generate reference sentence vectors for the segmented sentences.

[0065] In this embodiment, by performing sentence segmentation and word segmentation on the text to be aligned, the length of each sentence in the text to be aligned can be uniformly set, thereby enabling subsequent parallel processing and improving the efficiency of aligning the text to be aligned.

[0066] Furthermore, as an optional implementation, in S20, acquiring audio data corresponding to the text to be aligned, performing speech recognition on the audio data, and generating transcribed text of the audio data includes:

[0067] S201, based on the duration of speech pauses, perform endpoint detection on the audio data to determine the start and end frame markers for each audio segment.

[0068] S202, take a set of start frame markers and end frame markers as a basic segment, determine the first segment information, and divide the audio data according to the first segment information to obtain multiple sub-audio segments.

[0069] S203, perform speech recognition on each audio segment to generate transcribed text containing information from the first segment.

[0070] Specifically, a speech pause threshold is set. If a certain segment of the audio data has a speech pause greater than the speech pause threshold, then each sub-audio segment is divided accordingly.

[0071] The threshold for pauses in speech is set according to the actual situation and is not limited here.

[0072] Each segmented audio is treated as a basic paragraph. Speech recognition is performed on each segment to obtain the corresponding text, which serves as a basic paragraph of the transcribed text. Multiple basic paragraphs are combined to form the transcribed text.

[0073] In this embodiment, speech recognition is performed on the audio data to generate transcribed text with first paragraph information. The paragraph information identified from the audio data is used to segment the text to be aligned, avoiding text alignment through plain text, thereby improving the accuracy of text alignment.

[0074] Furthermore, as an optional implementation, in S30, obtaining paragraph information of the transcribed text as first paragraph information, and generating a paragraph vector for each basic paragraph in the transcribed text based on the first paragraph information includes:

[0075] S301. Based on the information from the first paragraph, the transcribed text of each basic paragraph is segmented into sentences to obtain a set of transcribed sentences for each basic paragraph.

[0076] S302, based on a pre-trained model, generates the transcribed sentence vector for each transcribed sentence in each base paragraph.

[0077] S303, sum and average the transcribed sentence vectors of each basic paragraph to obtain the paragraph vector of each basic paragraph.

[0078] Specifically, the transcribed text is segmented into sentences using a basic paragraph as the unit, resulting in multiple sets of transcribed sentences;

[0079] Based on the pre-trained model, the transcribed sentence vector of each transcribed sentence in each base paragraph is calculated, and the paragraph vector of each base paragraph is obtained by summing and averaging.

[0080] The formula for calculating paragraph vectors is as follows:

[0081]

[0082] in, The paragraph vector that identifies the i-th basic paragraph in the transcribed text. This represents the transcribed sentence vector of the k-th transcribed sentence in the i-th base paragraph of the transcribed text, where k ranges from 1 to T. i T i This represents the number of transcribed sentences in the i-th base paragraph.

[0083] The paragraph vectors corresponding to the entire transcribed text are represented by a set of paragraph vectors, as follows:

[0084]

[0085] Where EMb represents the set of paragraph vectors, and M represents the number of basic paragraphs in the transcribed text. This represents the paragraph vector of the first basic paragraph. This represents the paragraph vector of the second basic paragraph.

[0086] In this embodiment, by calculating a paragraph vector for each basic paragraph of the transcribed text, it is beneficial to segment the text to be aligned according to the paragraph vector, thereby improving the efficiency of segmenting the aligned text.

[0087] Furthermore, as an optional implementation, after S30, the following is included:

[0088] S31 generates speech feature vectors for each basic segment of the audio data using Mel-frequency cepstral coefficients.

[0089] S32, for each basic segment, the speech feature vector and the segment vector are fused to obtain the fused feature vector.

[0090] Mel-scale frequency cepstral coefficients (MFCCs) are cepstral parameters extracted in the Mel-scale frequency domain. The Mel scale can be used to describe the nonlinear characteristics of human ear frequencies. The conversion formula between the Mel scale and frequency is shown below, where f represents the frequency of the signal in Hz, and lg() is used to calculate the logarithm.

[0091] The specific formula is as follows:

[0092]

[0093] The Mel-frequency cepstral frequency is calculated using the above formula, and then the speech feature vector corresponding to each basic segment is calculated. The corresponding speech feature vector is fused with the segment vector to obtain the fused feature vector of each basic segment.

[0094] The formula for generating the fused feature vector is shown below:

[0095]

[0096] in, This represents the fusion feature vector of the i-th basic paragraph. Let s represent the paragraph vector of the i-th basic paragraph. i This represents the speech feature vector of the i-th basic segment. ∝ and β are engineering parameters, taking values ​​greater than 0 integers. In this embodiment, they can be set according to the actual scenario, and no specific limitation is made here.

[0097] As an optional implementation, the fused feature vector is used as a parameter for calculating vector similarity.

[0098] In this embodiment, the speech feature vector and the paragraph vector are fused to generate a fused feature vector, which can better preserve semantic features and ensure the accuracy of text alignment in subsequent text alignment.

[0099] Furthermore, as an optional implementation, in S40, based on the first paragraph information, the order of each basic paragraph in the transcribed text is determined, and the vector similarity between the paragraph vector of each basic paragraph and the reference sentence vector is calculated sequentially according to the order, resulting in a similarity set including:

[0100] S401, Based on the order of sentences to be aligned in the text to be aligned, set the current sentence to be aligned and record the current sentence order number.

[0101] S402, calculate the vector similarity between the reference sentence vector corresponding to the current sentence to be aligned and the paragraph vector of each basic paragraph of the transcribed text, and obtain the similarity value between the current sentence to be aligned and each basic paragraph.

[0102] Specifically, the order of sentences to be aligned refers to the order of the sentences to be aligned within the text to be aligned, and the current sentence to be aligned is set according to the order of the sentences to be aligned.

[0103] Preferably, if the current sentence to be aligned is the first sentence, then the vector similarity between the reference sentence vector corresponding to the current sentence to be aligned and the paragraph vector of the first basic paragraph of the transcribed text is calculated to obtain the similarity value;

[0104] If the current sentence to be aligned is not the first sentence, calculate the average vector of the reference sentence vector of the current sentence to be aligned and the reference sentence vector of the previous sentence to be aligned. Compare the average vector with the paragraph vector of the current base paragraph to obtain the similarity value. Combine the multiple similarity values ​​into a similarity set and record the largest similarity value.

[0105] In this embodiment, a similarity value is calculated, and the target text of the text to be aligned is determined based on the similarity value, thereby further accelerating the text alignment efficiency of the text to be aligned.

[0106] Furthermore, as an optional implementation, in S50, for each basic paragraph, the similarity value corresponding to the basic paragraph is compared with a preset similarity threshold to obtain a comparison result, and the basic paragraph corresponding to the text to be aligned is determined based on the comparison result, including the target text:

[0107] S501, compare the similarity values ​​of the basic paragraphs with the preset similarity threshold to obtain the comparison results.

[0108] S502, If the comparison result is that the similarity value is less than the preset similarity threshold, then the basic paragraph will be used as the target text for the text to be aligned.

[0109] S503, If the comparison result is that the similarity value is greater than the preset similarity threshold, then the above steps of comparing the size relationship between the similarity value and the similarity threshold are repeated in the order of the sentences to be aligned in the text to be aligned.

[0110] The preset similarity threshold is determined by a set of similarity values, and the highest similarity value is selected as the preset similarity threshold.

[0111] In step S503, if the comparison result is that the similarity value is greater than the preset similarity threshold, then the current similarity value is taken as the maximum similarity value, i.e., the preset similarity threshold.

[0112] In this embodiment, by comparing the similarity value with a preset similarity threshold, it is determined whether the base paragraph is the target text of the text to be aligned, thereby reducing the complexity of segmenting the text to be aligned and further ensuring the efficiency of text alignment.

[0113] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0114] In one embodiment, a text alignment device is provided, which corresponds one-to-one with the text alignment methods described in the above embodiments. For example... Figure 3 As shown, the text alignment device includes:

[0115] The text to be aligned processing module 31 is used to obtain the text to be aligned, segment the text to be aligned into sentences, obtain the sentences to be aligned, and generate a reference sentence vector of the sentences to be aligned.

[0116] The text transcription module 32 is used to acquire audio data corresponding to the text to be aligned, perform speech recognition on the audio data, and generate transcribed text of the audio data, wherein the transcribed text includes at least one basic paragraph.

[0117] The paragraph vector generation module 33 is used to obtain the paragraph information of the transcribed text as the first paragraph information, and generate the paragraph vector of each basic paragraph in the transcribed text based on the first paragraph information.

[0118] The similarity calculation module 34 is used to determine the order of each basic paragraph in the transcribed text based on the information of the first paragraph, and calculate the vector similarity between the paragraph vector of each basic paragraph and the vector of the reference sentence in the order of the first paragraph to obtain a similarity set, wherein the similarity set contains the similarity value corresponding to each basic paragraph.

[0119] The target text determination module 35 is used to compare the similarity value corresponding to each basic paragraph with a preset similarity threshold, obtain the comparison result, and determine the basic paragraph corresponding to the text to be aligned as the target text based on the comparison result.

[0120] The text alignment module 36 is used to summarize each target text in sequence to obtain the target aligned text of the text to be aligned.

[0121] Furthermore, as an optional implementation, the text processing module 31 to be aligned includes:

[0122] The text segmentation unit is used to segment the text to be aligned into sentences, resulting in a set of sentences to be aligned, which includes at least one sentence to be aligned.

[0123] The sentence segmentation unit is used to segment sentences in the set of sentences to be aligned using a word segmenter, resulting in segmented sentences.

[0124] The reference vector generation unit is used to generate reference sentence vectors for segmented sentences based on a pre-trained model.

[0125] Furthermore, as an optional implementation, the text transcription module 32 includes:

[0126] The endpoint detection unit is used to perform endpoint detection on audio data based on the duration of speech pauses, and to determine the start frame marker and end frame marker of each audio segment.

[0127] The paragraph information determination unit is used to take a set of start frame markers and end frame markers as a basic paragraph, determine the first paragraph information, and segment the audio data according to the first paragraph information to obtain multiple sub-audio segments.

[0128] The transcription text generation unit is used to perform speech recognition on each audio segment and generate transcribed text with information from the first segment.

[0129] Furthermore, as an optional implementation, the paragraph vector generation module 33 includes:

[0130] The transcription sentence segmentation unit is used to segment the transcribed text of each basic paragraph into sentences based on the information of the first paragraph, so as to obtain a set of transcribed sentences for each basic paragraph.

[0131] The transcription vector generation unit is used to generate the transcription sentence vector for each transcribed sentence in each base paragraph based on a pre-trained model.

[0132] The paragraph vector generation unit is used to sum and average the transcribed sentence vectors of each basic paragraph to obtain the paragraph vector of each basic paragraph.

[0133] Furthermore, as an optional implementation, the text alignment device also includes:

[0134] The speech feature generation module is used to generate speech feature vectors for each base segment of the audio data using Mel-frequency cepstral coefficients.

[0135] The fusion feature vector is used to fuse the speech feature vector with the paragraph vector for each base paragraph to obtain the fusion feature vector.

[0136] Furthermore, as an optional implementation, the similarity calculation module 34 includes:

[0137] The sequence recording unit is used to set the current sentence to be aligned based on the order of the sentences to be aligned in the text to be aligned, and to record the current sentence order number.

[0138] The similarity calculation unit is used to calculate the vector similarity between the reference sentence vector corresponding to the current sentence to be aligned and the paragraph vector of each basic paragraph of the transcribed text, so as to obtain the similarity value between the current sentence to be aligned and each basic paragraph.

[0139] Furthermore, as an optional implementation, the target text determination module 35 includes:

[0140] The comparison result generation unit is used to compare the similarity value of the basic paragraphs with the preset similarity threshold to obtain the comparison result.

[0141] The target text determination unit is used to determine the base paragraph as the target text of the text to be aligned if the similarity value is less than a preset similarity threshold.

[0142] The loop execution unit is used to perform the above steps of comparing the similarity value with the similarity threshold in a loop, according to the order of the sentences to be aligned in the text to be aligned, if the comparison result is that the similarity value is greater than the preset similarity threshold.

[0143] The terms "first" and "second" in the above-mentioned modules / units are only used to distinguish different modules / units and are not intended to specify which module / unit has a higher priority or any other limiting meaning. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products, or devices. The module divisions appearing in this application are merely logical divisions; in actual applications, different division methods may be used.

[0144] For specific limitations regarding the text alignment device, please refer to the limitations on the text alignment method above, which will not be repeated here. Each module in the aforementioned text alignment device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in the computer device, or stored in software in the memory of the computer device, so that the processor can call and execute the operations corresponding to each module.

[0145] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and the database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores data involved in the text alignment method. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a text alignment method.

[0146] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of the text alignment method described in the above embodiments, for example... Figure 2 The steps S10 to S60 shown, as well as other extensions and related steps of the method, are examples. Alternatively, when the processor executes a computer program, it implements the functions of each module / unit of the text alignment device in the above embodiments, for example... Figure 3 The functions of modules 31 to 36 are shown. To avoid repetition, they will not be described again here.

[0147] A processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of a computer device, connecting all parts of the computer device through various interfaces and lines.

[0148] Memory can be used to store computer programs and / or modules. The processor implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory. Memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area can store data created based on the use of the mobile phone (such as audio data, video data, etc.).

[0149] The memory can be integrated into the processor or set up separately from the processor.

[0150] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When executed by a processor, the computer program implements the steps of the text alignment method described in the above embodiments, for example... Figure 2 The steps S10 to S60 shown, as well as other extensions and related steps of the method, are examples. Alternatively, when a computer program is executed by a processor, it implements the functions of each module / unit of the text alignment device in the above embodiments, for example... Figure 3 The functions of modules 31 to 36 are shown. To avoid repetition, they will not be described again here.

[0151] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0152] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0153] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A text alignment method, characterized in that, include: Obtain the text to be aligned, segment the text to be aligned into sentences to obtain the text sentences to be aligned, and generate a reference sentence vector for the text sentences to be aligned; Acquire audio data corresponding to the text to be aligned, perform speech recognition on the audio data, and generate transcribed text of the audio data, wherein the transcribed text includes at least one basic paragraph; Obtain the paragraph information of the transcribed text as the first paragraph information, and generate a paragraph vector for each basic paragraph in the transcribed text based on the first paragraph information; Based on the first paragraph information, the order of each basic paragraph in the transcribed text is determined. According to the order, the vector similarity between the paragraph vector of each basic paragraph and the vector of the reference sentence is calculated in turn to obtain a similarity set, wherein the similarity set contains the similarity value corresponding to each basic paragraph. For each of the basic paragraphs, the similarity value corresponding to the basic paragraph is compared with a preset similarity threshold to obtain a comparison result. Based on the comparison result, the basic paragraph corresponding to the text to be aligned is determined as the target text. Following the aforementioned order, each target text is summarized sequentially to obtain the target aligned text of the text to be aligned. Based on the first paragraph information, the order of each basic paragraph in the transcribed text is determined. Following this order, the vector similarity between the paragraph vector of each basic paragraph and the reference sentence vector is calculated sequentially, resulting in a similarity set including: Based on the order of sentences to be aligned in the text to be aligned, set the current sentence to be aligned and record the current sentence order number; Calculate the vector similarity between the reference sentence vector corresponding to the current sentence to be aligned and the paragraph vector of each basic paragraph of the transcribed text, and obtain the similarity value between the current sentence to be aligned and each basic paragraph; For each of the basic paragraphs, the similarity value corresponding to the basic paragraph is compared with a preset similarity threshold to obtain a comparison result. Based on the comparison result, the basic paragraph corresponding to the text to be aligned is determined as the target text, including: The similarity values ​​of the base paragraphs are compared with the preset similarity threshold to obtain the comparison result; If the comparison result is that the similarity value is less than the preset similarity threshold, then the base paragraph is used as the target text of the text to be aligned; If the comparison result is that the similarity value is greater than the preset similarity threshold, then the above steps of comparing the size relationship between the similarity value and the similarity threshold are repeated in the order of the sentences to be aligned in the text to be aligned.

2. The text alignment method according to claim 1, characterized in that, The steps of obtaining the text to be aligned, segmenting the text to be aligned into sentences to obtain the sentences to be aligned, and generating a reference sentence vector for the sentences to be aligned include: The text to be aligned is segmented into sentences to obtain a set of sentences to be aligned, the set of sentences to be aligned including at least one sentence to be aligned. The sentences to be aligned in the set of sentences to be aligned are segmented using a word segmenter to obtain segmented sentences; Based on the pre-trained model, a reference sentence vector is generated for the segmented sentence.

3. The text alignment method according to claim 1, characterized in that, The step of acquiring audio data corresponding to the text to be aligned, performing speech recognition on the audio data, and generating transcribed text of the audio data includes: Based on the duration of speech pauses, endpoint detection is performed on the audio data to determine the start and end frame markers for each sub-audio segment. A set of start frame markers and end frame markers are used as a basic segment to determine the first segment information. Based on the first segment information, the audio data is segmented to obtain multiple sub-audio segments. Speech recognition is performed on each of the sub-audio segments to generate the transcribed text containing information from the first segment.

4. The text alignment method according to claim 1, characterized in that, The step of obtaining the paragraph information of the transcribed text as first paragraph information, and generating a paragraph vector for each basic paragraph in the transcribed text based on the first paragraph information, includes: Based on the information in the first paragraph, the transcribed text of each basic paragraph is segmented into sentences to obtain a set of transcribed sentences for each basic paragraph; Based on the pre-trained model, a transcribed sentence vector is generated for each transcribed sentence in each base paragraph; The transcribed sentence vectors of each base paragraph are summed and averaged to obtain the paragraph vector of each base paragraph.

5. The text alignment method according to claim 1, characterized in that, After obtaining the paragraph information of the transcribed text as first paragraph information, and generating a paragraph vector for each basic paragraph in the transcribed text based on the first paragraph information, the method includes: The speech feature vector of the audio data for each basic segment is generated using Mel-frequency cepstral coefficients. For each basic segment, the speech feature vector is fused with the segment vector to obtain a fused feature vector.

6. A text alignment device, characterized in that, include: The text to be aligned processing module is used to acquire the text to be aligned, segment the text to be aligned into sentences, obtain the text sentences to be aligned, and generate a reference sentence vector for the text sentences to be aligned. A text transcription module is used to acquire audio data corresponding to the text to be aligned, perform speech recognition on the audio data, and generate transcribed text of the audio data, wherein the transcribed text includes at least one basic paragraph; The paragraph vector generation module is used to obtain the paragraph information of the transcribed text as the first paragraph information, and generate the paragraph vector of each basic paragraph in the transcribed text based on the first paragraph information. The similarity calculation module is used to determine the order of each basic paragraph in the transcribed text based on the first paragraph information, and calculate the vector similarity between the paragraph vector of each basic paragraph and the reference sentence vector in the order to obtain a similarity set, wherein the similarity set contains the similarity value corresponding to each basic paragraph. The target text determination module is used to compare the similarity value corresponding to each base paragraph with a preset similarity threshold to obtain a comparison result, and determine the base paragraph corresponding to the text to be aligned as the target text based on the comparison result; The text alignment module is used to summarize each target text in the order described above to obtain the target aligned text of the text to be aligned. Based on the first paragraph information, the order of each basic paragraph in the transcribed text is determined. Following this order, the vector similarity between the paragraph vector of each basic paragraph and the reference sentence vector is calculated sequentially, resulting in a similarity set including: Based on the order of sentences to be aligned in the text to be aligned, set the current sentence to be aligned and record the current sentence order number; Calculate the vector similarity between the reference sentence vector corresponding to the current sentence to be aligned and the paragraph vector of each basic paragraph of the transcribed text, and obtain the similarity value between the current sentence to be aligned and each basic paragraph; For each of the basic paragraphs, the similarity value corresponding to the basic paragraph is compared with a preset similarity threshold to obtain a comparison result. Based on the comparison result, the basic paragraph corresponding to the text to be aligned is determined as the target text, including: The similarity values ​​of the base paragraphs are compared with the preset similarity threshold to obtain the comparison result; If the comparison result is that the similarity value is less than the preset similarity threshold, then the base paragraph is used as the target text of the text to be aligned; If the comparison result is that the similarity value is greater than the preset similarity threshold, then the above steps of comparing the size relationship between the similarity value and the similarity threshold are repeated in the order of the sentences to be aligned in the text to be aligned.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the text alignment method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the text alignment method as described in any one of claims 1 to 5.

Images