Segment processing method, device, equipment and storage medium

Abstract

The present disclosure relates to a segmentation processing method, device, equipment and storage medium. The method includes: obtaining a target text after speech recognition, the target text includes multiple sentences; obtaining a prediction result corresponding to each sentence in the multiple sentences through multiple prediction processes, the prediction result is used to indicate whether the corresponding sentence is a segmented sentence, and each prediction process is performed on one or more target sentences in the multiple sentences; in each prediction process, determining the semantic representation vector of each target sentence to be predicted this time, and determining the prediction result of the target sentence based on the semantic representation vector of the target sentence to be predicted this time; and segmenting the target text based on the prediction result of each sentence. The technical solution disclosed in the present disclosure can make the paragraph structure and chapter structure of the target text clearer, improve the readability of the target text, reduce the difficulty of the user's reading and comprehension of the target text, and thereby increase the speed at which the user reads the target text.

Description

Technical Field

[0001] The present disclosure relates to the field of information technology, and in particular to a segment processing method, apparatus, device, and storage medium. Background Art

[0002] With the continuous development of technology, it is now possible to convert audio or voice into text through speech recognition.

[0003] However, the text obtained after speech recognition may not be segmented, or may be segmented but with poor segmentation effect, resulting in poor readability of the text. Summary of the Invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a segmentation processing method, apparatus, device and storage medium.

[0005] In a first aspect, an embodiment of the present disclosure provides a segmentation processing method, including:

[0006] Acquire a target text after speech recognition, wherein the target text includes a plurality of sentences;

[0007] Obtaining a prediction result corresponding to each of the plurality of sentences through multiple prediction processes, wherein the prediction result is used to indicate whether the corresponding sentence is a segmented sentence, and each prediction process is performed on one or more target sentences in the plurality of sentences;

[0008] During each prediction process, determining a semantic representation vector for each target sentence in the one or more target sentences to be predicted, and determining a prediction result for any target sentence in the one or more target sentences to be predicted based on the semantic representation vector of the target sentence;

[0009] The target text is segmented according to the prediction result of each sentence in the multiple sentences.

[0010] In a second aspect, an embodiment of the present disclosure provides an audio processing method, including:

[0011] Acquiring audio data, wherein the audio data is audio data in an online education course, or the audio data is teaching audio data recorded in real time or at a historical time;

[0012] Performing speech recognition on the audio data to obtain a target text, where the target text includes a plurality of sentences;

[0013] Obtaining a prediction result corresponding to each of the plurality of sentences through multiple prediction processes, wherein the prediction result is used to indicate whether the corresponding sentence is a segmented sentence, and each prediction process is performed on one or more target sentences in the plurality of sentences;

[0014] During each prediction process, determining a semantic representation vector for each target sentence in the one or more target sentences to be predicted, and determining a prediction result for any target sentence in the one or more target sentences to be predicted based on the semantic representation vector of the target sentence;

[0015] According to the prediction result of each sentence in the plurality of sentences, the target text is pre-segmented to obtain a pre-segmentation result, and the pre-segmentation result is output.

[0016] In a third aspect, an embodiment of the present disclosure provides an audio processing method, including:

[0017] Acquiring audio data, where the audio data is audio data in a network conference, or the audio data is conference audio data recorded in real time or at a historical time;

[0018] Performing speech recognition on the audio data to obtain a target text, where the target text includes a plurality of sentences;

[0019] Obtaining a prediction result corresponding to each of the plurality of sentences through multiple prediction processes, wherein the prediction result is used to indicate whether the corresponding sentence is a segmented sentence, and each prediction process is performed on one or more target sentences in the plurality of sentences;

[0020] During each prediction process, determining a semantic representation vector for each target sentence in the one or more target sentences to be predicted, and determining a prediction result for any target sentence in the one or more target sentences to be predicted based on the semantic representation vector of the target sentence;

[0021] According to the prediction result of each sentence in the plurality of sentences, the target text is pre-segmented to obtain a pre-segmentation result, and the pre-segmentation result is output.

[0022] In a fourth aspect, an embodiment of the present disclosure provides a segment processing device, including:

[0023] An acquisition module is used to acquire a target text after speech recognition, wherein the target text includes a plurality of sentences;

[0024] a prediction module, configured to obtain a prediction result corresponding to each of the plurality of sentences through multiple prediction processes, wherein the prediction result is used to indicate whether the corresponding sentence is a segmented sentence, and each prediction process is performed on one or more target sentences in the plurality of sentences;

[0025] a determination module configured to determine, during each prediction process, a semantic representation vector of each target sentence in the one or more target sentences to be predicted, and determine a prediction result for any target sentence in the one or more target sentences to be predicted based on the semantic representation vector of the target sentence;

[0026] A segmentation module is used to segment the target text according to the prediction result of each sentence in the multiple sentences.

[0027] In a fifth aspect, an embodiment of the present disclosure provides an electronic device, including:

[0028] Memory;

[0029] processor; and

[0030] computer programs;

[0031] The computer program is stored in the memory and is configured to be executed by the processor to implement the method as described in the first aspect, the second aspect or the third aspect.

[0032] In a sixth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the method described in the first aspect, the second aspect, or the third aspect.

[0033] The segmentation processing method, device, equipment and storage medium provided by the embodiments of the present disclosure are essentially to determine which sentences in the target text are the ending sentences of the paragraphs based on the semantic representation vectors of each sentence in the target text, and then segment the target text according to the position of the ending sentences of the paragraphs to obtain the segmented target text. It can make the paragraph structure and chapter structure of the target text clearer, improve the readability of the target text, reduce the difficulty of the user's reading and comprehension of the target text, and thus increase the speed at which the user reads the target text. In addition, if subsequent downstream applications need to use the target text, using the target text after segmentation is beneficial to improving the operating efficiency of downstream applications compared to using the target text before segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0035] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0036] Figure 1 A flowchart of a segmentation processing method provided by an embodiment of the present disclosure;

[0037] Figure 2 A flowchart of a method for implementing S120 provided in an embodiment of the present disclosure;

[0038] Figure 3 for Figure 2 Schematic diagram of the method provided in;

[0039] Figure 4 A schematic diagram of a sliding window before and after sliding provided by an embodiment of the present disclosure;

[0040] Figure 5 A schematic diagram of a method for implementing S122 provided in the present disclosure;

[0041] Figure 6 A schematic diagram of a method for updating a sliding window using an adaptive sliding window according to an embodiment of the present disclosure;

[0042] Figure 7 A schematic diagram of another method for updating a sliding window using an adaptive sliding window provided in an embodiment of the present disclosure;

[0043] Figure 8 A flowchart of an audio processing method provided by an embodiment of the present disclosure;

[0044] Figure 9 A flowchart of another audio processing method provided by an embodiment of the present disclosure;

[0045] Figure 10 A schematic structural diagram of a segment processing device provided in an embodiment of the present disclosure;

[0046] Figure 11 A schematic diagram of the structure of an electronic device embodiment provided by the present disclosure. DETAILED DESCRIPTION

[0047] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features therein can be combined with each other in the absence of conflict.

[0048] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0049] Typically, existing speech recognition technology can convert audio or speech into text. However, the text obtained after speech recognition may not be segmented, or even if it is segmented, the segmentation method is to treat each sentence as a segment, resulting in poor segmentation effect. Both of these situations lead to poor readability of the text. To address this problem, the embodiments of the present disclosure provide a segmentation processing method, which is described below in conjunction with specific embodiments.

[0050] Figure 1 A flowchart of a segmentation processing method provided by an embodiment of the present disclosure. This embodiment is applicable to the case where the client performs segmentation processing on the text information after voice recognition, and the method can be executed by a segmentation processing device, which can be implemented in software and / or hardware, and the device can be configured in an electronic device, such as a terminal, specifically including but not limited to smart phones, PDAs, tablet computers, wearable devices with display screens, desktop computers, laptop computers, all-in-one computers, smart home devices, etc. Alternatively, this embodiment is applicable to the case where the server performs segmentation processing on the text information after voice recognition, and the method can be executed by a segmentation processing device, which can be implemented in software and / or hardware, and the device can be configured in an electronic device, such as a server. The segmentation processing method is introduced below using a terminal as an example. Figure 1 As shown, the specific steps of this method are as follows:

[0051] S110 , obtaining a target text after speech recognition, where the target text includes multiple sentences.

[0052] The target text refers to the recognition result of audio or speech using speech recognition technology, which is expressed as a text sequence. The target text includes multiple sentences, each of which is marked with a sentence-ending marker. There are various markers that can be used to indicate the end of a sentence, and punctuation marks can be used, for example. Furthermore, sentence-ending punctuation marks can include a period, a question mark, an exclamation mark, and the like.

[0053] It should be noted that, in practice, when performing speech recognition on audio or speech, the direct recognition results may or may not include a sentence-ending marker. If a sentence-ending marker is not included, it is necessary to add the sentence-ending marker to the recognition results to form the target text.

[0054] S120. Through multiple prediction processes, a prediction result corresponding to each sentence in the multiple sentences is obtained. The prediction result is used to indicate whether the corresponding sentence is a segmented sentence. Each prediction process is performed on one or more target sentences in the multiple sentences.

[0055] A segmentation sentence is the last sentence of a paragraph, i.e., the concluding sentence of a paragraph. For example, the target text includes n sentences arranged in sequence, namely sentence 1, sentence 2, sentence 3, ..., sentence n. If sentence 2 is a segmentation sentence, it means that sentence 2 and sentence 3 belong to different paragraphs.

[0056] The essence of this step is to divide the multiple sentences in the target text into multiple batches, each batch containing one or more target sentences. Each prediction is performed on all target sentences in the same batch. For example, if the target text is divided into N batches, then N predictions are performed.

[0057] S130. During each prediction process, determine the semantic representation vector of each target sentence in the one or more target sentences to be predicted, and determine the prediction result of any target sentence based on the semantic representation vector of any target sentence in the one or more target sentences to be predicted.

[0058] Among them, the semantic representation vector of a sentence refers to a parameter that can reflect whether the sentence is a segmented sentence. It is obtained based on the characteristics of the sentence and the characteristics of other sentences in the target text, and it reflects the closeness of the semantic logical relationship between the sentence and other sentences.

[0059] S140 : Segment the target text according to the prediction result of each sentence in the plurality of sentences.

[0060] There are many ways to implement this step. For example, if the prediction result indicates that any sentence is a segmented sentence, segmentation processing is performed at the end position of any sentence.

[0061] The segmentation processing may specifically include adding paragraph marks in the target file.

[0062] For example, assume the target text is: "First we xxxxxx. Second xxxxxx. Then xxxxxx". Among them, "xxxxxx" represents a description of hundreds of words. After processing using the disclosed technical solution, the final form displayed to the user is:

[0063] "First we xxxxxx."

[0064] "Secondly, xxxxxx."

[0065] "Then xxxxxx."

[0066] The essence of the above technical solution is to determine which sentences in the target text are the ending sentences of the paragraphs based on the semantic representation vectors of each sentence in the target text, and then segment the target text according to the position of the ending sentences of the paragraphs to obtain the segmented target text. It can make the paragraph structure and chapter structure of the target text clearer, improve the readability of the target text, reduce the difficulty of users reading and understanding the target text, and thus increase the speed at which users read the target text. Moreover, if subsequent downstream applications need to use the target text, using the target text after segmentation is beneficial to improving the operating efficiency of downstream applications compared to using the target text before segmentation.

[0067] Furthermore, in the above technical solution, the semantic representation vector of a sentence is calculated based on the features of the sentence and the features of other sentences in the target text. The multiple sentences in the target text are divided into multiple batches, and prediction processing is performed on the target sentences in each batch. Essentially, the semantic representation vector of each sentence is calculated based on the features of other adjacent sentences in the same batch. Compared to solutions that do not divide sentences into batches, this setting can reduce the amount of computation required to obtain the semantic representation vector of each sentence and has lower performance requirements for the terminal device.

[0068] The above technical solution can be applied to multiple scenarios, such as the scenario of processing the audio data generated by the teacher's lectures in the teaching process (including online teaching process and offline teaching process), and obtaining the text information after segmentation processing corresponding to the audio data, such as segmented course notes or knowledge point notes, etc., to help students understand the teaching content and improve the students' learning effect on the knowledge they have learned. The above technical solution can also be applied to the conference scenario, processing the audio data of each speaker during the meeting, and obtaining the text information after segmentation processing corresponding to the audio data, such as segmented meeting minutes or meeting work summary, etc., to help other members participating in the meeting understand the speaker's point of view and improve the other members participating in the meeting's understanding of the spirit of membership. The above technical solution can also be applied to the scenario of providing online segmentation processing services, such as users who need services upload audio data or target text, and transcribe it into segmented text based on the audio data or target text.

[0069] It should be noted that in the above technical solution, after executing S130, it may also include: using an identifier to mark the prediction results of each sentence, and the identifier used for segmented sentences is different from the identifier used for non-segmented sentences. For example, "segmented" is used as the identifier used for segmented sentences, and "non-segmented" is used as the identifier used for non-segmented sentences. Alternatively, "1" is used as the identifier used for segmented sentences, and "0" is used as the identifier used for non-segmented sentences. By marking, it is convenient for subsequent users to view the prediction results and it is also convenient for subsequent segmentation processing.

[0070] Optionally, based on the above-mentioned technical solutions, after executing S130, it may also include: outputting and displaying the prediction results of each sentence in the target text; obtaining the user's evaluation results of the prediction results of each sentence in the target text; and adjusting the prediction results of each sentence in the target text based on the evaluation results. Accordingly, S140 is replaced by: segmenting the target text according to the adjusted prediction results of each sentence in the multiple sentences. The essence of this setting is to present the terminal's prediction results for each sentence to the user so that the user can judge whether the prediction results are correct. If not, modification suggestions are given. The erroneous prediction results are corrected according to the modification suggestions. This setting can ensure the accuracy of the segmentation processing.

[0071] Figure 2 This is a flowchart of a method for implementing S130 provided in an embodiment of the present disclosure. Figure 3 for Figure 2 The principle diagram of the method provided in . It should be noted that due to Figure 2 The essence of is to use the sliding window to determine the semantic representation vector of each sentence in the target text and the result prediction in batches. In practice, each execution Figure 2 The method means completing the semantic representation vector determination and result prediction of a batch of target sentences. In practice, it needs to be executed multiple times. Figure 2 method until all sentences in the entire target text are traversed.

[0072] See also Figure 2 and Figure 3 , the method comprising:

[0073] S121. Obtain one or more target sentences to be predicted from multiple sentences using a sliding window.

[0074] The sliding window is used to select one or more target sentences in each batch from the target text for which semantic representation vectors need to be determined.

[0075] Figure 4 This is a schematic diagram of the sliding window before and after sliding provided by the embodiment of the present disclosure. Figure 4, the target text includes n sentences arranged in sequence, namely sentence 1, sentence 2, sentence 3, sentence 4, sentence 5, sentence 6, ..., sentence n. The dotted box represents a sliding window. The sliding direction of the sliding window is fixed and consistent with the direction of sentence arrangement in the target text. At the beginning, the sliding window includes sentence 1, sentence 2, and sentence 3, indicating that sentence 1, sentence 2, and sentence 3 are all target sentences for the first batch that need to determine the semantic representation vector and perform result prediction. After sentences 1, sentence 2, and sentence 3 have determined the semantic representation vectors and performed result prediction, the sliding window is updated (i.e., it slides). After sliding, the sliding window includes sentence 3, sentence 4, and sentence 5, indicating that sentences 3, sentence 4, and sentence 5 are all target sentences for the second batch that need to determine the semantic representation vector and perform result prediction. Repeat this process until the semantic representation vector determination and result prediction work of all sentences in the entire target text are completed.

[0076] It should be noted that Figure 4 In the example, both the first batch of target sentences and the second batch of target sentences include sentence 3. This is only a specific example in the present application and does not limit the present application. In practice, different batches of target sentences may include the same sentence or may not include the same sentence.

[0077] In practice, a maximum sliding window capacity can be defined to group sentences in the target text into batches. Optionally, the maximum sliding window capacity can be the maximum number of sentences or characters that the sliding window can accommodate. Furthermore, the maximum sliding window capacity can be set to a fixed value when determining target sentences for different batches, or it can be set to a variable value when determining target sentences for different batches. This application does not impose any restrictions on this.

[0078] Typically, in practice, the maximum capacity of the sliding window can be set to the maximum number of characters that the sliding window can accommodate, and when determining target sentences for different batches, the maximum capacity of the sliding window is a fixed value. For example, if the maximum capacity of the sliding window is 512 characters, and it is specified that the first character in the sliding window is a preset special character, such as "CLS", then the remaining space in the sliding window can hold a maximum of 511 characters. Therefore, when determining target sentences for different batches, it is necessary to ensure that the number of all target sentences in each batch is the maximum number of consecutive sentences that meets the condition that "the total number of characters is less than or equal to 511 characters". For example, if the total number of characters of three consecutive sentences is exactly equal to 511 characters, then the three consecutive sentences can be placed in the sliding window as target sentences for the same batch. If three consecutive sentences are sentence 1, sentence 2, and sentence 3, and the total number of characters in these three sentences is less than 511, and if sentence 4 is added, the total number of characters in sentences 1, 2, 3, and 4 will exceed 511, then sentences 1, 2, and 3 can be placed in the sliding window as the target sentences for the same batch. This setting can control the amount of computation required to calculate the semantic representation vector for each batch, so that the computation required to calculate the semantic representation vector for each batch is less than the maximum computation required by the terminal, thereby ensuring that the terminal can successfully execute the technical solution provided by this disclosure.

[0079] It should be noted that, in the above technical solution, the first character in the sliding window is specified to be a preset special character "CLS". This is merely a specific example of the present application and is not intended to limit the present application. In practice, the first character in the sliding window can also be set to be the first character of the first target sentence, i.e., no special character is specified. Alternatively, the first character in the sliding window can be set to be another preset special character, such as "C".

[0080] S122. Perform feature extraction on each target sentence in the sliding window to obtain a semantic representation vector for each target sentence in the sliding window.

[0081] There are many ways to implement this step. For example, each character in each target sentence can be used as a feature to extract it, and the convolutional neural network technology is used to calculate the semantic representation vector of each target sentence in the sliding window.

[0082] For example, Figure 5 A schematic diagram of a method for implementing S122 provided in the present disclosure. Figure 3 and Figure 5, assuming that the first character in the sliding window is a preset special character "CLS", E0 is the vector corresponding to the special character "CLS". At the same time, assuming that the sliding window includes 3 target sentences, and the 3 target sentences contain a total of n characters, E1, E2, E3, ... E n are the vectors corresponding to the n characters.

[0083] First, according to E0, E1, E2, E3, ... E n Calculate T 10 、T 11 、T 12 、T 13 、……T 1n Among them, T 10 、T 11 、T 12 、T 13 、……T 1n Can be different. For example, according to E0, E1, E2, E3, ... E n Calculate T 10 When E0 has the largest proportion; according to E0, E1, E2, E3, ... E n Calculate T 11 When , E1 has the largest proportion, and so on.

[0084] Secondly, according to T 10 、T 11 、T 12 、T 13 、……T 1n Calculate T 20 、T 21 、T 22 、T 23 、……T 2n The calculation process can refer to E0, E1, E2, E3, ... E n Calculate T 10 、T 11 、T 12 、T 13 、……T 1n process.

[0085] Then, according to T 20 Calculate T0, according to T 21 Calculate and get T1, and so on. Among them, T0 is the semantic representation vector corresponding to the special character "CLS", T0, T1, T2, ... T n are the semantic representation vectors corresponding to n characters in the three target sentences respectively.

[0086] Finally, according to T1, T2, ... T n, the semantic representation vectors of the three target sentences are calculated. For example, T1 and T2 are the semantic representation vectors corresponding to the characters in sentence 1, and the semantic representation vector of sentence 1 can be obtained based on T1 and T2. There are many methods for obtaining the semantic representation vector of sentence 1 based on T1 and T2, and this application does not limit this. For example, the semantic representation vector of sentence 1 can be obtained by averaging T1 and T2.

[0087] It is understandable that in Figure 5 In the process of obtaining T0, T1, T2, ... T n Previously, there were two levels of calculation process, where T 10 、T 11 、T 12 、T 13 、……T 1n Corresponding to the calculation process of the first level, T 20 、T 21 、T 22 、T 23 、……T 2n This corresponds to the calculation process of the second level. This is only a specific example of the present application, and is not intended to limit the present application. In other embodiments, the calculation process is not limited to two levels.

[0088] pass Figure 5 It's understandable that the semantic representation vector of any sentence within a sliding window is correlated with the other sentences within the same window. Therefore, using convolutional neural networks to calculate the semantic representation vector of each target sentence within a sliding window essentially involves determining the semantic representation vector of each sentence based on the semantic relevance between different target sentences within the same sliding window.

[0089] S123. Determine a prediction result of any target sentence according to the semantic representation vector of any target sentence among the one or more target sentences to be predicted this time.

[0090] The essence of this technical solution is to use a sliding window to identify target sentences in batches and determine the semantic representation vector for each target sentence in the batch, thereby obtaining prediction results for each target sentence. The semantic representation vector for each target sentence in the batch is determined by combining the features of the target sentence with those of other target sentences in the batch. This effectively reduces the computational effort required to calculate the semantic representation vector for each batch, lowering the performance requirements for the terminal device.

[0091] Based on the above technical solution, optionally, after S123, the method further includes: updating the sliding window, where the updated sliding window includes one or more target sentences to be predicted next. "Updating the sliding window" means controlling the sliding window to slide along its slidable direction to select the next batch of target sentences.

[0092] When executing the step of "updating the sliding window", the sliding window can be updated by using a fixed sliding window method or an adaptive sliding window method, and this application does not impose any restrictions on this.

[0093] If a fixed sliding window method is used to update the sliding window, the number of sliding window steps can be pre-set to m. If the starting position of the sliding window is sentence b when determining the target sentences in the a-th batch, the starting position of the sliding window is sentence b+m when determining the target sentences in the a+1-th batch. For example, if m=2, in the first prediction, the starting position of the sliding window is sentence 1, and the sliding window includes sentences 1, 2, and 3. This prediction is used to obtain prediction results for sentences 1 and 2; in the second prediction, the starting position of the sliding window is sentence 3, and the sliding window includes sentences 3, 4, 5, and 6. This prediction is used to obtain prediction results for sentences 3 and 4; in the third prediction, the starting position of the sliding window is sentence 5, and the sliding window includes sentences 5, 6, and 7. This prediction is used to obtain prediction results for sentences 5 and 6.

[0094] Adaptive sliding window technology does not pre-define the number of sliding window steps. Instead, the number of sliding window steps is determined based on the actual situation each time the sliding window is updated. The number of sliding window steps can be the same or different between two consecutive sliding window updates.

[0095] If the sliding window is updated using an adaptive sliding window method, optionally, the sliding window can be updated based on the prediction result of each target sentence to be predicted this time, and the updated sliding window includes one or more target sentences to be predicted next time.

[0096] Furthermore, if it is determined based on the prediction results of each target sentence to be predicted this time that there is a segmented sentence in the one or more target sentences to be predicted this time, the next sentence of the segmented sentence is used as the first sentence in the updated sliding window. Figure 6 This is a schematic diagram of a method for updating a sliding window using an adaptive sliding window provided in an embodiment of the present disclosure. Figure 6 , the first prediction results in sentence 2 being a segmented sentence. After the first prediction, the sliding window is updated to obtain the second batch of target sentences to be predicted. The starting position of the updated sliding window is sentence 3. A second prediction is performed on the second batch of target sentences to be predicted, and sentence 3 is a segmented sentence. After the second prediction, the sliding window is updated to obtain the third batch of target sentences to be predicted. The starting position of the updated sliding window is sentence 4.

[0097] Furthermore, if it is determined based on the prediction results of each target sentence to be predicted that there are multiple segmented sentences in the multiple target sentences to be predicted, the next sentence of the last segmented sentence in the multiple segmented sentences is used as the first sentence in the updated sliding window. For example, Figure 7 This is another schematic diagram of updating a sliding window using an adaptive sliding window method provided by an embodiment of the present disclosure. Figure 7 , through the first prediction, it is found that sentences 2, 3, and 5 are all segmented sentences. After the first prediction, the sliding window is updated to obtain the second batch of target sentences to be predicted. The starting position of the updated sliding window is sentence 6. Obviously, compared with the solution of using the next sentence of other segmented sentences except the last segmented sentence as the first sentence in the updated sliding window (for example, using sentence 3 as the first sentence in the updated sliding window), this setting can significantly reduce the number of repeated calculations of the semantic representation vector of the same target sentence, thereby improving the execution efficiency of the segmentation processing method.

[0098] Optionally, when executing the step of "updating the sliding window", whether the sliding window is updated by a fixed sliding window method or an adaptive sliding window method, each time the sliding window is updated, it must be ensured that the number of target sentences included in the sliding window is determined according to the maximum capacity of the sliding window.

[0099] Furthermore, based on the above technical solutions, after updating the sliding window according to the prediction results of each target sentence to be predicted this time, the method further includes: if the updated sliding window and the sliding window before the update include the same target sentence, then the prediction result of the target sentence obtained in the sliding window before the update is replaced by the prediction result of the target sentence obtained in the sliding window after the update. Figure 6 , during the first prediction, sentence 3 is not a segmented sentence; but during the second prediction, sentence 3 is a segmented sentence. In this case, sentence 3 is finally determined to be a segmented sentence, that is, the second prediction result of sentence 3 is used to replace the first prediction result of sentence 3.

[0100] The reason why the two prediction results of sentence 3 are different is that the semantic representation vector of any sentence in the sliding window is correlated with other sentences in the sliding window when using the sliding window method. It reflects the closeness of the semantic logical relationship between it and other sentences in the sliding window. For example, Figure 6 In the first prediction, the semantic representation vector of sentence 3 is related to sentences 1 and 2; while in the second prediction, the semantic representation vector of sentence 3 is related to sentences 4, 5, and 6. Therefore, the two prediction results of sentence 3 are different.

[0101] Those skilled in the art will understand that within a paragraph, each sentence describes the same topic; different paragraphs often have different topics. This results in a close semantic and logical relationship between sentences within the same paragraph, while the semantic and logical relationship between sentences belonging to different paragraphs is loose and not close. Therefore, if sentence A and the immediately following sentence B have a very close semantic and logical relationship, then sentence A is not a segmented sentence; if sentence A and the immediately following sentence B have a loose semantic and logical relationship, then sentence A is a segmented sentence.

[0102] Continue to see Figure 6 The reason why sentence 3 was not predicted as a segmented sentence during the first prediction was that sentence 4 was not considered during the first prediction process, and the degree of semantic and logical relationship between sentences 3 and 4 could not be determined. This resulted in an incorrect prediction result for sentence 3.

[0103] By setting that if the updated sliding window and the sliding window before the update include the same target sentence, the prediction result of the target sentence in the sliding window before the update is replaced by the prediction result of the target sentence in the sliding window after the update, the wrong prediction result can be corrected, thereby improving the accuracy of the prediction result of each target sentence in the target text.

[0104] To ensure consistent prediction accuracy for each target sentence in the target text, a comparison between the fixed sliding window method and the adaptive sliding window method reveals that the fixed sliding window method increases the number of identical target sentences in two consecutive batches, resulting in repeated predictions of the same target sentence. However, the adaptive sliding window method, due to its ability to self-correct prediction results, can reduce the number of identical target sentences in two consecutive batches. This reduces the number of repeated predictions of the same target sentence, improving the method's operational efficiency and reducing its operational costs.

[0105] Based on the same inventive concept, the present disclosure also provides an audio processing method, Figure 8 This is a flowchart of an audio processing method provided by an embodiment of the present disclosure. Figure 8 , the audio processing method includes:

[0106] S210: Acquire audio data, where the audio data is audio data in an online education course, or the audio data is teaching audio data recorded in real time or at a historical time;

[0107] S220 , performing speech recognition on the audio data to obtain a target text, where the target text includes a plurality of sentences.

[0108] S230. Through multiple prediction processes, a prediction result corresponding to each sentence in the multiple sentences is obtained. The prediction result is used to indicate whether the corresponding sentence is a segmented sentence. Each prediction process is performed on one or more target sentences in the multiple sentences.

[0109] S240. During each prediction process, determine the semantic representation vector of each target sentence in the one or more target sentences to be predicted, and determine the prediction result of any target sentence based on the semantic representation vector of any target sentence in the one or more target sentences to be predicted.

[0110] S250 , pre-segmenting the target text according to the prediction result of each sentence in the plurality of sentences, obtaining a pre-segmentation result, and outputting the pre-segmentation result.

[0111] The essence of the above technical solution is to apply the segmentation processing method provided by the embodiment of the present disclosure to the teaching scenario, process the audio data generated by the teacher's explanation of the teaching content in the teaching process (including online teaching process and offline teaching process), and obtain the text information after segmentation processing corresponding to the audio data (such as segmented course notes or knowledge point notes, etc.), so as to help students understand the teaching content and improve the students' learning effect of the knowledge they have learned.

[0112] Based on the same inventive concept, the present disclosure also provides an audio processing method, Figure 9 This is a flowchart of an audio processing method provided by an embodiment of the present disclosure. Figure 9 , the audio processing method includes:

[0113] S310: Acquire audio data, where the audio data is audio data in a network conference, or the audio data is conference audio data recorded in real time or at a historical time.

[0114] S320: Perform speech recognition on the audio data to obtain a target text, where the target text includes multiple sentences.

[0115] S330. Through multiple prediction processes, a prediction result corresponding to each sentence in the multiple sentences is obtained. The prediction result is used to indicate whether the corresponding sentence is a segmented sentence. Each prediction process is performed on one or more target sentences in the multiple sentences.

[0116] S340. During each prediction process, determine the semantic representation vector of each target sentence in the one or more target sentences to be predicted, and determine the prediction result of any target sentence based on the semantic representation vector of any target sentence in the one or more target sentences to be predicted.

[0117] S350 , pre-segmenting the target text according to the prediction result of each sentence in the plurality of sentences, obtaining a pre-segmentation result, and outputting the pre-segmentation result.

[0118] The essence of the above technical solution is to apply the segmentation processing method provided by the embodiment of the present disclosure to a conference scenario, process the audio data generated by each speaker's speech during the meeting, and obtain text information after segmentation processing corresponding to the audio data (such as segmented meeting minutes or meeting work summary, etc.), so as to help other members participating in the meeting understand the speaker's point of view and improve the other members participating in the meeting's understanding of the spirit of membership.

[0119] Figure 10 The schematic diagram of the structure of a segment processing device provided by the embodiment of the present disclosure. The segment processing device provided by the embodiment of the present disclosure can execute the processing flow provided by the segment processing method embodiment, such as Figure 10 As shown, the segment processing device includes:

[0120] An acquisition module 410 is configured to acquire a target text after speech recognition, wherein the target text includes a plurality of sentences;

[0121] Prediction module 420, configured to obtain a prediction result corresponding to each of the plurality of sentences through multiple prediction processes, wherein the prediction result indicates whether the corresponding sentence is a segmented sentence, and each prediction process is performed on one or more target sentences in the plurality of sentences;

[0122] a determination module 430 configured to determine, during each prediction process, a semantic representation vector of each target sentence in the one or more target sentences to be predicted, and determine a prediction result for any target sentence in the one or more target sentences to be predicted based on the semantic representation vector of the target sentence;

[0123] The segmentation module 440 is configured to segment the target text according to the prediction result of each sentence in the plurality of sentences.

[0124] Optionally, the determination module is configured to:

[0125] Obtain one or more target sentences to be predicted from the multiple sentences through a sliding window;

[0126] Feature extraction is performed on each target sentence in the sliding window to obtain a semantic representation vector of each target sentence in the sliding window.

[0127] Optionally, the device further includes an update module, which is further configured to:

[0128] After determining the prediction result of any target sentence according to the semantic representation vector of the one or more target sentences to be predicted this time, the sliding window is updated according to the prediction result of each target sentence to be predicted this time, and the updated sliding window includes the one or more target sentences to be predicted next time.

[0129] Optionally, the update module is further configured to:

[0130] If, based on the prediction results of each target sentence to be predicted this time, it is determined that there is a segmented sentence in the one or more target sentences to be predicted this time, the next sentence of the segmented sentence is used as the first sentence in the updated sliding window; or

[0131] If it is determined based on the prediction results of each target sentence to be predicted this time that there are multiple segmented sentences in the multiple target sentences to be predicted this time, the next sentence of the last segmented sentence in the multiple segmented sentences is used as the first sentence in the updated sliding window.

[0132] Optionally, the determination module is further configured to:

[0133] After updating the sliding window according to the prediction result of each target sentence to be predicted this time, if the updated sliding window and the sliding window before the update include the same target sentence, the prediction result of the target sentence obtained in the sliding window before the update is replaced by the prediction result of the target sentence obtained in the sliding window after the update.

[0134] Optionally, the semantic representation vector of any target sentence in the sliding window is related to other target sentences in the sliding window.

[0135] Figure 10 The segmented processing device of the illustrated embodiment can be used to execute the technical solution of the above-mentioned method embodiment. Its implementation principle and technical effects are similar and will not be described in detail here.

[0136] The internal functions and structure of the segment processing device are described above. The device can be implemented as an electronic device. Figure 11 This is a schematic diagram of the structure of an electronic device embodiment provided by the present disclosure. Figure 11 As shown, the electronic device includes a memory 151 and a processor 152 .

[0137] Memory 151 is used to store programs. In addition to the aforementioned programs, memory 151 may also be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, images, videos, and the like.

[0138] The memory 151 can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0139] The processor 152 is coupled to the memory 151 and executes the program stored in the memory 151 to:

[0140] Acquire a target text after speech recognition, wherein the target text includes a plurality of sentences;

[0141] Obtaining a prediction result corresponding to each of the plurality of sentences through multiple prediction processes, wherein the prediction result is used to indicate whether the corresponding sentence is a segmented sentence, and each prediction process is performed on one or more target sentences in the plurality of sentences;

[0142] During each prediction process, determining a semantic representation vector for each target sentence in the one or more target sentences to be predicted, and determining a prediction result for any target sentence in the one or more target sentences to be predicted based on the semantic representation vector of the target sentence;

[0143] The target text is segmented according to the prediction result of each sentence in the multiple sentences.

[0144] Further, if Figure 11 As shown, the electronic device may further include: a communication component 153, a power component 154, an audio component 155, a display 156 and other components. Figure 11 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 11 Components shown.

[0145] The communication component 153 is configured to facilitate wired or wireless communication between the electronic device and other devices. The electronic device can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 153 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 153 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0146] The power supply component 154 provides power to various components of the electronic device. The power supply component 154 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device.

[0147] The audio component 155 is configured to output and / or input audio signals. For example, the audio component 155 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 151 or transmitted via the communication component 153. In some embodiments, the audio component 155 also includes a speaker for outputting audio signals.

[0148] The display 156 includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0149] In addition, an embodiment of the present disclosure further provides a computer-readable storage medium on which a computer program is stored. The computer program is executed by a processor to implement the segmentation processing method described in the above embodiment.

[0150] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0151] The foregoing description is intended only to provide specific embodiments of the present disclosure, intended to enable those skilled in the art to understand and implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the embodiments described herein, but rather to be construed in the broadest manner consistent with the principles and novel features disclosed herein.

Claims

1. A segmentation processing method, wherein: The method comprises: Acquire a target text after speech recognition, wherein the target text includes a plurality of sentences; Obtaining a prediction result corresponding to each of the plurality of sentences through multiple prediction processes, wherein the prediction result is used to indicate whether the corresponding sentence is a segmented sentence, and each prediction process is performed on one or more target sentences in the plurality of sentences; During each prediction process, determining the semantic representation vector of each target sentence in the one or more target sentences to be predicted this time through a sliding window, and determining a prediction result of any target sentence in the one or more target sentences to be predicted this time based on the semantic representation vector of the target sentence; Segmenting the target text according to the prediction result of each sentence in the plurality of sentences; After determining the prediction result of any target sentence among the one or more target sentences to be predicted this time based on the semantic representation vector of the target sentence, the method further includes: The sliding window is updated according to the prediction result of each target sentence to be predicted this time, and the updated sliding window includes one or more target sentences to be predicted next time.

2. The method according to claim 1, wherein Determine the semantic representation vector of each target sentence in one or more target sentences to be predicted, including: Obtain one or more target sentences to be predicted from the multiple sentences through a sliding window; Feature extraction is performed on each target sentence in the sliding window to obtain a semantic representation vector of each target sentence in the sliding window.

3. The method according to claim 2, wherein: The sliding window is updated according to the prediction result of each target sentence to be predicted this time, including: If, based on the prediction results of each target sentence to be predicted this time, it is determined that there is a segmented sentence in the one or more target sentences to be predicted this time, the next sentence of the segmented sentence is used as the first sentence in the updated sliding window; or If it is determined based on the prediction results of each target sentence to be predicted this time that there are multiple segmented sentences in the multiple target sentences to be predicted this time, the next sentence of the last segmented sentence in the multiple segmented sentences is used as the first sentence in the updated sliding window.

4. The method according to claim 2, wherein: After updating the sliding window according to the prediction result of each target sentence to be predicted this time, the method further includes: If the updated sliding window and the sliding window before the update include the same target sentence, the prediction result of the target sentence obtained in the sliding window before the update is replaced by the prediction result of the target sentence obtained in the sliding window after the update.

5. The method according to claim 1 or 2, wherein: The semantic representation vector of any target sentence in the sliding window is related to other target sentences in the sliding window.

6. An audio processing method, wherein: include: Acquiring audio data, wherein the audio data is audio data in an online education course, or the audio data is teaching audio data recorded in real time or at a historical time; Performing speech recognition on the audio data to obtain a target text, where the target text includes a plurality of sentences; Obtaining a prediction result corresponding to each of the plurality of sentences through multiple prediction processes, wherein the prediction result is used to indicate whether the corresponding sentence is a segmented sentence, and each prediction process is performed on one or more target sentences in the plurality of sentences; During each prediction process, determining a semantic representation vector for each target sentence in the one or more target sentences to be predicted, and determining a prediction result for any target sentence in the one or more target sentences to be predicted based on the semantic representation vector of the target sentence; According to the prediction result of each sentence in the plurality of sentences, the target text is pre-segmented to obtain a pre-segmentation result, and the pre-segmentation result is output.

7. An audio processing method, wherein: include: Acquiring audio data, where the audio data is audio data in a network conference, or the audio data is conference audio data recorded in real time or at a historical time; Performing speech recognition on the audio data to obtain a target text, where the target text includes a plurality of sentences; Obtaining a prediction result corresponding to each of the plurality of sentences through multiple prediction processes, wherein the prediction result is used to indicate whether the corresponding sentence is a segmented sentence, and each prediction process is performed on one or more target sentences in the plurality of sentences; During each prediction process, determining the semantic representation vector of each target sentence in the one or more target sentences to be predicted this time through a sliding window, and determining a prediction result of any target sentence in the one or more target sentences to be predicted this time based on the semantic representation vector of the target sentence; Pre-segmenting the target text according to the prediction result of each sentence in the plurality of sentences to obtain a pre-segmentation result, and outputting the pre-segmentation result; After determining the prediction result of any target sentence among the one or more target sentences to be predicted this time based on the semantic representation vector of the target sentence, the method further includes: The sliding window is updated according to the prediction result of each target sentence to be predicted this time, and the updated sliding window includes one or more target sentences to be predicted next time.

8. A segment processing device, wherein: include: An acquisition module is used to acquire a target text after speech recognition, wherein the target text includes a plurality of sentences; a prediction module, configured to obtain a prediction result corresponding to each of the plurality of sentences through multiple prediction processes, wherein the prediction result is used to indicate whether the corresponding sentence is a segmented sentence, and each prediction process is performed on one or more target sentences in the plurality of sentences; a determination module configured to determine, during each prediction process, a semantic representation vector of each target sentence in the one or more target sentences to be predicted this time through a sliding window, and determine a prediction result for any target sentence in the one or more target sentences to be predicted this time based on the semantic representation vector of the target sentence; A segmentation module, configured to segment the target text according to the prediction result of each sentence in the plurality of sentences; The determination module is further configured to update the sliding window according to the prediction result of each target sentence to be predicted this time, wherein the updated sliding window includes one or more target sentences to be predicted next time.

9. An electronic device, wherein: include: Memory; processor; as well as computer programs; The computer program is stored in the memory and configured to be executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images