Long speech streaming method, system, device and storage medium

Abstract

The application provides a long speech streaming output method, system, device and storage medium. The method comprises the following steps: obtaining a first target speech segment and a previous result of all function modules under a previous speech segment; splicing the first target speech segment and a previous temporary part of a first function unit to obtain a first target fixed part and a first target temporary part of the first function unit under the first target speech segment; splicing the first target fixed part of the first function unit under the first target speech segment and a previous temporary part of a next function unit under the previous speech segment to obtain a first target fixed part and a first target temporary part of the next function unit under the first target speech segment; repeating the step until a first target fixed part and a first target temporary part of a second function unit under the first target speech segment are obtained; and outputting a first on-screen display result. The application improves speech recognition accuracy and continuity of post-processing, and improves user experience.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of speech processing, in particular to a long speech streaming output method, system, device and storage medium. BACKGROUND

[0002] Long speech real-time recognition is a technology of converting speech signals into text information, and is widely used in various scenarios, including conference recording and video live real-time captioning. In these situations, the system will recognize the user's speech in real time while the user is speaking, and display the recognition result on the screen in real time.

[0003] Generally, when the speech starts to be detected, the system starts to recognize the valid speech in real time, and then takes the current optimal recognition result as an intermediate result to update the screen display at intervals. Once a certain length of silence is detected, the recognition process ends, and the system outputs the final recognition result of the current speech segment and performs subsequent processing. However, in actual application, the intermediate result is only the local optimal result at the current time, and the intermediate result at the next time is not a simple addition of the intermediate result at the previous time, and the displayed earlier recognition result may also be modified. This results in that the real-time recognition result seen by the user is not coherent and jumps, and the user experience is poor. In addition, the unified post-processing after the recognition ends will cause the increase of processing data volume, computation and delay. SUMMARY

[0004] Therefore, the embodiments of the present application aim to provide a long speech streaming output method, system and medium, to solve the technical problems that in the prior art, the display result jumps and is not coherent when the long speech is recognized in real time, and the data processing volume is large and the delay is high in the post-processing process, resulting in poor user experience.

[0005] In a first aspect, the present application provides a streaming output method of a long speech streaming output system, the long speech streaming output system comprising functional modules and an output module which perform functional processing steps in a preset order, the functional modules comprising a first functional unit which performs the functional processing steps first and a second functional unit which performs the functional processing steps last, the second functional unit being connected with the output module, and the streaming output method comprising the following steps:

[0006] obtaining a first target speech segment and prior results of all functional modules under a previous speech segment, the prior results of each module comprising a prior fixed part and a prior temporary part;

[0007] splicing the recognition text corresponding to the first target speech segment and the prior temporary part of the first functional unit, performing recognition result updating on the spliced part, and obtaining a first target fixed part and a first target temporary part corresponding to the first functional unit under the first target speech segment;

[0008] splicing the first target fixed part corresponding to the first function unit under the first target speech segment and the previous temporary part corresponding to the next function unit under the previous speech segment to obtain the first target fixed part and the first target temporary part corresponding to the next function unit under the first target speech segment, and repeating the step until the first target fixed part and the first target temporary part corresponding to the second function unit under the first target speech segment are obtained;

[0009] outputting, by the output module, a first on-screen display result, the first on-screen display result including the splicing result of the first target fixed part corresponding to the second function unit and the first target temporary part of all function modules.

[0010] Optionally, the streaming output method further includes the following steps:

[0011] obtaining a second target speech segment after the first target speech segment;

[0012] when it is determined that the second target speech segment has a first silence duration, and the first silence duration is greater than a first preset duration and less than a second preset duration, splicing the first target temporary part corresponding to the first function unit under the first target speech segment and the recognized text corresponding to the second target speech segment, updating the recognition result of the spliced part, obtaining an updated result, and taking the updated result as the fixed part of the first function unit;

[0013] Optionally, when it is determined that the second target speech segment has a first silence duration, and the first silence duration is greater than a first preset duration and less than a second preset duration, the second target fixed part and the second target temporary part are provided for the function units other than the first function unit under the second target speech segment.

[0014] Optionally, outputting, by the output module, a second on-screen display result, the second on-screen display result including the splicing result of the second target fixed part corresponding to the second function unit and the second target temporary part of all function modules.

[0015] Optionally, the streaming output method further includes the following steps:

[0016] obtaining a third target speech segment after the second target speech segment, and performing function processing on the third target speech segment to obtain a third target fixed part and a third target temporary part corresponding to the first function unit under the third target speech segment;

[0017] splicing the third target fixed part corresponding to the first function unit under the third target voice segment and the second target temporary part corresponding to the next function unit under the second target voice segment to obtain a third target fixed part and a third target temporary part corresponding to the next function unit under the third target voice segment, and repeating the step until a third target fixed part and a third target temporary part corresponding to the second function unit under the third target voice segment are obtained;

[0018] outputting, by the output module, a third on-screen display result, the third on-screen display result including the splicing result of the third target fixed part corresponding to the second function unit and the third target temporary part of all function modules.

[0019] Optionally, the streaming output method further includes the following steps:

[0020] obtaining a fourth target voice segment;

[0021] when it is determined that the second silence duration exists in the fourth target voice segment and the second silence duration is greater than the second preset duration, regarding the temporary part of all function units as a fixed part, and outputting, by the output module, the fixed part of the second function unit.

[0022] Optionally, the streaming output method further includes the following steps:

[0023] obtaining an end identifier corresponding to each of the plurality of function units;

[0024] regarding the temporary part of the function unit corresponding to the end identifier as a fixed part after the end identifier is determined.

[0025] Optionally, the streaming output method further includes the following steps:

[0026] obtaining a fifth target voice segment;

[0027] after it is determined that the fifth target voice segment is a sentence initial voice segment, performing function processing on the fifth target voice segment to obtain a fifth target fixed part and a fifth target temporary part corresponding to the first function unit under the fifth target voice segment;

[0028] performing function processing on the fifth target fixed part corresponding to the first function unit under the fifth target voice segment to obtain a fifth target fixed part and a fifth target temporary part corresponding to the next function unit after the first function unit under the fifth target voice segment, and repeating the step until a fifth target fixed part and a fifth target temporary part corresponding to the second function unit under the fifth target voice segment are obtained;

[0029] The output module outputs a third on-screen display result, the third on-screen display result comprising a splicing result of the fifth target fixed part corresponding to the second functional unit and the fifth target temporary part of all functional modules.

[0030] Optionally, the plurality of functional units comprises an identification unit, a punctuation unit, a sensitive word unit, a smoothing unit and a reverse text regularization unit.

[0031] Optionally, the first functional unit is the identification unit, and the second functional unit is the sensitive word unit.

[0032] Optionally, the fixed part and the temporary part in the on-screen display result output by the output module are displayed in different colors or different forms.

[0033] In a second aspect, a long speech streaming output system is provided, comprising functional modules performing functional processing steps in a preset order and an output module, the functional modules comprising a first functional unit performing a functional processing step first and a second functional unit performing a functional processing step last, the second functional unit being connected to the output module, the long speech streaming output system further comprising:

[0034] The first functional unit is configured to splice the identification text corresponding to the first target speech segment and a previous temporary part under the first functional unit in a previous speech segment, and perform identification result updating processing on the spliced part, to obtain a first target fixed part and a first target temporary part corresponding to the first functional unit under the first target speech segment.

[0035] The second functional unit is configured to splice the first target fixed part of the previous functional unit and a previous temporary part under the second functional unit in a previous speech segment, and perform processing, to obtain a first target fixed part and a first target temporary part corresponding to the second functional unit under the first target speech segment.

[0036] The output module is configured to perform on-screen display on a splicing result of the first target fixed part corresponding to the second functional unit and the first target temporary part of all functional modules, to obtain an on-screen display result.

[0037] In a third aspect, an apparatus is provided, comprising a processor, a memory, a communication interface and a communication bus, the processor, the memory and the communication interface completing communication with each other through the communication bus.

[0038] The memory is used to store at least one executable instruction, the executable instruction causing the processor to perform operations corresponding to the streaming output method as described above.

[0039] In a fourth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. When the computer program is executed by a processor, the steps of the stream output method described above are implemented.

[0040] According to the first aspect of the present application, by dividing the processing results of all functional modules under the first target voice segment into a first target fixed part and a first target temporary part, and outputting the on-screen display result by using the output module, the fixed part of the on-screen display result is the first target fixed part corresponding to the second functional unit, and the temporary part of the on-screen display result is the spliced result after splicing the first target temporary parts of all functional modules. The fixed part of the on-screen display result will not change due to the increase of the audio stream, thereby solving the problem of the jump and incoherence of the on-screen display result in the prior art when recognizing long voice in real time. In addition, in the technical solution of the present application, each functional module starts processing work at the beginning of the voice, thereby solving the problems of large data processing amount, large calculation amount and high delay caused by unified post-processing after recognition in the prior art.

[0041] Further, by setting two preset time lengths, comparing the silence time length with the first preset time length and the second preset time length respectively, and obtaining different situations for next step processing according to different steps, the problems of low recognition accuracy, incoherent post-processing and poor user experience caused by the delay of the display result due to the setting of only one silence time length in the prior art are solved.

[0042] The above description is only a summary of the technical solution of the present application. In order to more clearly understand the technical means of the present application and to implement the content of the description, the following will describe the preferred embodiments of the present application in detail. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 A schematic flow chart of the stream output of the first target voice segment according to one embodiment of the present application is shown;

[0044] Figure 2 A schematic diagram of the stream processing according to one embodiment of the present application is shown;

[0045] Figure 3 A schematic flow chart of the stream output of the voice segment at the beginning of the sentence according to one embodiment of the present application is shown;

[0046] Figure 4 A schematic flow chart of the stream output of the second target voice segment and the third target voice segment according to another embodiment of the present application is shown;

[0047] Figure 5 A schematic diagram of the stream processing according to another embodiment of the present application is shown;

[0048] Figure 6 A schematic flow chart of streaming of a fourth target speech segment according to another embodiment of the application is shown;

[0049] Figure 7 A schematic diagram of streaming according to another embodiment of the application is shown;

[0050] Figure 8 A schematic diagram of different silence durations of streaming according to an embodiment of the application is shown;

[0051] Figure 9 A block diagram of a long speech streaming system according to an embodiment of the application is shown;

[0052] Figure 10 A block diagram of a long speech streaming device according to an embodiment of the application is shown. DETAILED DESCRIPTION

[0053] In order to make the above objectives, features and advantages of the application more clear and comprehensible, specific embodiments of the application will be described below in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only intended to explain the application and not to limit the application. In addition, it should be noted that only the parts related to the application are shown in the drawings for the purpose of description. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts fall within the scope of the present application.

[0054] The terms "comprising" and "having" and any variations thereof in the present application are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed or optionally further includes other steps or units inherent to these processes, methods, products or devices.

[0055] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase that the phrase in various places in the specification are not necessarily all referring to the same embodiment, or are necessarily referring to different or alternative embodiments. It is explicitly contemplated that embodiments described herein can be combined with each other.

[0056] Figure 1 A schematic flow chart of streaming of a first target speech segment according to an embodiment of the application is shown. Figure 2 A schematic diagram of streaming according to an embodiment of the application is shown. AsFigure 1 As shown, the voice streaming method comprises:

[0057] In step S110, a first target voice segment and prior results of all functional modules under a previous voice segment are obtained, and the prior results of each module comprise a prior fixed part and a prior temporary part.

[0058] In some embodiments, the plurality of functional units of the functional module may, for example, comprise an identification unit, a punctuation unit, a sensitive word unit, a smoothing unit, an inverse text regularization unit, a translation unit, a segmentation unit, and a timestamp prediction unit. The following is described by way of example with the plurality of functional units comprising an identification unit, a punctuation unit, and a sensitive word unit.

[0059] In one embodiment, the first functional unit is an identification unit, and the second functional unit is a sensitive word unit. In order to achieve better results when processing the audio segment, some functional units in the functional module need to be processed in a predetermined order. For example, the first functional unit is an identification unit, which is used to perform text identification processing on the obtained audio segment first, so as to facilitate the next post-processing operation. The next functional unit of the first functional unit is a punctuation unit, which is used to avoid that two characters across punctuation are incorrectly processed. The second functional unit connected with the output module is a sensitive word unit, which can avoid the appearance of sensitive words when screening. In actual application, the processing order of each processing unit can be adjusted appropriately according to the functions implemented by the specific processing unit.

[0060] In step S120, the recognized text corresponding to the first target voice segment and the prior temporary part of the first functional unit are spliced, and the spliced part is subjected to recognition result updating to obtain a first target fixed part and a first target temporary part corresponding to the first functional unit under the first target voice segment.

[0061] In combination with Figure 2 , the first target fixed part and the first target temporary part corresponding to the first functional unit under the first target voice segment are determined. For example, Figure 2As shown, the speech segment between t1 and t2 represents the first target speech segment, and the speech segment before t1 represents the previous speech segment. The recognized text corresponding to the speech segment between t1 and t2 is spliced with the temporary part AU1 of the recognized result of the speech segment before t1, the spliced part is updated to obtain the fixed part AF2 and the temporary part AU2 of the recognized result of the speech segment between t1 and t2. The specific process of the recognition result update is that after the first target speech segment t1-t2 is obtained, the recognition unit recognizes the current time, and the temporary part AU1 of the recognized result of the speech segment before t1 may change, so the recognized result after the fixed result AF1 of the speech segment before t1 needs to be updated to obtain a better recognized result. The fixed part AF2 and the temporary part AU2 of the current time recognition result are determined, that is, AF2 represents the first target fixed part corresponding to the first functional unit under the first target speech segment. AU2 represents the first target temporary part corresponding to the first functional unit under the first target speech segment.

[0062] In step S130, the first target fixed part corresponding to the first functional unit under the first target speech segment and the previous temporary part corresponding to the next functional unit under the previous speech segment are spliced to obtain the first target fixed part and the first target temporary part corresponding to the next functional unit under the first target speech segment, and the step is repeated until the first target fixed part and the first target temporary part corresponding to the second functional unit under the first target speech segment are obtained.

[0063] In combination Figure 2 , the first target fixed part and the first target temporary part corresponding to the second functional unit under the first target speech segment are determined. As shown in Figure 2 AF2 and the previous temporary part BU1 corresponding to the punctuation unit under the speech segment before t1 are spliced, and the spliced result is punctuated by the punctuation unit to obtain the fixed part BF2 and the temporary part BU2 of the processing result corresponding to the punctuation unit under the speech segment between t1 and t2. BF2 represents the first target fixed part corresponding to the next functional unit under the first target speech segment. BU2 represents the first target temporary part corresponding to the next functional unit under the first target speech segment. BF2 and the previous temporary part CU1 corresponding to the sensitive word unit under the speech segment before t1 are spliced, and the spliced result is filtered by the sensitive word unit to obtain the fixed part CF2 and the temporary part CU2 corresponding to the sensitive word unit under the speech segment between t1 and t2. CF2 represents the first target fixed part corresponding to the second functional unit under the first target speech segment. CU2 represents the first target temporary part corresponding to the second functional unit under the first target speech segment.

[0064] In step S140, the output module outputs the first on-screen display result, which includes the first target fixed part corresponding to the second function unit and the spliced result of the first target temporary part of all function modules.

[0065] In some embodiments, the fixed part and the temporary part in the on-screen display result output by the output module are displayed in different colors or different forms.

[0066] In combination Figure 2 , the first on-screen display result is determined. As shown in Figure 2 , CF2 is taken as the fixed part OF2 of the output result of the output module for the speech segment between t1 and t2 。 AU2, BU2 and CU2 are spliced, and the spliced result is sent to the sensitive word unit for sensitive word filtering processing, and the obtained processing result is taken as the temporary part OU2 of the output result of the output module for the speech segment between t1 and t2. OF2 represents the fixed part of the first on-screen display result. OU2 represents the temporary part of the first on-screen display result.

[0067] According to the present embodiment, by dividing the processing results of all function modules under the first target speech segment into the first target fixed part and the first target temporary part, the output module outputs the on-screen display result, wherein the fixed part of the on-screen display result is the first target fixed part corresponding to the second function unit, and the temporary part of the on-screen display result is the spliced result of the first target temporary part of all function modules. The fixed part of the on-screen display result will not change due to the increase of the audio stream, thereby solving the problem of the on-screen display result jumping and being incoherent in real-time recognition of long speech in the prior art. In addition, in the present technical solution, the fixed result and the temporary result are displayed on the screen in different colors or different forms, and the user can clearly observe the different colors or different forms of the text on the screen. For example, the fixed part of the on-screen result is displayed in black, and the temporary part of the on-screen result is displayed in red, and the user can choose to read after waiting for the red part to change to black, or read both parts at the same time according to personal reading habits. The present application uses the above on-screen display method, and the user can view the on-screen result after audio processing more clearly and conveniently. In addition, in the present technical solution, each function module starts processing work at the beginning of the speech, thereby solving the problems of large data processing amount, large calculation amount and high delay caused by unified post-processing after recognition in the prior art.

[0068] In other embodiments, the speech segment before t1 is subjected to sentence head detection, and after it is determined to be a sentence head speech segment, the speech segment is subjected to function processing. Figure 3 A schematic flow chart of the sentence head speech segment stream output of one embodiment of the present application is shown, which includes the following steps:

[0069] Step S210, a fifth target speech segment is obtained.

[0070] In combination with Figure 2 , the fifth target speech segment is determined. The fifth target speech segment is any speech segment detected as a beginning-of-sentence speech segment. As shown in Figure 2 , the speech segment before t1 represents the fifth target speech segment.

[0071] Step S220, after determining that the fifth target speech segment is a beginning-of-sentence speech segment, the fifth target speech segment is functionally processed to obtain a fifth target fixed part and a fifth target temporary part corresponding to a first functional unit under the fifth target speech segment.

[0072] In combination with Figure 2 , the fifth target fixed part and the fifth target temporary part corresponding to the first functional unit under the fifth target speech segment are determined. As shown in Figure 2 , the speech segment before t1 is recognized and text-processed to obtain a fixed part AF1 and a temporary part AU1 of the recognition result corresponding to a recognized unit under the segment. AF1 represents the fifth target fixed part corresponding to the first functional unit under the fifth target speech segment. AU1 represents the fifth target temporary part corresponding to the first functional unit under the fifth target speech segment.

[0073] Step S230, the fifth target fixed part corresponding to the first functional unit under the fifth target speech segment is functionally processed to obtain a fifth target fixed part and a fifth target temporary part corresponding to a next functional unit after the first functional unit under the fifth target speech segment, and the step is repeated until a fifth target fixed part and a fifth target temporary part corresponding to a second functional unit under the fifth target speech segment are obtained.

[0074] In combination with Figure 2 , the fifth target fixed part and the fifth target temporary part corresponding to the second functional unit under the fifth target speech segment are determined. As shown in Figure 2 , AF1 is sent to a punctuation unit for punctuation processing to obtain a fixed part BF1 and a temporary part BU1 of the processing result corresponding to the punctuation unit under the speech segment before t1. BF1 represents the fifth target fixed part corresponding to the next functional unit after the first functional unit under the fifth target speech segment. BU1 represents the fifth target temporary part corresponding to the next functional unit after the first functional unit under the fifth target speech segment. BF1 is sent to a sensitive word unit for sensitive word filtering processing to obtain a fixed part CF1 and a temporary part CU1 of the processing result corresponding to the sensitive word unit under the speech segment before t1. CF1 represents the fifth target fixed part corresponding to the second functional unit under the fifth target speech segment. CU1 represents the fifth target temporary part corresponding to the second functional unit under the fifth target speech segment.

[0075] In step S240, the third on-screen display result is output by the output module. The third on-screen display result includes the fifth target fixed part corresponding to the second function unit and the spliced result of the fifth target temporary part of all function modules.

[0076] In combination Figure 2 , the third on-screen display result is determined. As shown in Figure 2 , CF1 is taken as the fixed part OF1 of the output result of the output module before the speech segment t1. AU1, BU1 and CU1 are spliced, and the spliced result is sent to the sensitive word unit for sensitive word filtering processing. The obtained processing result is taken as the temporary part OU1 of the output result of the output module before the speech segment t1. OF1 represents the fixed part of the third on-screen display result. OU1 represents the temporary part of the third on-screen display result.

[0077] Figure 4 A schematic flow chart of streaming of the second target speech segment and the third target speech segment according to another embodiment of the present application is shown. Figure 5 A schematic diagram of another embodiment of the streaming of the present application is shown. As shown in Figure 4 , the speech streaming method includes:

[0078] In step S310, the second target speech segment after the first target speech segment is obtained.

[0079] In combination Figure 5 , the second target speech segment is determined. As shown in Figure 5 , the speech segment between t2 and t3 represents the second target speech segment after the first target speech segment.

[0080] In step S320, when it is determined that the second target speech segment has a first silence duration, and the first silence duration is greater than a first preset duration and less than a second preset duration, the first target temporary part of the first function unit corresponding to the first target speech segment and the recognized text corresponding to the second target speech segment are spliced, the spliced part is updated, the updated result is obtained, and the updated result is taken as the fixed part of the first function unit.

[0081] In combination Figure 5 , the fixed part of the first function unit is determined. As shown in Figure 5As shown, there is a silence duration between t2 and t3, which is represented as a first silence duration existing in the second target speech segment, and the first silence duration is greater than the first preset duration and less than the second preset duration. In some embodiments, the first preset duration is set to 300-500 ms, and the second preset duration is set to 800-2000 ms. When it is detected that there is a first silence duration in the speech segment, the first target temporary part AU2 corresponding to the speech segment between t1 and t2 and the recognition text corresponding to the speech segment between t2 and t3 are spliced by the recognition unit, the spliced part is updated, and the updated result AF3 represents the fixed part of the first function unit. At this time, since the recognition is completed, all recognition results corresponding to the audio are fixed, and therefore the first function unit has no temporary part.

[0082] Step S330, when it is determined that the second target speech segment has a first silence duration, and the first silence duration is greater than the first preset duration and less than the second preset duration, the second target fixed part and the second target temporary part of the function unit other than the first function unit under the second target speech segment.

[0083] In combination Figure 5 , the second target fixed part and the second target temporary part of the function unit other than the first function unit under the second target speech segment are determined. As Figure 5 shown, AF3 and the previous temporary part BU2 corresponding to the speech segment between t1 and t2 under the punctuation unit are spliced, the spliced result is punctuated by the punctuation unit, and the fixed part BF3 and the temporary part BU3 of the punctuation unit corresponding to the speech segment between t2 and t3 are obtained. BF3 represents the second target fixed part of the punctuation unit under the second target speech segment. BU3 represents the second target temporary part of the punctuation unit under the second target speech segment.

[0084] BF3 and the previous temporary part CU2 corresponding to the speech segment between t1 and t2 under the sensitive word unit are spliced, the spliced result is filtered by the sensitive word unit, and the fixed part CF3 and the temporary part CU3 of the sensitive word unit corresponding to the speech segment between t2 and t3 are obtained. CF3 represents the second target fixed part of the second function unit under the second target speech segment. CU3 represents the second target temporary part of the second function unit under the second target speech segment.

[0085] Step S340, outputting the second on-screen display result by using the output module, the second on-screen display result including the second target fixed part corresponding to the second function unit and the spliced result of the second target temporary part of all function modules.

[0086] In combination Figure 5, determine the second on-screen display result. As shown in Figure 5 , CF3 is taken as the fixed part OF3 of the output result of the voice segment under t2 to t3 of the output module. CU3 and BU3 are spliced, and the spliced result is sent to the sensitive word unit for sensitive word filtering processing, and the obtained processing result is taken as the temporary part OU3 of the output result of the voice segment under t2 to t3 of the output module. OF3 represents the fixed part of the second on-screen display result. OU3 represents the temporary part of the second on-screen display result.

[0087] Step S350, obtain the third target voice segment after the second target voice segment, and perform function processing on the third target voice segment to obtain the third target fixed part and the third target temporary part corresponding to the first function unit of the third target voice segment.

[0088] In combination with Figure 5 , determine the third target fixed part and the third target temporary part corresponding to the first function unit of the third target voice segment. As shown in Figure 5 , the voice segment between t3 and t1' represents the third target voice segment after the second target voice segment. The voice segment is recognized to obtain the fixed part AF1' and the temporary part AU1' corresponding to the recognition unit of the voice segment between t3 and t1'. AF1' represents the third target fixed part corresponding to the first function unit of the third target voice segment. AU1' represents the third target temporary part corresponding to the first function unit of the third target voice segment.

[0089] Step S360, splice the third target fixed part corresponding to the first function unit of the third target voice segment and the second target temporary part corresponding to the next function unit under the second target voice segment to obtain the third target fixed part and the third target temporary part corresponding to the next function unit under the third target voice segment. Repeat the step until the third target fixed part and the third target temporary part corresponding to the second function unit under the third target voice segment are obtained.

[0090] In combination with Figure 5 , determine the third target fixed part and the third target temporary part corresponding to the second function unit of the third target voice segment. As shown in Figure 5As shown, the AF1' and the punctuation unit are spliced with the previous temporary part BU3 corresponding to the speech segment between t2 and t3, the spliced result is punctuated by the punctuation unit, and the fixed part BF1' and the temporary part BU1' corresponding to the punctuation unit under the speech segment between t3 and t1' are obtained. BF1' represents the third target fixed part corresponding to the next functional unit under the third target speech segment. BU1' represents the third target temporary part corresponding to the next functional unit under the third target speech segment. The BF1' and the sensitive word unit are spliced with the previous temporary part CU3 corresponding to the speech segment between t2 and t3, the spliced result is sensitive word filtered by the sensitive word unit, and the fixed part CF1' and the temporary part CU1' corresponding to the sensitive word unit under the speech segment between t3 and t1' are obtained. CF1' represents the third target fixed part corresponding to the second functional unit under the third target speech segment. CU1' represents the third target temporary part corresponding to the second functional unit under the third target speech segment.

[0091] In step S370, the third on-screen display result is output by using the output module. The third on-screen display result includes the third target fixed part corresponding to the second functional unit and the spliced result of the third target temporary part of all functional modules.

[0092] In combination Figure 5 , the third on-screen display result is determined. As shown in Figure 6 , the CF1' is taken as the fixed part OF1' of the output result of the output module under the speech segment between t3 and t1'. AU1', BU1' and CU1' are spliced, and the spliced result is sent to the sensitive word unit for sensitive word filtering processing. The obtained processing result is taken as the temporary part OU1' of the output result of the output module under the speech segment between t3 and t1'. OF1' represents the fixed part of the third on-screen display result. OU1' represents the temporary part of the third on-screen display result.

[0093] Compared with the above embodiment, only the first silence duration is added in the embodiment, and other steps of the above embodiment are still applicable to the embodiment, and will not be described in detail here.

[0094] Figure 7 A schematic flow chart of streaming output of a fourth target speech segment according to still another embodiment of the application is shown. Figure 6 A schematic diagram of streaming processing according to still another embodiment of the application is shown. As shown in Figure 7 , the speech streaming output method includes:

[0095] In step S410, the fourth target speech segment is obtained.

[0096] In combination Figure 7determines a fourth target voice segment. The fourth target voice segment is any voice segment with a second silence duration. As shown in Figure 7 the voice segment between t2 and t4 represents the fourth target voice segment.

[0097] In step S420, when it is determined that the fourth target voice segment has the second silence duration and the second silence duration is greater than the second preset duration, the temporary part of all functional units is regarded as the fixed part, and the fixed part of the second functional unit is output by the output module.

[0098] Compared with the above embodiment, the second silence duration is added in the embodiment, and other steps of the above embodiment are still applicable to the embodiment, and thus will not be described in detail.

[0099] In combination with Figure 7 , the fixed part of the output module is determined. As shown in Figure 8 , the voice segment between t2 and t4 has a silence duration, which represents the second silence duration of the fourth target voice segment, and the second silence duration is greater than the second preset duration. In some embodiments, the second preset duration is set to 800-2000 ms. At this time, the temporary part BU3 of the punctuation unit is regarded as the fixed part BF4 of the punctuation unit under the voice segment between t2 and t4. The temporary part CU3 of the sensitive word unit and the temporary part BU3 of the punctuation unit are spliced to obtain the temporary part of the sensitive word unit under the voice segment between t2 and t4, and the temporary part is regarded as the fixed part CF4 of the sensitive word unit under the voice segment between t2 and t4. The fixed part CF4 of the sensitive word unit under the voice segment between t2 and t4 is regarded as the fixed part OF4 of the output module under the fourth target voice segment, and the OF4 is displayed on the screen. Under the fourth target voice segment, the functional module and the output module do not have temporary parts.

[0100] In other embodiments, a plurality of ending identifiers corresponding to the functional units can be obtained, and the temporary part of the functional unit corresponding to the ending identifier is regarded as the fixed part after the ending identifier is determined. For example, the punctuation module detects the end of the recognized text, and when it is detected that the recognized text is the end, the ending identifier corresponding to the punctuation module is obtained, and the temporary result of the punctuation module under the voice segment is regarded as the fixed result. At this time, the post-processing of the punctuation module is completed, and there is no temporary result.

[0101] Figure 8 A schematic diagram of different silence durations of the stream output of the embodiment of the application is shown.

[0102] As shown in Figure 9As shown, when a first target speech segment with a first silence duration is detected, the processing of the recognition unit in the processing module ends, and the processing of the other post-processing units in the processing module has not ended. When another speech segment with a second silence duration is detected, the processing of the other post-processing units in the processing module ends, and the output module outputs the on-screen display result.

[0103] According to the above embodiment, by setting two preset durations, comparing the silence duration with the first preset duration and the second preset duration respectively, and processing the different cases according to different steps, the problems of low recognition accuracy, incoherent post-processing, and poor user experience caused by the delay of the display result due to the setting of only one silence duration in the prior art are solved.

[0104] The embodiment of the present application also provides a long speech streaming output system, which comprises Figure 1 As shown, the long speech streaming output system comprises a function module 101 and an output module 102, which perform function processing steps in a preset order. The function module 101 is configured to perform function processing on speech segments. The output module 102 is configured to perform on-screen display on the spliced result of the first target fixed part corresponding to the second function unit and the first target temporary part of all function modules, to obtain an on-screen display result.

[0105] In some optional embodiments, the function module 101 comprises a first function unit which performs the function processing steps first and a second function unit which performs the function processing steps last and is connected to the output module. The first function unit is configured to splice the first target speech segment and the previous temporary part of the first function unit under the previous speech segment, to obtain the first target fixed part and the first target temporary part of the first function unit corresponding to the first target speech segment. The second function unit is configured to splice and process the first target fixed part of the previous function unit and the previous temporary part of the second function unit under the previous speech segment, to obtain the first target fixed part and the first target temporary part of the second function unit corresponding to the first target speech segment.

[0106] The embodiment of the present application also provides a long speech streaming output device, which comprises a processor 201, a memory 202, and a computer program stored in the memory 202 and configured to be executed by the processor 201, and the processor 201 implements the long speech streaming output method of any of the above embodiments when executing the computer program.

[0107] The processor 201 implements the steps in the above long speech streaming output method embodiments when executing the computer program, for example Figure 9All the steps of the long speech streaming method are shown. Alternatively, the processor 201 implements the functions of the modules / units in the long speech streaming system described above when executing the computer program, for example ​ The functions of the modules of the long speech streaming system are shown.

[0108] By way of example, the computer program can be segmented into one or more modules, one or more modules are stored in the memory 202 and executed by the processor 201 to complete the present application. One or more modules can be a series of computer program instructions capable of completing a specific function, which is used to describe the execution process of the computer program in the long speech streaming system.

[0109] The processor 201 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The processor 201 is the control center of the long speech streaming system, and connects all parts of the long speech streaming system through various interfaces and lines.

[0110] The memory 202 can be used to store computer programs and / or modules, and the processor 201 realizes various functions of the long speech streaming system by running or executing the computer programs and / or modules stored in the memory 202, and calling the data stored in the memory 202. The memory 202 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required by a function, etc.; the data storage area can store data created according to the use of the long speech streaming system, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device.

[0111] The modules / units of the long speech streaming output system can be stored in a computer readable storage medium if they are implemented in the form of software function units and sold or used as independent products. Based on this understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms, etc. The computer readable medium can include any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0112] The embodiments of the present application also provide a computer readable storage medium, and the method according to the embodiments of the present application can be implemented in hardware, firmware or recorded in a storage medium, or be implemented by computer code originally stored in a remote storage medium or a non-transitory machine readable storage medium and to be stored in a local storage medium through network downloading, so that the method described herein can be processed by such software on a storage medium using a general computer, a special processor or programmable or special hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk or a solid state disk, etc. Further, the storage medium can also include a combination of the above-mentioned types of memories. It can be understood that the computer, the processor, the microprocessor controller or the programmable hardware include storage components that can store or receive software or computer code, when the software or computer code is accessed and executed by the computer, the processor or the hardware, the method shown in the above embodiments is implemented.

[0113] The above-mentioned embodiments only express several embodiments of the present application, and the description is more specific and detailed, but it cannot be understood as a limitation on the scope of the patent. It should be noted that for ordinary skilled persons in the art, without departing from the concept of the present application, several modifications and improvements can be made, which are within the scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A streaming method of a long speech streaming system, characterized by, The long speech streaming output system comprises functional modules and an output module, which perform functional processing steps in a preset order. The functional modules comprise a first functional unit, which performs the first functional processing step, and a second functional unit, which performs the last functional processing step. The second functional unit is connected to the output module. The streaming output method comprises the following steps: obtaining a first target speech segment and a prior result of all functional modules under a previous speech segment, wherein the prior result of each module comprises a prior fixed part and a prior temporary part; splicing the recognition text corresponding to the first target speech segment and the prior temporary part of the first functional unit, updating the recognition result of the spliced part, and obtaining a first target fixed part and a first target temporary part corresponding to the first functional unit under the first target speech segment; splicing the first target fixed part corresponding to the first functional unit under the first target speech segment and the prior temporary part corresponding to the next functional unit under the previous speech segment, obtaining a first target fixed part and a first target temporary part corresponding to the next functional unit under the first target speech segment, and repeating the step until a first target fixed part and a first target temporary part corresponding to the second functional unit under the first target speech segment are obtained; outputting a first on-screen display result using the output module, wherein the first on-screen display result comprises the first target fixed part corresponding to the second functional unit and the spliced result of the first target temporary part of all functional modules.

2. The streaming-out method according to claim 1, wherein Further comprising the following steps: obtaining a second target speech segment after the first target speech segment; when it is determined that the second target speech segment has a first silence duration, and the first silence duration is greater than a first preset duration and less than a second preset duration, splicing the first target temporary part corresponding to the first functional unit under the first target speech segment and the recognition text corresponding to the second target speech segment, updating the recognition result of the spliced part, obtaining an updated result, and taking the updated result as the fixed part of the first functional unit; Optionally, when it is determined that the second target speech segment has a first silence duration, and the first silence duration is greater than a first preset duration and less than a second preset duration, all functional units except the first functional unit have a second target fixed part and a second target temporary part under the second target speech segment; Optionally, outputting a second on-screen display result using the output module, wherein the second on-screen display result comprises the second target fixed part corresponding to the second functional unit and the spliced result of the second target temporary part of all functional modules.

3. The streaming-out method according to claim 2, wherein Further comprising the following steps: obtaining a third target speech segment after the second target speech segment, and performing functional processing on the third target speech segment to obtain a third target fixed part and a third target temporary part corresponding to the first functional unit under the third target speech segment; splicing the third target fixed part corresponding to the first function unit under the third target voice segment and the second target temporary part corresponding to the next function unit under the second target voice segment to obtain the third target fixed part and the third target temporary part corresponding to the next function unit under the third target voice segment, and repeating the step until the third target fixed part and the third target temporary part corresponding to the second function unit under the third target voice segment are obtained; outputting, by the output module, a third on-screen display result, the third on-screen display result comprising the splicing result of the third target fixed part corresponding to the second function unit and the third target temporary part of all function modules.

4. The streaming-out method according to claim 2 or 3, characterized by, Further comprising the following steps: obtaining a fourth target voice segment; when it is determined that the second silence duration exists in the fourth target voice segment and the second silence duration is greater than the second preset duration, regarding the temporary part of all function units as the fixed part, and outputting, by the output module, the fixed part of the second function unit.

5. The streaming-out method according to claim 2 or 3, wherein Further comprising the following steps: obtaining an end identifier corresponding to each of the plurality of function units; after determining the end identifier, regarding the temporary part of the function unit corresponding to the end identifier as the fixed part.

6. The streaming-out method according to any one of claims 1 to 3, wherein, Further comprising the following steps: obtaining a fifth target voice segment; after determining that the fifth target voice segment is a sentence initial voice segment, performing function processing on the fifth target voice segment to obtain a fifth target fixed part and a fifth target temporary part corresponding to the first function unit under the fifth target voice segment; performing function processing on the fifth target fixed part corresponding to the first function unit under the fifth target voice segment to obtain a fifth target fixed part and a fifth target temporary part corresponding to the next function unit after the first function unit under the fifth target voice segment, and repeating the step until the fifth target fixed part and the fifth target temporary part corresponding to the second function unit under the fifth target voice segment are obtained; outputting, by the output module, a third on-screen display result, the third on-screen display result comprising the splicing result of the fifth target fixed part corresponding to the second function unit and the fifth target temporary part of all function modules; Optionally, the plurality of function units comprise an identification unit, a punctuation unit, a sensitive word unit, a smoothing unit and a reverse text regularization unit. Optionally, the first function unit is the identification unit, and the second function unit is the sensitive word unit.

7. The streaming-out method according to any one of claims 1 to 3, wherein, The fixed part and the temporary part in the on-screen display result output by the output module are displayed in different colors or different forms.

8. A long speech streaming system, characterized by, The long voice streaming output system comprises a function module for performing function processing steps in a preset order and an output module, the function module comprises a first function unit for performing the function processing steps first and a second function unit for performing the function processing steps last, the second function unit is connected with the output module, and the long voice streaming output system further comprises: The first function unit is configured to splice the recognition text corresponding to the first target voice segment and the previous temporary part under the first function unit in the previous voice segment, and perform recognition result updating processing on the spliced part, to obtain the first target fixed part and the first target temporary part corresponding to the first function unit under the first target voice segment. The second function unit is configured to splice the first target fixed part of the previous function unit and the previous temporary part under the second function unit in the previous voice segment, and perform processing, to obtain the first target fixed part and the first target temporary part corresponding to the second function unit under the first target voice segment. The output module is configured to splice the first target fixed part corresponding to the second function unit and the first target temporary part of all function modules, and display the spliced result on the screen, to obtain the on-screen display result.

9. An apparatus, comprising: Comprise: A processor, a memory, a communication interface and a communication bus, the processor, the memory and the communication interface complete mutual communication through the communication bus; The memory is used to store at least one executable instruction, and the executable instruction makes the processor execute the operation corresponding to the streaming output method in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to realize the steps of the streaming output method in any one of claims 1-7.

Images