Method and device for correcting boundary buffer of real-time speech recognition results

Abstract

The present invention relates to a method and device for buffering and correcting speech boundary information in real-time speech recognition results. The method comprises obtaining speech recognition results and identifying speech boundary information in the speech recognition results; buffering speech boundaries within a preset time period to obtain boundary information of the speech boundaries; and when new speech boundary information is buffered, comparing the new speech boundary information with existing speech boundary information and correcting the existing speech boundary information based on the comparison result. By providing buffering to correct speech boundary information, the present invention can improve the fault tolerance of real-time analysis technology based on time boundaries, thereby preventing the execution results of boundary analysis from being affected.

Description

Technical Field

[0001] The present invention belongs to the technical field of speech processing, and in particular relates to a method and device for correcting a boundary buffer of a real-time speech recognition result. Background Art

[0002] In the existing technology, there is a real-time analysis algorithm that performs silence analysis and interruption analysis based on the boundaries of speech recognition results. During the real-time speech recognition process, this algorithm, due to boundary correction and recognition delay, can cause errors and delays in the recognition result boundaries, significantly interfering with the algorithm and resulting in erroneous results.

[0003] In the first case, the recognition boundary is corrected, and the correction boundary may become larger or smaller. When this happens, if the speech is very short, such as only one or two words, the ending boundary may be equal to or even smaller than the starting boundary, resulting in incorrect results.

[0004] During real-time speech recognition, to improve efficiency and user experience, we use a technology that displays the results on the screen in real time, enabling simultaneous recognition while you speak. With this technology, recognition results are returned as the voice stream continues to flow. As speech continues to flow, the recognition engine may revise the returned partial recognition results. This is because new recognition results may have higher confidence than older ones, and higher-confidence recognition results may also correspond to more accurate word boundaries. Therefore, the recognition results may be subject to revision.

[0005] In the second case, recognition latency can cause the recognition boundaries returned by two voice streams in a call to differ by several seconds at the same time. This can cause the later sentence to be sent to the boundary analysis component before the newer one, leading to analysis errors.

[0006] The speech recognition engine uses VAD (Voice Activity Detect) technology for voice activity detection. It is used to quickly identify the start / end position of actual speech in the sound stream, eliminate noise, silence and other non-speech sounds, and avoid wasting recognition resources. Compared with the speech recognition process, VAD executes very quickly, but its accuracy is lower than the boundary detection based on the recognition engine. Because in call center calls, agents often speak much more than customers, the delay in agent voice stream recognition is greater than the delay in customer voice recognition. When the agent's voice is sent to the recognition engine, the delay to receiving the recognition result may be 500ms, but the delay of the customer's voice may be within 100ms, resulting in disordered recognition boundary returns.

[0007] In the above two situations, the speech recognition results are affected by boundary correction and recognition delay, which affects the execution results of boundary analysis. Summary of the Invention

[0008] In view of this, the purpose of the present invention is to overcome the shortcomings of the existing technology and provide a method and device for correcting the boundary buffer of real-time speech recognition results to solve the problem in the existing technology that the speech recognition results are affected by boundary correction and recognition delay, which affects the execution results of boundary analysis.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: a method for correcting the boundary buffer of real-time speech recognition results, comprising:

[0010] Acquire a speech recognition result, and identify speech boundary information in the speech recognition result;

[0011] Using a historical boundary buffer to perform buffering processing on speech boundaries within a preset time period to obtain boundary information of the speech boundaries;

[0012] When new speech boundary information is buffered, the new speech boundary information is compared with the existing speech boundary information, and the boundary information of the existing speech boundary is modified according to the comparison result.

[0013] Furthermore, the boundary information of the speech boundary includes:

[0014] Time boundary value, the role of the sentence corresponding to the boundary, and the boundary type;

[0015] The boundary type includes a start boundary and an end boundary.

[0016] Furthermore, when new speech boundary information is buffered, the new speech boundary information is compared with the existing speech boundary information, and the boundary information of the existing speech boundary is corrected according to the comparison result, including:

[0017] When a new speech boundary is buffered, determining whether there is a boundary having the same role as the sentence corresponding to the new speech boundary;

[0018] If so, the time boundary value of the new speech boundary is compared to see whether it is greater than or equal to the time boundary value of the existing speech boundary. If it is less than, it is determined that confusion occurs after the boundary correction, and the existing speech boundary is corrected again.

[0019] Furthermore, the step of correcting the existing speech boundary again includes:

[0020] The time boundary value of the new speech boundary is corrected to the time boundary value of the existing speech boundary.

[0021] Furthermore, the step of correcting the existing speech boundary again further includes:

[0022] After the boundary information of the existing speech boundary is corrected, the new speech boundary information is buffered again.

[0023] Furthermore, the acquiring of speech recognition results and identifying speech boundary information in the speech recognition results include:

[0024] Acquiring acoustic feature information and language feature information of the speech data to be recognized;

[0025] According to the acoustic feature information and language feature information, a target recognition path matching the language data to be recognized is determined in a pre-constructed static word graph, the word node information of each word node in the candidate recognition path of the voice data to be recognized is recorded, and a speech recognition result of the voice data to be recognized is obtained according to the target recognition path. Based on the word identifier of each recognized word in the speech recognition result and the word node information of each word node in the candidate recognition path, backtracking is performed along the target recognition path to obtain the speech boundary information of each recognized word in the speech recognition result in the voice data to be recognized.

[0026] The present application provides a device for correcting a boundary buffer of a real-time speech recognition result, comprising:

[0027] An acquisition module, configured to acquire a speech recognition result and identify speech boundary information in the speech recognition result;

[0028] A processing module, configured to perform buffering processing on speech boundaries within a preset time period using a historical boundary buffer to obtain boundary information of the speech boundaries;

[0029] The correction module is used to compare the new speech boundary information with the existing speech boundary information when there is new speech boundary information to be buffered, and to correct the boundary information of the existing speech boundary according to the comparison result.

[0030] An embodiment of the present application provides a computer device, comprising a processor, and a memory connected to the processor;

[0031] The memory is used to store a computer program, and the computer program is used to execute the method for correcting the boundary buffer of the real-time speech recognition result provided by any of the above embodiments;

[0032] The processor is configured to call and execute a computer program in a memory.

[0033] The present invention adopts the above technical solution, and the beneficial effects that can be achieved include:

[0034] The present invention provides a method and device for buffering and correcting boundaries in real-time speech recognition results. The method comprises obtaining speech recognition results and identifying speech boundary information in the speech recognition results; buffering speech boundaries within a preset time period to obtain boundary information of the speech boundaries; and when new speech boundary information is buffered, comparing the new speech boundary information with existing speech boundary information and correcting the existing speech boundary information based on the comparison result. By providing buffering to correct speech boundary information, the present invention can improve the fault tolerance of real-time analysis techniques based on time boundaries, thereby preventing impacts on the execution results of boundary analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0036] Figure 1 A schematic diagram of the steps of the method for correcting the boundary buffer of real-time speech recognition results according to the present invention;

[0037] Figure 2 This is a schematic diagram of the boundary buffer correction result of the real-time speech recognition result of the present invention;

[0038] Figure 3 This is a schematic diagram of the boundary buffer correction and sorting results of the real-time speech recognition results of the present invention;

[0039] Figure 4 This is a structural diagram of the device for correcting the boundary buffer of real-time speech recognition results according to the present invention. DETAILED DESCRIPTION

[0040] To make the objectives, technical solutions, and advantages of the present invention more apparent, the technical solutions of the present invention will be described in detail below. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other implementations obtained by those of ordinary skill in the art without inventive effort are within the scope of protection of the present invention.

[0041] The following describes a specific method and device for correcting the boundary buffer of real-time speech recognition results provided in an embodiment of the present application in conjunction with the accompanying drawings.

[0042] like Figure 1 As shown, the speech real-time recognition result boundary buffer correction method provided in the embodiment of the present application includes:

[0043] S101, obtaining a speech recognition result, and identifying speech boundary information in the speech recognition result;

[0044] S102, using a historical boundary buffer to perform buffering processing on speech boundaries within a preset time period to obtain boundary information of the speech boundaries;

[0045] S103: When there is new speech boundary information to be buffered, the new speech boundary information is compared with the existing speech boundary information, and the existing speech boundary information is modified according to the comparison result.

[0046] It is understandable that in the present application, obtaining the speech recognition result and identifying the speech boundary information in the speech recognition result can be implemented by a recognition engine, which can be implemented using existing technologies. For example, the specific steps may be:

[0047] Acquiring acoustic feature information and language feature information of the speech data to be recognized;

[0048] Based on the acoustic feature information and language feature information, a target recognition path matching the language data to be recognized is determined in a pre-constructed static word graph, the word node information of each word node in the candidate recognition path of the voice data to be recognized is recorded, and the voice recognition result of the voice data to be recognized is obtained according to the target recognition path. Based on the word identifier of each recognized word in the voice recognition result and the word node information of each word node in the candidate recognition path, backtracking is performed along the target recognition path to obtain the word boundary information of each recognized word in the voice data to be recognized in the voice recognition result.

[0049] The working principle of the real-time speech recognition result boundary buffer correction method is as follows: This application introduces a historical boundary buffer to buffer the speech boundaries returned by the recognition engine over a period of time. When new speech boundary information is buffered, the new speech boundary information is compared with the existing speech boundary information, and the existing speech boundary information is corrected based on the comparison result.

[0050] The preset time can be set according to actual conditions and is not limited in this application. For example, it can be 2s or 3s.

[0051] In some embodiments, such as Figure 2 As shown, the boundary information of the speech boundary includes:

[0052] Time boundary value, the role of the sentence corresponding to the boundary, and the boundary type;

[0053] The boundary type includes a start boundary and an end boundary.

[0054] like Figure 2In which A and B are the roles of the sentence corresponding to the boundary; start and end are boundary types, where start is the start boundary and end is the end boundary; 100, 200, and 300 are time boundary values.

[0055] In some embodiments, when new speech boundary information is buffered, the new speech boundary information is compared with existing speech boundary information, and the existing speech boundary information is modified based on the comparison result, including:

[0056] When a new speech boundary is buffered, determining whether there is a boundary having the same role as the sentence corresponding to the new speech boundary;

[0057] If so, the time boundary value of the new speech boundary is compared to see whether it is greater than or equal to the time boundary value of the existing speech boundary. If it is less than, it is determined that confusion occurs after the boundary correction, and the existing speech boundary is corrected again.

[0058] Specifically, first compare the roles of the sentences in the new speech boundary information with the roles of the sentences in the existing speech boundary information. If they exist, for example, A exists in both, or B exists in both, then it proves that the same boundary exists. At this time, compare whether the time boundary value of the new speech boundary is greater than or equal to the time boundary value of the existing speech boundary. If it is less than, it is determined that confusion has occurred after the boundary correction, and the existing speech boundary is corrected again. Figure 2 A string consists of a triple: role-boundary type-time point corresponding to the boundary. For example, "A-start-100" means that the time point at which role A starts speaking is 100. The boundary type includes the start boundary and the end boundary.

[0059] Preferably, the step of correcting the existing speech boundary again includes:

[0060] The time boundary value of the new speech boundary is corrected to the time boundary value of the existing speech boundary.

[0061] For example, Figure 2 As shown, the time boundary value of the B start boundary before correction is 100, but the time boundary value of the B end boundary shown in the new voice boundary is 180, so the time boundary value of the existing voice boundary is modified to 180.

[0062] In some embodiments, the step of correcting the existing speech boundary again further includes:

[0063] The boundary information of all speech boundaries processed in the buffer is sorted in ascending order according to the time boundary value, and the minimum time boundary value of the end boundary is compared with the maximum time boundary value of the start boundary in turn. If the difference exceeds the preset delay time, the boundary information is output in ascending order.

[0064] Specifically, after the boundary correction process, the boundaries entering the buffer are sorted in ascending order, and the smallest boundary is compared with the largest boundary in turn. If the difference exceeds the specified delay time, the boundaries are output in ascending order for the boundary analysis algorithm to solve the boundary disorder problem. Figure 3 As shown, before sorting, the time boundary value of B's start boundary is 200, and the time boundary value of the new speech boundary is 180. After sorting, the time boundary value of B's start boundary is modified to 180, and the time boundary value 230 of B's open and end boundaries is placed before the time boundary value 300 of A's end boundary and after the time boundary value 180 of B's start boundary.

[0065] In some embodiments, the present invention further comprises:

[0066] After the boundary information of the existing speech boundary is corrected, the new speech boundary information is buffered again.

[0067] like Figure 4 As shown, the embodiment of the present application provides a device for correcting a boundary buffer of a real-time speech recognition result, comprising:

[0068] An acquisition module 401 is configured to acquire a speech recognition result and identify speech boundary information in the speech recognition result;

[0069] The processing module 402 is configured to perform buffering processing on the speech boundary within a preset time period using a historical boundary buffer to obtain boundary information of the speech boundary;

[0070] The correction module 403 is configured to compare new speech boundary information with existing speech boundary information when there is new speech boundary information to be buffered, and to correct the existing speech boundary information according to the comparison result.

[0071] The working principle of the speech real-time recognition result boundary buffer correction device provided in the embodiment of the present application is as follows: the acquisition module 401 obtains the speech recognition result and identifies the speech boundary information in the speech recognition result; the processing module 402 uses the historical boundary buffer to buffer the speech boundary within a preset time period to obtain the boundary information of the speech boundary; when there is new speech boundary information to be buffered, the correction module 403 compares the new speech boundary information with the existing speech boundary information, and corrects the boundary information of the existing speech boundary according to the comparison result.

[0072] Preferably, the correction module includes:

[0073] a judging unit, configured to judge, when a new speech boundary is buffered, whether there is a boundary having the same role as the sentence corresponding to the new speech boundary;

[0074] If there is a correction unit, the time boundary value of the new speech boundary is compared to see whether it is greater than or equal to the time boundary value of the existing speech boundary. If it is less than, it is determined that confusion has occurred after the boundary correction, and the existing speech boundary is corrected again.

[0075] Preferably, the correction unit includes:

[0076] The first correction subunit is configured to correct the time boundary value of the new speech boundary to the time boundary value of the existing speech boundary.

[0077] Preferably, it also includes:

[0078] The second correction subunit is used to sort the boundary information of all speech boundaries processed in the buffer in ascending order according to the time boundary value, and compare the minimum time boundary value of the end boundary with the maximum time boundary value of the start boundary in turn. If the difference exceeds the preset delay time, the boundary information is output in ascending order.

[0079] Preferably, it also includes:

[0080] The buffer processing module is used to buffer the new speech boundary information again after correcting the boundary information of the existing speech boundary.

[0081] An embodiment of the present application provides a computer device, comprising a processor, and a memory connected to the processor;

[0082] The memory is used to store a computer program, and the computer program is used to execute the acquisition of speech recognition results provided in any of the above embodiments and identify speech boundary information in the speech recognition results;

[0083] Using a historical boundary buffer to perform buffering processing on speech boundaries within a preset time period to obtain boundary information of the speech boundaries;

[0084] When new speech boundary information is buffered, the new speech boundary information is compared with the existing speech boundary information, and the boundary information of the existing speech boundary is modified according to the comparison result;

[0085] The processor is used to call and execute computer programs in the memory.

[0086] In summary, the present invention provides a method and apparatus for buffering and correcting boundaries of real-time speech recognition results. The method comprises obtaining speech recognition results and identifying speech boundary information in the speech recognition results; buffering speech boundaries within a preset time period to obtain boundary information of the speech boundaries; and when new speech boundary information is buffered, comparing the new speech boundary information with existing speech boundary information, and correcting the existing speech boundary information based on the comparison results. By providing buffering to correct speech boundary information, the present invention can improve the fault tolerance of real-time analysis techniques based on time boundaries, thereby avoiding affecting the execution results of boundary analysis.

[0087] It can be understood that the method embodiment provided above corresponds to the device embodiment above, and the corresponding specific contents can be referenced to each other and will not be repeated here.

[0088] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) that contain computer-usable program code.

[0089] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0090] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction method, which is implemented in the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0091] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0092] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for correcting boundary buffer of real-time speech recognition results, characterized in that: include: Acquire a speech recognition result, and identify speech boundary information in the speech recognition result; Using a historical boundary buffer to perform buffering processing on speech boundaries within a preset time period to obtain boundary information of the speech boundaries; When new speech boundary information is buffered, the new speech boundary information is compared with the existing speech boundary information, and the boundary information of the existing speech boundary is modified according to the comparison result; The boundary information of the speech boundary includes: Time boundary value, the role of the sentence corresponding to the boundary, and the boundary type; Wherein, the boundary type includes a start boundary and an end boundary; When new speech boundary information is buffered, the new speech boundary information is compared with the existing speech boundary information, and the existing speech boundary information is corrected according to the comparison result, including: When a new speech boundary is buffered, determining whether there is a boundary having the same role as the sentence corresponding to the new speech boundary; If yes, then compare whether the time boundary value of the new speech boundary is greater than or equal to the time boundary value of the existing speech boundary; if less than, it is determined that the boundary is disordered after correction, and the existing speech boundary is corrected again; The step of correcting the existing speech boundary again includes: The time boundary value of the new speech boundary is corrected to the time boundary value of the existing speech boundary.

2. The method according to claim 1, characterized in that The step of correcting the existing speech boundary again further includes: The boundary information of all speech boundaries processed in the buffer is sorted in ascending order according to the time boundary value, and the minimum time boundary value of the end boundary is compared with the maximum time boundary value of the start boundary in turn. If the difference exceeds the preset delay time, the boundary information is output in ascending order.

3. The method according to claim 1, characterized in that Also includes: After the boundary information of the existing speech boundary is corrected, the new speech boundary information is buffered again.

4. The method according to claim 1, wherein The acquiring of the speech recognition result and identifying speech boundary information in the speech recognition result include: Acquiring acoustic feature information and language feature information of the speech data to be recognized; According to the acoustic feature information and language feature information, a target recognition path matching the language data to be recognized is determined in a pre-constructed static word graph, the word node information of each word node in the candidate recognition path of the voice data to be recognized is recorded, and a speech recognition result of the voice data to be recognized is obtained according to the target recognition path. Based on the word identifier of each recognized word in the speech recognition result and the word node information of each word node in the candidate recognition path, backtracking is performed along the target recognition path to obtain the speech boundary information of each recognized word in the speech recognition result in the voice data to be recognized.

5. A device for correcting boundary buffer of real-time speech recognition results, characterized in that: include: An acquisition module, configured to acquire a speech recognition result and identify speech boundary information in the speech recognition result; A processing module, configured to perform buffering processing on speech boundaries within a preset time period using a historical boundary buffer to obtain boundary information of the speech boundaries; The correction module is used to compare the new speech boundary information with the existing speech boundary information when there is new speech boundary information to be buffered, and to correct the boundary information of the existing speech boundary according to the comparison result.

Images