Speech recognition method and device, electronic equipment and storage medium

Abstract

This application provides a speech recognition method, apparatus, electronic device, and storage medium. The speech recognition method includes: acquiring an audio stream; recognizing the audio stream to obtain a token text sequence, the token text sequence including multiple token texts and time information corresponding to each token text; traversing the multiple token texts according to the time information; if it is determined that the length of the paragraph before the current token text is greater than a preset length, or the time difference between the current token text and the previous token text is greater than a preset duration, segmenting before the current token text to divide the token text sequence into multiple paragraphs; adding punctuation marks to each of the multiple paragraphs, and outputting the paragraphs with punctuation marks, thereby enabling reasonable segmentation that conforms to semantic logic and improving the accuracy of subsequent semantic understanding.

Description

Technical Field

[0001] This application belongs to the field of speech processing, and in particular relates to speech recognition methods, devices, electronic devices and storage media. Background Technology

[0002] Speech recognition technology, especially Automatic Speech Recognition (ASR), can convert speech into text in real-time or near real-time, and is widely used in scenarios such as meeting transcription, interview recording, and short video subtitles. Current speech recognition methods, after obtaining text from the input speech, generally segment the text and add punctuation according to a single rule. This can easily lead to unreasonable segmentation and punctuation, resulting in semantic misunderstandings. Summary of the Invention

[0003] In view of this, embodiments of this application provide a speech recognition method, apparatus, electronic device, and storage medium, which can improve segmentation quality and punctuation quality, thereby improving the accuracy of semantic understanding.

[0004] The first aspect of this application provides a speech recognition method, including: Get the audio stream; The audio stream is identified to obtain a token text sequence, which includes multiple token texts and time information corresponding to each token text. According to the time information, the multiple token texts are traversed. If it is determined that the length of the paragraph before the current token text is greater than the preset length, or the time difference between the current token text and the previous token text is greater than the preset duration, the current token text is segmented to divide the token text sequence into multiple paragraphs. Add punctuation marks to each of the multiple paragraphs, and output the paragraph with the punctuation marks.

[0005] In one embodiment, after dividing the token text sequence into multiple paragraphs, the method further includes: The time information of each paragraph is generated based on the token text at the beginning and the token text at the end of each paragraph.

[0006] In one embodiment, outputting a paragraph containing the punctuation marks includes: The paragraph containing the punctuation marks and the time information of the paragraph are output through a message queue.

[0007] In one embodiment, adding punctuation marks to the plurality of paragraphs includes: For each paragraph, create a punctuation model instance; Add the punctuation marks to the paragraph based on the punctuation model instance; Release the punctuation model instance.

[0008] In one embodiment, after acquiring the audio stream, the method further includes: The audio stream is padded with a tail, the length of which is determined based on the sampling rate of the audio stream.

[0009] In one embodiment, acquiring an audio stream includes: Obtain the speech recognition task and the identifier corresponding to the speech recognition task; The speech recognition task is deduplicated based on the identifier; If the deduplication check passes, and the resource usage of the speech recognition task is less than the current total resource usage, the audio stream is obtained.

[0010] In one embodiment, after acquiring the audio stream, the method further includes: When the speech recognition task ends or an error occurs, the resources occupied by the speech recognition task are deducted from the current total resource usage through atomic operations, and the cache of the speech recognition task is cleared.

[0011] A second aspect of this application provides a voice recognition device, including: The acquisition module is used to acquire audio streams; The recognition module is used to recognize the audio stream and obtain a token text sequence, wherein the token text sequence includes multiple token texts and time information corresponding to each token text; The segmentation module is used to traverse the multiple token texts according to the time information. If it is determined that the length of the paragraph before the current token text is greater than a preset length, or the time difference between the current token text and the previous token text is greater than a preset duration, the current token text is segmented before the current token text to divide the token text sequence into multiple paragraphs. The output module is used to add punctuation marks to the multiple paragraphs respectively, and output the paragraphs with the punctuation marks.

[0012] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the speech recognition method as described in the first aspect above.

[0013] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the speech recognition method as described in the first aspect above.

[0014] The fifth aspect of this application provides a computer program product that, when run on an electronic device, causes the electronic device to execute the speech recognition method described in any one of the first aspects.

[0015] The beneficial effects of this application embodiment compared with the prior art are as follows: by recognizing the acquired audio stream, a token text sequence is obtained. Multiple token texts are traversed according to time information. If it is determined that the length of the paragraph before the current token text is greater than a preset length, or the time difference between the current token text and the previous token text is greater than a preset duration, the current token text is segmented to divide the token text sequence into multiple paragraphs. This can fit the semantic logic and reasonably segment the token text sequence. Then, punctuation marks are added to each of the multiple paragraphs, and the paragraphs with punctuation marks are output. This can improve the quality of punctuation marks, thereby improving the accuracy of subsequent semantic understanding and enhancing the readability of the recognition results. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0017] Figure 1 This is a schematic diagram illustrating the implementation process of a speech recognition method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the speech recognition device provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0018] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0019] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0020] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0021] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0022] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0023] The speech recognition method provided in this application is illustrated below.

[0024] Please see the appendix Figure 1 The speech recognition method provided in one embodiment of this application includes S101 to S104.

[0025] S101: Get audio stream.

[0026] Specifically, the audio stream can be real-time voice data captured by the microphone, audio files uploaded by the client, audio data obtained from media stream parsing, or streaming audio data packets pushed by a third-party service, etc. The audio stream is passed in as a continuous data stream.

[0027] In one embodiment, upon receiving a speech recognition task request, the system obtains the speech recognition task, its corresponding identifier, and the task size. Each speech recognition task corresponds to a unique identifier. First, a deduplication check is performed on the speech recognition tasks based on the identifiers, comparing the task identifier with the identifiers of existing tasks in processing in the cache. If the identifier of a speech recognition task exists in the cache, the task is determined to be a duplicate, and the system returns to a processing status without repeatedly consuming system resource quotas. If the identifier of a speech recognition task does not exist in the cache, the task is determined to be a new task, and the system proceeds to the subsequent resource admission judgment process.

[0028] When determining resource access, the current total resource usage is determined by reading atomic variables maintained by the system. These atomic variables support atomic operations, allowing safe reading and modification in a multi-threaded environment without data races. The resource usage of the speech recognition task is determined based on its size, and the sum of the current total resource usage and the speech recognition task's resource usage is calculated. If the result is greater than or equal to a preset resource threshold, the speech recognition task is rejected and a resource limitation error is returned. If the result is less than the resource threshold, a Compare-And-Swap (CAS) atomic operation is used to atomically increase the resource usage. If CAS fails, it indicates that another thread has simultaneously modified the resource usage; therefore, the speech recognition task request is recursively retried until CAS succeeds or the calculation result exceeds the resource threshold, thus ensuring resource consistency under high concurrency. After a successful CAS, the speech recognition task's identifier is written to a cache with an expiration time, marking the task as being in a processing state, completing resource access, and allowing audio stream acquisition. The speech recognition task can be submitted to a thread pool for asynchronous execution, thus avoiding blocking the main process.

[0029] In one embodiment, after acquiring the audio stream, end-padding is performed on the audio stream by adding silence frames to the end of the audio stream. The end-padding is a sampling rate adaptive end-padding, and the length of the end-padding is determined according to the sampling rate of the audio stream. For example, the end-padding length is the product of a preset ratio and the sampling rate. The preset ratio can be set according to requirements, so that the silence duration is approximately the same under different sampling rates without modifying the processing logic. For example, if the preset ratio is 0.8, and the sampling rate is 8kHz, the end-padding length is approximately 0.8 seconds of silence (approximately 6400 points); if the sampling rate is 16kHz, the end-padding length is approximately 0.8 seconds of silence (approximately 12800 points). End-padding avoids missing end-of-speech recognition content, improving the accuracy of speech recognition.

[0030] In one embodiment, after acquiring the speech recognition task, if the size of the speech recognition task is detected to be zero, the speech recognition task is output as completed, and the subsequent speech recognition process is not entered, thus saving system resources.

[0031] S102: Recognize the audio stream to obtain a token text sequence, the token text sequence including multiple token texts and time information corresponding to each token text.

[0032] Specifically, a streaming speech recognition processing engine can be invoked to perform real-time recognition processing on the audio stream, outputting a sequence of token texts. The token text is the smallest recognition unit output by the speech recognition model, and can be a character or a word. The time information can be a timestamp, representing the start time point of each token text within the audio stream, used to mark the position of that token text in the audio. Adding time information to each token text provides data support for subsequent time alignment.

[0033] It is understandable that the recognition result can be output synchronously during the input of the audio stream, without waiting for the entire audio stream to be input, in order to improve the real-time performance of speech recognition.

[0034] In one embodiment, after obtaining the token text sequence, it is determined whether the lengths of the multiple token texts and the lengths of the multiple time information are consistent. If the lengths of the multiple token texts and the multiple time information are inconsistent or empty, an error message is output to ensure the legality of subsequent data processing. If the lengths of the multiple token texts and the multiple time information are consistent and not empty, the subsequent segmentation steps are performed.

[0035] In one embodiment, after obtaining the token text sequence, token texts with abnormal time information or disordered text can be removed to improve the accuracy of subsequent segmentation. Alternatively, the token text can be initially identified, and if duplicate recognition, missed recognition, or abnormal sentence segmentation is determined, the token text can be corrected to improve the overall accuracy of speech recognition.

[0036] S103: Traverse the multiple token texts according to the time information. If it is determined that the length of the paragraph before the current token text is greater than a preset length, or the time difference between the current token text and the previous token text is greater than a preset duration, segment the token text before the current token text to divide the token text sequence into multiple paragraphs.

[0037] Specifically, starting from the second token text, the entire token text sequence is traversed. For the current token text, the time difference between the current token text and the previous token text is calculated, and the length of the current paragraph is maintained, i.e., the cumulative length of the token text starting from the beginning of the paragraph (the cumulative text length of the paragraph). Simultaneously, dual threshold checks are performed: whether the time difference is greater than a preset duration, and whether the current paragraph length is greater than a preset length. The preset duration and preset length are configurable parameters that can be optimized for different application scenarios such as meetings, interviews, and short videos. For example, the preset duration is 2.0 seconds, and the preset length is 500 characters. When either condition is met—that is, if the time difference between the current token text and the previous token text is greater than the preset duration, or the length of the current paragraph is greater than the preset length—segmentation is performed before the current token text. The segmentation position is located between the last token text of the current paragraph and the first token text of the next paragraph, ensuring clear paragraph boundaries, continuous and non-overlapping time information, and no loss of time information.

[0038] If segmentation is based solely on the time difference between the current and previous token texts, long sentences can easily be broken into multiple paragraphs when the audio stream is fast-paced or continuous, affecting semantic integrity. If segmentation is based solely on the cumulative text length of the paragraphs, pauses can be easily overlooked when the audio stream is slow-paced or contains natural pauses, resulting in poor punctuation and an inability to accurately reflect the natural rhythm of the audio stream. By combining the time difference between the current and previous token texts with the cumulative text length of the paragraphs, both paragraph duration and length can be constrained simultaneously, avoiding the shortcomings of segmentation based on a single threshold and improving the readability of the output.

[0039] In one embodiment, during the segmentation of the token text sequence, information such as the current audio stream's speech rate, audio stream type, and the context of the speech recognition task can be obtained. Based on this information, the preset duration and preset length can be adaptively adjusted. For example, the preset duration can be reduced when the speech rate is fast and increased when the speech rate is slow, making the segmentation results more consistent with the actual context.

[0040] In one embodiment, after segmenting the token text sequence into paragraphs, semantic similarity detection can be performed on adjacent paragraphs. If the semantic similarity of adjacent paragraphs is greater than a preset value and the total length of adjacent paragraphs is less than a set length value, then the adjacent paragraphs are merged into one paragraph, thereby avoiding unnecessary segmentation caused by short pauses and conforming to semantic logic.

[0041] In one embodiment, after segmentation, the time information of each segment is generated based on the token text at the start and end of each segment. Specifically, the start time information of a segment is the time information of the token text at the start of the segment; for example, the start time information of the first segment is the time information of the first token text, and the end time information of the segment is the time information of the token text at the end of the segment. Compared to obtaining all token texts in chronological order and then outputting them, output efficiency is improved. Compared to pushing all token texts in real time, this embodiment adds time information to each segment, allowing each segment to carry independent, continuous, and accurate start and end time information, facilitating subsequent retrieval, alignment, and positioning based on the time information.

[0042] The process iterates through all token texts and adds time information to each paragraph, then outputs a list of paragraphs. If the last paragraph is not empty, it is also added to the list to ensure the end paragraph is complete and not omitted.

[0043] S104: Add punctuation marks to the multiple paragraphs respectively, and output the paragraph with the punctuation marks.

[0044] Specifically, the segmented punctuation processing engine is invoked to input each paragraph into the punctuation model, resulting in the paragraphs with added punctuation as output by the punctuation model. The punctuation model is a pre-trained model; for example, it could be a convolutional neural network. A model trained using a Convolutional Neural Network (CNN-BiLSTM) combined with Byte Pair Encoding (BPE).

[0045] In one embodiment, for each paragraph, a punctuation model instance is created to load the weights of the punctuation model and complete the inference environment configuration for the punctuation model. Based on the punctuation model instance, the paragraph is input into the punctuation model to obtain the paragraph with added punctuation marks output by the punctuation model. After adding punctuation marks to the paragraph, the punctuation model instance is released to free up memory and video memory resources, preventing the punctuation model from residing in memory for an extended period. Multiple paragraphs can be added to in parallel to improve computational efficiency.

[0046] In one embodiment, after adding punctuation marks to each paragraph, the punctuation marks can be optimized based on information such as paragraph length, speech rate, and pause positions. For example, punctuation marks can be added at faster speech rates to keep the punctuation marks consistent with the speech rhythm and improve the matching degree between the output paragraph text and the actual scene.

[0047] In one embodiment, for each paragraph, the paragraph is first preprocessed by converting the English text in the paragraph to lowercase and unifying the text format to fit the punctuation model, and then the preprocessed paragraph is input into the punctuation model.

[0048] In one embodiment, when the punctuation model outputs a paragraph with added punctuation, it can simultaneously output a confidence score. The higher the confidence score, the higher the accuracy of the punctuation marks, which facilitates subsequent filtering or verification based on the confidence score and improves the overall accuracy of speech recognition.

[0049] In one embodiment, if an anomaly occurs during the segmentation or punctuation addition process, the segment with added punctuation is output in a downgraded manner, without losing all recognition results, ensuring that some results are usable and avoiding the problem of losing all recognition results.

[0050] In one embodiment, after adding punctuation marks to a paragraph, the paragraph with added punctuation, its start time information, its end time information, and a status identifier are encapsulated, and the encapsulated result is added to a message queue. The status identifier indicates completion or failure, and the message queue can be a Kafka distributed message queue. The encapsulated result is pushed to a designated topic (e.g., a speech recognition result topic) via the message queue. Downstream systems subscribe to and consume the results according to the designated topic, thereby obtaining recognition results with time information in real time, meeting the needs of various application scenarios such as real-time transcription, subtitle generation, evidence recording, and retrieval alignment.

[0051] In one embodiment, when the speech recognition task ends, the resources occupied by the speech recognition task are deducted from the current total resource usage through atomic operations, the cache records of the speech recognition task are cleared, and logs related to the speech recognition task are recorded.

[0052] When an anomaly occurs in the speech recognition task, the resources occupied by the speech recognition task are deducted from the current total resource usage through atomic operations, the cache records of the speech recognition task are cleared, and a status notification of task failure is output.

[0053] Regardless of whether the speech recognition task is completed normally or terminated abnormally, the corresponding resources will be released to prevent resource hoarding, ensure that the rate limiting count is consistent with the business status, and prevent failed speech recognition tasks from still occupying resource quotas.

[0054] In the above embodiments, by performing deduplication checks on speech recognition tasks, duplicate tasks can be avoided from occupying system resources and resource utilization can be improved; by using CAS atomic operations to increase and decrease resource usage, accurate resource counting and stable system operation can be ensured in high-concurrency scenarios; by performing resource rollback and cache cleanup when speech recognition tasks end normally or abnormally, resource hoarding can be prevented and a closed-loop resource management system can be achieved; by performing adaptive tail padding on the audio stream, the integrity of the streaming recognition results can be ensured and tail content loss can be avoided; by performing an early termination mechanism on zero-length audio files, invalid calculations can be reduced and system overhead can be lowered; by using time and length thresholds as segmentation criteria, This approach ensures that paragraph boundaries align more closely with semantic logic and that segmentation is more logical. Protecting the beginning and end of paragraphs guarantees the integrity of the content. Generating time information for each paragraph facilitates time alignment, retrieval, and location by downstream systems. Creating and releasing punctuation model instances separately for each paragraph reduces memory usage and improves system concurrency. Implementing fallback pushes in abnormal scenarios, outputting only paragraphs with added punctuation, ensures some recognition results are usable and improves system robustness. Outputting paragraphs with punctuation and time information via a message queue enables near real-time result push, meeting the needs of various application scenarios such as real-time transcription and subtitle generation.

[0055] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0056] Corresponding to the speech recognition method described in the above embodiments, Figure 2 A structural block diagram of a speech recognition device provided in an embodiment of this application is shown. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0057] like Figure 2 As shown, the speech recognition device includes an acquisition module 21, a recognition module 22, a segmentation module 23, and an output module 24.

[0058] Module 21 is used to acquire audio streams; The recognition module 22 is used to recognize the audio stream and obtain a token text sequence, wherein the token text sequence includes multiple token texts and time information corresponding to each token text; The segmentation module 23 is used to traverse the multiple token texts according to the time information. If it is determined that the length of the paragraph before the current token text is greater than a preset length, or the time difference between the current token text and the previous token text is greater than a preset duration, the current token text is segmented before the current token text to divide the token text sequence into multiple paragraphs. Output module 24 is used to add punctuation marks to the multiple paragraphs respectively and output the paragraphs with the punctuation marks.

[0059] In one embodiment, the segmentation module 23 is further configured to: The time information of each paragraph is generated based on the token text at the beginning and the token text at the end of each paragraph.

[0060] In one embodiment, the output module 24 is specifically used for: The paragraph containing the punctuation marks and the time information of the paragraph are output through a message queue.

[0061] In one embodiment, the output module 24 is specifically used for: For each paragraph, create a punctuation model instance; Add the punctuation marks to the paragraph based on the punctuation model instance; Release the punctuation model instance.

[0062] In one embodiment, the acquisition module 21 is further configured to: The audio stream is padded with a tail, the length of which is determined based on the sampling rate of the audio stream.

[0063] In one embodiment, the acquisition module 21 is further configured to: Obtain the speech recognition task and the identifier corresponding to the speech recognition task; The speech recognition task is deduplicated based on the identifier; If the deduplication check passes, and the resource usage of the speech recognition task is less than the current total resource usage, the audio stream is obtained.

[0064] In one embodiment, the acquisition module 21 is further configured to: When the speech recognition task ends or an error occurs, the resources occupied by the speech recognition task are deducted from the current total resource usage through atomic operations, and the cache of the speech recognition task is cleared.

[0065] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0066] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may be a desktop computer, laptop, handheld computer, or cloud server, etc.

[0067] like Figure 3 As shown, the electronic device in this embodiment includes: a processor 31, a memory 32, and a computer program 33 stored in the memory 32 and executable on the processor 31. When the processor 31 executes the computer program 33, it implements the steps in the above-described speech recognition method embodiment, for example... Figure 1 Steps S101 to S104 are shown. Alternatively, when the processor 31 executes the computer program 33, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 2 The functions of the acquisition module 21 to the output module 24 are shown.

[0068] For example, the computer program 33 may be divided into one or more modules / units, which are stored in the memory 32 and executed by the processor 31 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 33 in the electronic device.

[0069] Those skilled in the art will understand that Figure 3 This is merely an example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0070] The processor 31 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0071] The memory 32 can be an internal storage unit of the electronic device, such as a hard drive or memory. The memory 32 can also be an external storage device of the electronic device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 32 can include both internal and external storage units. The memory 32 is used to store the computer program and other programs and data required by the electronic device. The memory 32 can also be used to temporarily store data that has been output or will be output.

[0072] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0073] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0074] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0075] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0076] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0077] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0078] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0079] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A speech recognition method, characterized in that, include: Get the audio stream; The audio stream is identified to obtain a token text sequence, which includes multiple token texts and time information corresponding to each token text. According to the time information, the multiple token texts are traversed. If it is determined that the length of the paragraph before the current token text is greater than the preset length, or the time difference between the current token text and the previous token text is greater than the preset duration, the current token text is segmented to divide the token text sequence into multiple paragraphs. Add punctuation marks to each of the multiple paragraphs, and output the paragraph with the punctuation marks.

2. The speech recognition method according to claim 1, characterized in that, After dividing the token text sequence into multiple segments, the process further includes: The time information of each paragraph is generated based on the token text at the beginning and the token text at the end of each paragraph.

3. The speech recognition method according to claim 2, characterized in that, Output the paragraph containing the punctuation marks, including: The paragraph containing the punctuation marks and the time information of the paragraph are output through a message queue.

4. The speech recognition method according to claim 1, characterized in that, Add punctuation marks to each of the multiple paragraphs, including: For each paragraph, create a punctuation model instance; Add the punctuation marks to the paragraph based on the punctuation model instance; Release the punctuation model instance.

5. The speech recognition method according to claim 1, characterized in that, After acquiring the audio stream, the following is also included: The audio stream is padded with a tail, the length of which is determined based on the sampling rate of the audio stream.

6. The speech recognition method according to claim 1, characterized in that, Acquiring the audio stream includes: Obtain the speech recognition task and the identifier corresponding to the speech recognition task; The speech recognition task is deduplicated based on the identifier; If the deduplication check passes, and the resource usage of the speech recognition task is less than the current total resource usage, the audio stream is obtained.

7. The speech recognition method according to claim 6, characterized in that, After acquiring the audio stream, the following is also included: When the speech recognition task ends or an error occurs, the resources occupied by the speech recognition task are deducted from the current total resource usage through atomic operations, and the cache of the speech recognition task is cleared.

8. A voice recognition device, characterized in that, include: The acquisition module is used to acquire audio streams; The recognition module is used to recognize the audio stream and obtain a token text sequence, wherein the token text sequence includes multiple token texts and time information corresponding to each token text; The segmentation module is used to traverse the multiple token texts according to the time information. If it is determined that the length of the paragraph before the current token text is greater than a preset length, or the time difference between the current token text and the previous token text is greater than a preset duration, the current token text is segmented before the current token text to divide the token text sequence into multiple paragraphs. The output module is used to add punctuation marks to the multiple paragraphs respectively, and output the paragraphs with the punctuation marks.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the speech recognition method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the speech recognition method as described in any one of claims 1 to 7.

Images