Speech recognition method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a voice recognition method and device, electronic equipment, a computer readable storage medium and a computer program product, and relates to the technical field of computers, in particular to the technical fields of voice recognition, voice transcription, intelligent network disks and the like. Determining first text content and first speaker information corresponding to the first text content; second speaker information corresponding to an end paragraph in an identified text is determined, and the identified text is determined based on a voice segment before the first voice segment in the recording process; and updating an end paragraph of the recognized text based on the first speaker information, the second speaker information and the first text content to obtain a target recognized text.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and more specifically, to technical fields such as speech recognition, speech transcription, and smart network disks, and in particular to a speech recognition method, device, electronic device, and storage medium. Background Art

[0002] With the continuous development of technologies in fields such as natural language processing and deep learning, automatic speech recognition and speech transcription technologies are currently widely used in conference records, voice assistants, customer service systems and other fields. Summary of the Invention

[0003] The embodiments of the present disclosure provide a speech recognition method, apparatus, electronic device, computer-readable storage medium, and computer program product.

[0004] In the first aspect, an embodiment of the present disclosure proposes a speech recognition method, including: determining a first text content and first speaker information corresponding to the first text content based on a first voice segment obtained during a recording process; determining second speaker information corresponding to the last paragraph in the recognized text, wherein the recognized text is determined based on the voice segment before the first voice segment during the recording process; updating the last paragraph of the recognized text based on the first speaker information, the second speaker information and the first text content to obtain a target recognized text.

[0005] In the second aspect, an embodiment of the present disclosure proposes a speech recognition device, including: a speech recognition module, configured to determine a first text content and first speaker information corresponding to the first text content based on a first speech segment obtained during the recording process; a text processing module, configured to determine the second speaker information corresponding to the last paragraph in the recognized text; based on the first speaker information, the second speaker information and the first text content, the last paragraph of the recognized text is updated to obtain a target recognized text, wherein the recognized text is determined based on the speech segment before the first speech segment during the recording process.

[0006] In a third aspect, an embodiment of the present disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can implement the speech recognition method described in any implementation method in the first aspect when executing the instructions.

[0007] In a fourth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, which are used to enable a computer to implement the speech recognition method described in any implementation method of the first aspect when executed.

[0008] In a fifth aspect, an embodiment of the present disclosure provides a computer program product comprising a computer program, which, when executed by a processor, can implement the speech recognition method described in any implementation manner in the first aspect.

[0009] The speech recognition method and device provided by the embodiments of the present disclosure can reduce the huge pressure on the memory caused by the full update of data during the recording process by updating only the text content of the last paragraph in the recognized text each time the recognition processing of a speech segment is completed during the recording process, and then updating the text content as a whole at the end of the recording process, thereby improving the stability of operation and enhancing the user experience.

[0010] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Other features, objects and advantages of the present disclosure will become more apparent from a reading of the detailed description of non-limiting embodiments made with reference to the following drawings: Figure 1 is an exemplary system architecture in which the present disclosure may be applied; Figure 2 A flowchart of a speech recognition method provided in an embodiment of the present disclosure; Figure 3 A flowchart of a speech recognition method provided by another embodiment of the present disclosure; Figure 4 For this disclosure Figure 3 The process flow diagram of step 302 in the embodiment shown; Figure 5 A flowchart of an application scenario of a speech recognition method provided in an embodiment of the present disclosure; Figure 6 A structural block diagram of a speech recognition device provided in an embodiment of the present disclosure; Figure 7 A schematic diagram of the structure of an electronic device suitable for executing a speech recognition method provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0012] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description. It should be noted that the embodiments in the present disclosure and the features in the embodiments can be combined with each other unless there is a conflict.

[0013] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information in the technical solutions disclosed herein comply with relevant laws and regulations and do not violate public order and good morals. In the technical solutions disclosed herein, the user's authorization or consent is obtained before obtaining or collecting user personal information.

[0014] Figure 1 An exemplary system architecture 100 is shown to which the speech recognition method and apparatus according to the embodiments of the present disclosure can be applied.

[0015] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. Network 104 is a medium for providing communication links between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0016] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Terminal devices 101, 102, and 103 can be various electronic devices with display screens that support receiving server messages, including but not limited to smartphones, tablet computers, e-book readers, electronic players, laptop computers, and desktop computers, etc.

[0017] Terminal devices 101, 102, and 103 can be either hardware or software. When hardware is used, they can be various electronic devices. When software is used, they can be installed in the electronic devices listed above. They can be implemented as multiple software programs or software modules (e.g., multiple software modules for providing distributed services) or as a single software program or software module. This is not specifically limited here.

[0018] The server 105 can be a server that provides various services, such as a backend server that receives requests sent by terminal devices that establish communication connections with it. The backend server can receive and analyze the requests sent by the terminal devices, and generate processing results that are fed back to the terminal devices (for example, performing voice recognition processing on the user's voice data received to generate target text corresponding to the voice data). The server can be hardware or software. When the server is hardware, it can be various electronic devices that provide various services to the terminal devices. When the server is software, it can be implemented as multiple software programs or software modules that provide various services to the terminal devices, or it can be implemented as a single software program or software module that provides various services to the terminal devices. No specific limitations are given here.

[0019] The server 105 can provide various services through various built-in applications. Taking an application that can provide speech recognition services as an example, the server 105 can achieve the following effects when running the application: first, based on the first voice segment obtained during the recording process, the first text content and the first speaker information corresponding to the first text content are determined; then, the second speaker information corresponding to the last paragraph in the recognized text is determined, wherein the recognized text is determined based on the voice segment before the first voice segment during the recording process; finally, based on the first speaker information, the second speaker information and the first text content, the last paragraph of the recognized text is updated to obtain the target recognized text.

[0020] The speech recognition methods provided in the subsequent embodiments of this disclosure are generally executed by the server 105, and accordingly, the speech recognition device is generally also disposed in the server 105. However, it should also be noted that, when the terminal devices 101, 102, and 103 also have sufficient computing capabilities and computing resources, the speech recognition methods can also be executed by the terminal devices 101, 102, and 103. Accordingly, the speech recognition device can also be disposed in the terminal devices 101, 102, and 103.

[0021] It should be understood that Figure 1 The number of terminal devices, network devices and servers in the embodiment is only for illustration. Any number of terminal devices, network devices and servers may be provided as required.

[0022] refer to Figure 2 , Figure 2 The process of the speech recognition method provided according to one embodiment of the present disclosure is shown. The process 200 may include the following steps: Step 201 : determining first text content and first speaker information corresponding to the first text content based on a first voice segment obtained during a recording process.

[0023] This step is intended to be performed by the execution subject of the speech recognition method (e.g. Figure 1The server 105 or other device in the embodiment converts the acquired first voice segment into first text content in text form through voice recognition technology, and determines first speaker information corresponding to the first text content based on the first voice segment.

[0024] During recording in some application scenarios, such as conferences, a recording device can be used to capture the user's voice input in real time to obtain a first voice segment. The first voice segment can be a continuous audio stream. It should be understood that in other application scenarios, the first voice segment can also be obtained based on pre-stored audio and video data.

[0025] In some embodiments, the above-mentioned execution entity can obtain the first text content based on the acquired first voice segment through voice recognition processing, and can identify the corresponding speaker information by inputting the first voice segment into a deep neural network model as the first speaker information corresponding to the first text content.

[0026] In some implementations, the execution entity may further associate and store the first text content with the first voiceprint feature.

[0027] Step 202: Determine the second speaker information corresponding to the last paragraph in the recognized text.

[0028] This step is intended to determine the second speaker information corresponding to the last paragraph in the recognized text by the execution subject. The recognized text is determined based on the recognized voice segment, which can be, for example, the voice segment before the first voice segment in the recording process.

[0029] In some implementations, the execution entity may determine the second speaker information corresponding to the last paragraph of the recognized text based on the correspondence between the text content recorded during the recording process and the corresponding speaker information.

[0030] Step 203 : Based on the first speaker information, the second speaker information and the first text content, the last paragraph of the recognized text is updated to obtain a target recognized text.

[0031] This step aims to update the last paragraph of the recognized text according to the first speaker, the second speaker and the first text content by the execution subject, and obtain the target recognized text according to the updated recognized text.

[0032] In some embodiments, if the execution entity determines that the first speaker information and the second speaker information are consistent, the execution entity may further determine the semantic similarity between the first text content and the text content of the current last paragraph. If the semantic similarity is greater than a first preset similarity threshold, the first text content and the last paragraph are merged into one paragraph. If the semantic similarity is less than the first preset similarity threshold, the first text content is used as the new last paragraph. For example, the semantic similarity between the first text content and the text content of the last paragraph may be determined based on contextual information or keywords determined based on the first text content and contextual information or keywords determined based on the last paragraph.

[0033] The speech recognition method of the above embodiment of the present disclosure first determines the first text content and the first speaker information corresponding to the first text content based on the first voice segment obtained during the recording process; then determines the second speaker information corresponding to the last paragraph in the recognized text, wherein the recognized text is determined based on the voice segment before the first voice segment during the recording process; finally, based on the first speaker information, the second speaker information and the first text content, the last paragraph of the recognized text is updated to obtain the target recognized text. This method obtains the target recognized text by updating only the text content of the last paragraph in the recognized text each time the recognition processing of a voice segment is completed during the recording process, and then updating the text content as a whole at the end of the recording process. This can reduce the huge pressure on the memory caused by the full update of data during the recording process, thereby improving the stability of operation and enhancing the user experience.

[0034] Continue to refer Figure 3 , Figure 3 The figure shows the process of a speech recognition method provided according to another embodiment of the present disclosure. The speech recognition method provided in this embodiment can be applied to various application scenarios such as meeting minutes, court trial records, customer service consultation archives, and can be applied to various terminal devices, electronic devices, software applications, and smart network disks.

[0035] like Figure 3 As shown, the process 300 may include the following steps: Step 301: Acquire a first voice segment.

[0036] In this embodiment, the execution subject (eg Figure 1 The server 105) obtains a first voice segment by collecting the user's voice input during the recording process.

[0037] In some optional implementations, the execution entity further performs time marking processing on the first voice segment, so that the first voice segment carries time marking information, and the time marking information may be a timestamp.

[0038] Step 302: Determine first text content and speaker information corresponding to the first text content.

[0039] In this embodiment, the execution entity may perform speech recognition on the acquired first voice segment, convert the voice data into text, and determine corresponding speaker information based on the first voice segment.

[0040] In some implementations, timestamp information may be added to the determined first text, which may include, for example, the start time and end time of the first voice segment.

[0041] refer to Figure 4 As shown, in an exemplary embodiment, step 302 may specifically include the following sub-steps 401 to 404.

[0042] Step 401: Perform speech recognition on a first speech segment to obtain first text content.

[0043] In step 401 , ASR (Automatic Speech Recognition) or STT (Speech to Text) technology may be used to convert the first voice segment into text data to obtain first text content.

[0044] Step 402: Match the first voiceprint feature extracted from the first speech segment with the registered voiceprint features in the voiceprint dictionary.

[0045] In step 402, a preset feature extraction algorithm can be used to extract characteristic parameters from the first voice segment to generate a voiceprint feature code as the first voiceprint feature. The voiceprint feature code can be a string of digital identifiers or a feature vector. The extracted first voiceprint feature is then matched with registered voiceprint features in a voiceprint dictionary.

[0046] The Voiceprint Dictionary is used to store and manage the voiceprint feature codes of different speakers and their corresponding identity information.

[0047] Exemplarily, matching can be performed by calculating the similarity between the first voiceprint feature and registered voiceprint features in the voiceprint dictionary. If the calculated similarity is greater than a preset similarity threshold, it is determined that the match is successful.

[0048] Step 403: Determine the speaker information corresponding to the successfully matched registered voiceprint feature as the first speaker information corresponding to the first text content.

[0049] If the first voiceprint feature is successfully matched to a registered voiceprint in the voiceprint dictionary, the speaker information corresponding to the registered voiceprint is determined as the first speaker information corresponding to the first text content. If the match is unsuccessful, the match can be repeated immediately or after a certain time interval.

[0050] Step 404 (optional): associate the first text content with the first voiceprint feature and store it in a local data storage linked table.

[0051] In some optional implementations, the execution entity may further associate the first text content with the first voiceprint feature and store it in a data storage linked list (ContentList). During the recording process, the text content recognized based on the multiple voice segments and the associated voiceprint features of the segments are sequentially stored in the data storage linked list.

[0052] As an exemplary embodiment, the data storage list can also be stored in a local file, for example, in JSON (JavaScript Object Notation) format. Storing the data storage list in a local file reduces the pressure on the buffer caused by storing intermediate files and allows for backtracking of the stored local file as needed. For example, if the first voiceprint feature fails to be successfully matched to a registered voiceprint in step 403, the data storage list can be read after a certain time interval to rematch the first voiceprint feature with the registered voiceprint features in the voiceprint dictionary.

[0053] Step 303: Determine the second speaker information corresponding to the last paragraph of the recognized text.

[0054] In this embodiment, recognized text may be obtained based on text content determined by recognizing the acquired voice segments during the recording process. The recognized text may include text content obtained based on multiple recognized voice segments.

[0055] In some embodiments, the recognized text includes the text content of at least one paragraph and may also include speaker information corresponding to each text content. For example, the corresponding speaker information may be added at the beginning of the text content of each paragraph, or at the beginning of the first paragraph of a series of text content from the same speaker.

[0056] In some implementations, the recognized text may further include timestamp information corresponding to the text content, such as the start time of the text content.

[0057] In an exemplary embodiment, the execution entity may determine the second speaker information corresponding to the last paragraph of the recognized text based on the speaker information corresponding to the text content included in the recognized text.

[0058] In some other embodiments, the above-mentioned execution entity can also determine the voiceprint feature associated with the last paragraph in the recognized text based on the stored data storage list, record it as the second voiceprint feature, and match the second voiceprint feature with the registered voiceprint features in the voiceprint dictionary to determine the second speaker information corresponding to the last paragraph in the recognized text.

[0059] Step 304: Determine whether the first speaker information is consistent with the second speaker information.

[0060] In this embodiment, based on the above step 303, the first speaker information is compared with the second speaker information corresponding to the last paragraph of the recognized text to determine whether they are consistent. If they are consistent, step 305 is executed; if they are inconsistent, step 307 is executed.

[0061] Step 305: merge the first text content into the last paragraph of the recognized text.

[0062] In this embodiment, if it is determined that the first speaker information is consistent with the second speaker information, the first text content is merged into the last paragraph of the recognized text to ensure the coherence of the text content of the same speaker.

[0063] Step 306 : Segment the merged last paragraph based on the first segmentation rule to obtain the target recognition text.

[0064] In this embodiment, after the first text content is merged into the last paragraph of the recognized text, in order to ensure the readability and aesthetics of the recognized text, the merged last paragraph is segmented based on the first segmentation rule to obtain text content with higher readability.

[0065] In some exemplary embodiments, the first segmentation rule may include segmenting the merged final paragraph in response to the number of sentences in the merged final paragraph reaching a preset sentence threshold. The number of sentences in each segmented paragraph should be less than or equal to the preset sentence threshold (e.g., 5 sentences, 10 sentences, etc.) to improve the readability of the displayed text content.

[0066] In some exemplary embodiments, the first segmentation rule may further include: responsive to the time interval between adjacent sentences in the merged final paragraph exceeding a preset time threshold, performing segmentation between adjacent sentences. Based on speaking habits, a long pause between adjacent sentences generally indicates a low degree of semantic relevance. Therefore, segmentation is performed if the time interval between adjacent sentences exceeds a preset time threshold (e.g., 10s, 15s, etc.).

[0067] In other implementations, the semantic similarity between adjacent sentences may also be considered. In some cases, if the semantic similarity between adjacent sentences exceeds a preset threshold, segmentation will not be performed even if the number of sentences in a paragraph has reached a preset threshold or the time interval between adjacent sentences has exceeded a preset threshold. This ensures semantic integrity and content coherence.

[0068] Step 307: Add the first text content as a new final paragraph to the recognized text to obtain the target recognized text.

[0069] In this embodiment, if it is determined that the first speaker information is inconsistent with the second speaker information, the first text content is added as a new end paragraph to the recognized text.

[0070] In some implementations, when the first text content is added as a new end paragraph to the recognized text, first speaker information and / or timestamp information corresponding to the first text content may also be added at the beginning of the new end paragraph.

[0071] In some implementations, the new end paragraph may be further segmented based on the first segmentation rule.

[0072] Step 308 : In response to detecting that the last paragraph of the recognized text is updated, refreshing and displaying the updated last paragraph.

[0073] In this embodiment, by updating the last paragraph of the recognized text, an updated recognized text, ie, the target recognized text, is obtained. At this point, the speech recognition and corresponding text processing operations for the first speech segment have been preliminarily completed.

[0074] As an exemplary embodiment, the recognized text can also be displayed. For example, the recognized text can be displayed to the user in real time on a display page, making it easier for the user to view the recognized text content. During the recording process, if the last paragraph of the recognized text is detected to have been updated, when the recognized text is refreshed, the displayed content before the last paragraph can be kept unchanged, and only the updated last paragraph can be refreshed and displayed. By refreshing and displaying the latest last paragraph, the target recognized text is displayed.

[0075] Among them, the above-mentioned strategy of partially refreshing only the content of the last paragraph of the recognized text can avoid page jitter caused by updating the recognized text with full data during the recording process, thereby improving the user experience.

[0076] Step 309: Determine whether the recording process is finished.

[0077] In this embodiment, based on step 308, it is determined whether the recording process has ended. If it is determined that the recording process has ended, step 310 is executed. Otherwise, the process returns to step 301 to continue processing the next acquired new voice segment.

[0078] Step 310: rematch the voiceprint features of the segments in the data storage list with the registered voiceprint features in the voiceprint dictionary.

[0079] In this embodiment, after the recording is completed, the voiceprint features of the segments in the data storage list are read in sequence and re-matched with the registered voiceprint features in the voiceprint dictionary.

[0080] In actual applications, when matching voiceprint features extracted from speech clips with registered voiceprint features in a voiceprint dictionary, the matching results may not be accurate. This is because, on the one hand, the extracted voiceprint features are affected by the quality of the voice data collection. For example, environmental noise and device distortion can reduce the reliability of voiceprint features, and the user's emotional fluctuations or health conditions (such as a cold) may also cause fluctuations or deviations in voiceprint features. On the other hand, matching algorithms based on voiceprint features may have misidentification errors. For example, when the sample data for speech recognition is small, it is difficult to generate a high-precision voiceprint feature code. Therefore, the accuracy of the matching depends on more voice input, and continuous learning and optimization are required to ensure the accuracy of the matching results.

[0081] Step 311: Update the speaker information corresponding to the text content in the target recognition text.

[0082] In this embodiment, based on the voiceprint features of the segments read from the data storage linked list, the speaker information corresponding to the registered voiceprint features in the voiceprint dictionary is re-matched, and the speaker information corresponding to the text content in the target recognition text is updated.

[0083] As mentioned above, in some cases, it may not be possible to successfully match the registered voiceprint features in the voiceprint dictionary, or the matching results may be inaccurate. By performing a secondary match and updating the speaker information corresponding to the text content, the integrity and accuracy of the speaker information corresponding to the text content can be further ensured.

[0084] Step 312: Segment the updated target recognition text based on the second segmentation rule and refresh the display.

[0085] In this embodiment, after completing the update of the speaker information corresponding to the text content in the target recognition text, the updated target recognition text is segmented based on the second segmentation rule; and the target recognition text after segmentation is refreshed and displayed as a whole, thereby realizing the display of the updated and segmented target recognition text.

[0086] Specifically, when updating the speaker information corresponding to the text content in the target recognition text, the speaker information of any inaccurate matching results will be updated. As a result, the speaker information corresponding to the text content in the updated target recognition text will change. Therefore, the text content in the updated target recognition text needs to be re-segmented based on certain segmentation rules.

[0087] In some exemplary embodiments, the second segmentation rule may include: for a paragraph in the updated target recognition text, in response to determining that the text content of the paragraph corresponds to multiple speaker information, splitting the paragraph into multiple paragraphs, wherein the text content of each of the split paragraphs corresponds to only one speaker information. For example, if the same paragraph includes text content corresponding to different speaker information, segmentation will be performed at the position where the speaker information changes.

[0088] In some exemplary embodiments, the second segmentation rule may also include: in response to determining that the split paragraph and the adjacent paragraph correspond to the same speaker information, merging the split paragraph and the adjacent paragraph into a new paragraph, and segmenting the new paragraph based on the first segmentation rule. For example, the updated target recognition text includes consecutive paragraphs 1, 2, and 3. Wherein paragraph 2 corresponds to 3 speakers, then paragraph 2 is split into 3 paragraphs, namely paragraph 2-1, paragraph 2-2, and paragraph 2-3, which correspond to different speakers respectively. In this case, it can be further determined whether paragraph 2-1 and its adjacent paragraph 1 correspond to the same speaker information. If so, paragraph 2-1 can be merged with paragraph 1, and the merged paragraph content can be segmented based on the aforementioned first segmentation rule. Similarly, the same operation can be performed on paragraphs 2-3 and paragraph 3.

[0089] The speech recognition method provided by the above-mentioned embodiments of the present disclosure updates only the text content of the last paragraph of the recognized text and partially refreshes the text content displayed on the display page each time the recognition processing of a new speech segment is completed during the recording process, and the entire text content is updated at the end of the recording process. This can reduce the memory pressure caused by the full data update during the recording process and the page jitter caused by the overall refresh of the display page, thereby improving the operation stability and enhancing the user experience. By associating the text content with the voiceprint features in a local data storage linked table and storing them, the voiceprint features of the segment in the data storage linked table are read at the end of the recording and re-matched with the registered voiceprint features in the voiceprint dictionary, which can further improve the accuracy of the matched speaker information. The speaker information corresponding to the text content in the target recognized text is updated based on the re-matched speaker information, thereby ensuring the consistency between the text content in the target recognized text and the corresponding speaker information, further improving the user experience.

[0090] To facilitate a deeper understanding of the technical solution of the present disclosure, the speech recognition method of an embodiment of the present disclosure will be exemplarily described below with reference to an application scenario.

[0091] refer to Figure 5 In an exemplary application scenario, the process 500 of the speech recognition method of the embodiment of the present disclosure includes the following steps: Step 501: During the recording process, the ASR SDK (Automatic Speech Recognition Software Development Kit) returns the recognition completion content (paragraph content) and the voiceprint sn code (voiceprint feature code).

[0092] Step 502: store the paragraph content and the voiceprint sn code into ContentList (data storage linked list).

[0093] In this step, the data storage linked list is stored in JSON format, and the data storage linked list may include paragraph content data and the voiceprint sn code associated with the paragraph content data.

[0094] Step 503: Write the data storage linked list into a local file as a data source.

[0095] Step 504: The ASR SDK returns the speaker per and voiceprint sn code array.

[0096] For example, the speaker per and voiceprint sn code arrays can use a key-value pair structure, where the key value corresponds to the voiceprint sn code and the value corresponds to the speaker per. The key-value structure can be used to configure parameters, transmit metadata, or process recognition results.

[0097] For example, the newly generated speaker per and voiceprint sn code array can be stored in the voiceprint storage dictionary PerDic, or the speaker per and voiceprint sn code array can be updated. The voiceprint storage dictionary PerDic is written to a local file in JSON format for easy access at any time.

[0098] Step 505: Get the last paragraph data of the data source.

[0099] In this step, the identified paragraph content in the data source is read from the local file, and the last paragraph data in the data source is obtained.

[0100] Step 506: Compare speaker information.

[0101] In this step, the current speaker per is matched from the voiceprint storage dictionary PerDic according to the voiceprint sn code returned in step 501, and the speaker information corresponding to the last paragraph data in the data source is determined, and the current speaker per is compared with the speaker information corresponding to the last paragraph data to see if they are consistent.

[0102] Step 507: Update the last paragraph data in the data source.

[0103] In this step, if it is determined that the speaker information corresponding to the last paragraph data is consistent with the current speaker per, the last paragraph data in the data source is updated. Specifically, the paragraph content content is added to the end of the last paragraph data in the data source and merged with it.

[0104] As an exemplary implementation, in this step, the last merged paragraph data may also be segmented. For details, please refer to the description in the aforementioned step 306, which will not be repeated here.

[0105] Step 508: insert a new paragraph data at the end of the data source.

[0106] In this step, if it is determined that the speaker information corresponding to the last paragraph data is inconsistent with the current speaker per, a new paragraph data is inserted at the end of the data source.

[0107] Step 509: Refresh the last paragraph data in the data source.

[0108] In this step, in response to detecting that the last paragraph data in the data source has been updated, the real-time recording page is partially refreshed to refresh the updated last paragraph data.

[0109] Step 510 , confirm whether the recording is completed, if so, execute step 511 , otherwise return to step 501 .

[0110] Step 511: Obtain the data storage linked list and voiceprint storage dictionary from the local file.

[0111] In this step, when the recording is completed, the ASR SDK reads the data storage list and voiceprint storage dictionary in the local file to obtain the paragraph content and the associated voiceprint sn code, as well as the speaker per and voiceprint sn code array.

[0112] Step 512: Update the data source and re-segment.

[0113] In this step, the associated voiceprint sn codes in the data storage linked list are traversed and matched against the speaker per and voiceprint sn code array in the voiceprint storage dictionary PerDic to obtain the corresponding speaker information. All paragraph data in the data source is updated and the updated paragraph data is re-segmented. The specific implementation of this step can be referred to the description of the above step 312 and will not be repeated here.

[0114] Step 513: Refresh the real-time recording page as a whole.

[0115] Refresh the real-time recording page as a whole based on the updated data source.

[0116] The speech recognition method of the above-mentioned embodiment of the present disclosure can reduce the memory pressure caused by the full data update during the recording process and the page jitter caused by the overall refresh of the display page, thereby improving the operation stability and enhancing the user experience; it can also improve the accuracy of the matched speaker information, further improving the user experience.

[0117] Further references Figure 6 As an implementation of the methods shown in the above figures, the present disclosure provides a speech recognition device. Figure 2 、 Figure 3 、 Figure 4 as well as Figure 5 The apparatus corresponds to the method embodiment or processing flow shown in , and can be specifically applied to a server, a terminal device or various other electronic devices.

[0118] like Figure 6 As shown, the speech recognition device 600 of this embodiment may include a speech recognition module 610 and a text processing module 620. The speech recognition module 610 is configured to determine a first text content and first speaker information corresponding to the first text content based on a first speech segment acquired during the recording process. The text processing module 620 is configured to determine second speaker information corresponding to the last paragraph in the recognized text; based on the first speaker information, the second speaker information, and the first text content, the last paragraph of the recognized text is updated to obtain a target recognized text, wherein the recognized text is determined based on the speech segment preceding the first speech segment during the recording process.

[0119] In some optional implementations of this embodiment, the speech recognition module 610 is further configured to perform speech recognition on the first speech segment to obtain the first text content; match the first voiceprint feature extracted from the first speech segment with the registered voiceprint features in the voiceprint dictionary; and determine the speaker information corresponding to the successfully matched registered voiceprint feature as the first speaker information corresponding to the first text content.

[0120] In some optional implementations of this embodiment, the speech recognition device 600 further includes a display module 630, which is configured to display the recognized text; and in response to detecting that the last paragraph of the recognized text is updated, refresh the display of the updated last paragraph.

[0121] In some optional implementations of this embodiment, the speech recognition module 610 is further configured to associate the first text content with the first voiceprint feature and store it in a local data storage linked list, wherein the data storage linked list includes the text content and the fragment voiceprint features obtained by recognizing multiple speech segments and stored in sequence during the recording process; the text processing module 620 is further configured to re-match the fragment voiceprint features in the data storage linked list with the registered voiceprint features in the voiceprint dictionary in response to the end of the recording process; update the speaker information corresponding to the text content in the target recognition text based on the speaker information re-matched with the fragment voiceprint features and the text content associated in the data storage linked list; and segment the updated target recognition text based on the second segmentation rule.

[0122] In this embodiment, other features of the speech recognition module 610, the text processing module 620 and the display module 630 in the speech recognition device 600 and the technical effects thereof can be referred to in the respective Figure 2 、 Figure 3 、 Figure 4 as well as Figure 5 The relevant descriptions corresponding to the method embodiments or processing flows shown in are not repeated here.

[0123] This embodiment exists as an apparatus embodiment corresponding to the above-mentioned method embodiment, and can reduce the memory pressure caused by the full data update during the recording process and the page jitter caused by the overall refresh of the display page, thereby improving the operation stability and enhancing the user experience; it can also further improve the accuracy of the matched speaker information, ensure the consistency of the text content in the target recognition text and the corresponding speaker information, and further enhance the user experience.

[0124] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can implement the speech recognition method described in any of the above embodiments when executing.

[0125] According to an embodiment of the present disclosure, the present disclosure further provides a readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to implement the speech recognition method described in any of the above embodiments when executed.

[0126] According to an embodiment of the present disclosure, the present disclosure further provides a computer program product, which, when executed by a processor, can implement the speech recognition method described in any of the above embodiments.

[0127] Next reference Figure 7 , Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0128] like Figure 7 As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. RAM 703 may also store various programs and data required for the operation of device 700. Computing unit 701, ROM 702, and RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to bus 704.

[0129] Various components in device 700 are connected to I / O interface 705, including an input unit 706, such as a keyboard, mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a magnetic disk, optical disk, etc.; and a communication unit 709, such as a network card, modem, wireless communication transceiver, etc. The communication unit 709 allows device 700 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0130] The computing unit 701 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as the speech recognition method. For example, in some embodiments, the speech recognition method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed onto the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the speech recognition method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the speech recognition method by any other suitable means (e.g., via firmware).

[0131] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0132] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0133] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0134] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0135] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0136] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers and establishing a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host. This is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and virtual private server (VPS) services.

[0137] According to the technical solution of the embodiment of the present disclosure, the memory pressure caused by the full data update during the recording process and the page jitter caused by the overall refresh of the display page can be reduced, thereby improving the stability of operation and enhancing the user experience; it can also further improve the accuracy of the matched speaker information, ensure the consistency of the text content in the target recognition text with the corresponding speaker information, and further enhance the user experience.

[0138] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0139] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A speech recognition method, comprising: Determining first text content and first speaker information corresponding to the first text content based on a first voice segment obtained during the recording process; Determining second speaker information corresponding to a last paragraph in the recognized text, wherein the recognized text is determined based on a speech segment preceding the first speech segment during the recording process; Based on the first speaker information, the second speaker information and the first text content, the last paragraph of the recognized text is updated to obtain a target recognized text.

2. The method according to claim 1, wherein The updating the last paragraph of the recognized text based on the first speaker information, the second speaker information, and the first text content to obtain a target recognized text includes: In response to determining that the first speaker information is consistent with the second speaker information, merging the first text content into the last paragraph of the recognized text; Based on the first segmentation rule, the merged last paragraph is segmented to obtain the target recognition text.

3. The method according to claim 2, wherein: The segmentation process of the merged last paragraph based on the first segmentation rule includes: In response to the number of sentences included in the merged final paragraph reaching a preset sentence number threshold, segmenting the merged final paragraph; and / or, In response to the time interval between adjacent sentences in the merged last paragraph exceeding a preset time threshold, segmentation processing is performed between the adjacent sentences.

4. The method according to claim 1, wherein updating the last paragraph of the recognized text based on the first speaker information, the second speaker information, and the first text content to obtain a target recognized text comprises: In response to determining that the first speaker information is inconsistent with the second speaker information, the first text content is added as a new end paragraph to the recognized text to obtain a target recognized text.

5. The method according to any one of claims 1 to 4, further comprising: displaying the recognized text; In response to detecting that the last paragraph of the recognized text is updated, the updated last paragraph is refreshed and displayed.

6. The method according to claim 1, wherein The determining of the first text content and the first speaker information corresponding to the first text content includes: Performing speech recognition on the first speech segment to obtain the first text content; matching the first voiceprint feature extracted from the first voice segment with the registered voiceprint features in the voiceprint dictionary; The speaker information corresponding to the successfully matched registered voiceprint feature is determined as the first speaker information corresponding to the first text content.

7. The method according to claim 1, wherein The target recognition text also includes speaker information corresponding to the text content.

8. The method according to claim 7, further comprising: storing the first text content and the first voiceprint feature in association with each other in a local data storage linked table, wherein the data storage linked table includes the text content and the segment voiceprint features sequentially stored during the recording process and obtained by recognizing the multiple voice segments; In response to the end of the recording process, re-matching the voiceprint features of the segments in the data storage linked list with the registered voiceprint features in the voiceprint dictionary; updating the speaker information corresponding to the text content in the target recognition text according to the speaker information re-matched by the voiceprint feature of the segment and the text content associated with the data stored in the linked table; Based on the second segmentation rule, the updated target recognition text is segmented.

9. The method according to claim 8, wherein The step of segmenting the updated target recognition text based on the second segmentation rule includes: For a paragraph in the updated target recognition text, in response to determining that the text content of the paragraph corresponds to multiple speaker information, splitting the paragraph into multiple paragraphs, wherein the text content of each of the split paragraphs corresponds to only one speaker information; In response to determining that the split paragraph and the adjacent paragraph correspond to the same speaker information, the split paragraph and the adjacent paragraph are merged into a new paragraph, and the new paragraph is segmented based on the first segmentation rule.

10. A speech recognition device, comprising: a speech recognition module configured to determine first text content and first speaker information corresponding to the first text content based on a first speech segment acquired during the recording process; a text processing module configured to determine second speaker information corresponding to the last paragraph in the recognized text; Based on the first speaker information, the second speaker information and the first text content, the last paragraph of the recognized text is updated to obtain a target recognized text, wherein the recognized text is determined based on a speech segment preceding the first speech segment during the recording process.

11. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable the computer to execute the method according to any one of claims 1-9.

13. A computer program product comprising a computer program which, when executed by a processor, implements the method according to claim 1 Any method described in 9.

Images