Voice data processing method, device and system, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a voice data processing method, device and system, electronic equipment and a computer readable storage medium, and relates to the technical field of computers, in particular to the technical fields of voice recognition, voice transcription, cloud storage and the like, the method is applied to a business server side, and the method comprises the steps that an audio data block sent by a browser side is received; in response to the fact that the audio data block is not marked as retransmission data, sending the audio data block to a cloud server for voice transcription processing; and sending the voice transcription result returned by the cloud server to the browser side. According to the method, the audio data blocks transmitted to the cloud server in real time can be ensured, so that the real-time voice transfer requirement of a user is met.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, specifically to the fields of speech recognition, speech transcription, and cloud storage, and particularly to a speech data processing method, apparatus, system, electronic device, and storage medium. Background Technology

[0002] With the continuous development of technologies such as natural language processing and deep learning, automatic speech recognition and speech-to-text technology are now widely used in various scenarios such as meeting recording, remote learning and voice assistants to achieve the function of recording and transcribing. Summary of the Invention

[0003] This disclosure provides a voice data processing method, apparatus, system, electronic device, and storage medium.

[0004] In a first aspect, embodiments of this disclosure propose a voice data processing method applied to a business server. The method includes: receiving an audio data block sent by a browser; in response to the audio data block not being marked as retransmission data, sending the audio data block to a cloud server for speech transcription processing; and sending the speech transcription result returned by the cloud server to the browser.

[0005] Secondly, this disclosure also proposes a voice data processing method applied to a browser. The method includes: dividing the collected recording data into multiple audio data blocks and storing them in a local database; reading the audio data blocks from the local database and sending them to a service server; and, in response to determining that the audio data block transmission failed, marking the audio data block as retransmission data and sending it to the service server again.

[0006] Thirdly, this disclosure also proposes a voice data processing method applied to a system, which includes a browser and a service server. The method includes: the browser dividing the collected recording data into multiple audio data blocks and storing them in a local database; reading the audio data blocks from the local database and sending them to the service server; in response to determining that the audio data block failed to be sent, marking the audio data block as retransmission data and sending it to the service server again; the service server receiving the audio data block sent by the browser; in response that the audio data block was not marked as retransmission data, sending the audio data block to a cloud server for speech-to-text processing; and sending the speech-to-text result returned by the cloud server to the browser.

[0007] Fourthly, embodiments of this disclosure provide an apparatus for use in a browser, the apparatus comprising: a acquisition module configured to divide acquired recording data into multiple audio data blocks; a first storage module configured to store the audio data blocks in a local database; a first processing module configured to read the audio data blocks from the local database and send them to a service server; and, in response to determining that the audio data block transmission has failed, marking the audio data block as retransmission data and sending it to the service server again.

[0008] Fifthly, this disclosure also proposes an apparatus for use on a business server, the apparatus comprising: a second processing module configured to receive an audio data block sent by a browser; in response to the audio data block not being marked as retransmission data, sending the audio data block to a cloud server for speech-to-text processing; and sending the speech-to-text result returned by the cloud server to the browser; and a second storage module configured to append and save the audio data block to the data object in response to the audio data block corresponding to the same speech processing task as a stored data object.

[0009] Sixthly, embodiments of this disclosure propose a system including a browser and a service server. The browser is configured to segment collected recording data into multiple audio data blocks and store them in a local database, read audio data blocks from the local database and send them to the service server. In response to determining that the audio data block transmission failed, the browser marks the audio data block as retransmission data and sends it to the service server again. The service server is configured to receive the audio data blocks sent by the browser, and in response that the audio data block is not marked as retransmission data, send the audio data block to a cloud server for speech-to-text processing, and send the speech-to-text result returned by the cloud server to the browser.

[0010] In a seventh aspect, embodiments of this disclosure provide an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement the voice data processing method as described in any implementation of the first aspect.

[0011] Eighthly, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions that enable a computer to implement the voice data processing method as described in any implementation of the first aspect.

[0012] The voice data processing method, apparatus, and system provided in this disclosure allow the service server to determine whether a received audio data block is marked as retransmitted data, thus confirming whether the audio data block is real-time transmitted data. The real-time transmitted audio data block is then sent to a cloud server for speech-to-text processing, ensuring that the data sent to the cloud server is real-time transmitted and meeting the user's real-time speech-to-text needs.

[0013] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0014] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is an exemplary system architecture to which this disclosure can be applied; Figure 2 A flowchart of a voice data processing method provided in an embodiment of this disclosure; Figure 3 A flowchart of a voice data processing method provided in another embodiment of this disclosure; Figure 4 A flowchart of a voice data processing method provided in another embodiment of this disclosure; Figure 5 A flowchart of a voice data processing method provided in another embodiment of this disclosure; Figure 6A , Figure 6B and Figure 6C A flowchart of a voice data processing method provided in another embodiment of this disclosure; Figure 7 A flowchart of a voice data processing method in a specific application scenario provided by an embodiment of this disclosure; Figure 8 A structural block diagram of the voice data processing apparatus and system provided in the embodiments of this disclosure; Figure 9 This is a schematic diagram of the structure of an electronic device suitable for performing a voice data processing method, provided as an embodiment of the present disclosure. Detailed Implementation

[0015] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0016] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information in this technical solution comply with relevant laws and regulations and do not violate public order and good morals. In this technical solution, user authorization or consent is obtained before acquiring or collecting user personal information.

[0017] Figure 1 An exemplary system architecture 100 is shown for a voice data processing method and apparatus to which embodiments of the present disclosure may be applied.

[0018] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0019] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Terminal devices 101, 102, and 103 can be various electronic devices with displays that support receiving messages from the server, including but not limited to smartphones, tablets, e-book readers, electronic players, laptops, and desktop computers, etc.

[0020] Terminal devices 101, 102, and 103 can be either hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices. When terminal devices 101, 102, and 103 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software programs or software modules (e.g., multiple software modules used to provide distributed services), or as a single software program or software module, such as a client / server application or a web browser application. No specific limitations are made here.

[0021] Server 105 can be a server providing various services, such as a backend server that receives requests sent by terminal devices with which it has established communication connections. The backend server can receive and analyze the requests sent by the terminal devices, and generate processing results to feed back to the terminal devices (e.g., performing speech recognition or speech-to-text processing on received and collected voice data to generate target text corresponding to the voice data). The server can be hardware or software. When the server is hardware, it can be various electronic devices providing various services to the terminal devices. When the server is software, it can be implemented as multiple software programs or software modules providing various services to the terminal devices, or it can be implemented as a single software program or software module providing various services to the terminal devices. No specific limitations are made here.

[0022] Server 105 can provide various services through its built-in applications. Taking an application that provides voice data processing services as an example, when running this application, server 105 can achieve the following: First, it receives audio data blocks sent by the browser; then, in response to the audio data blocks not being marked as retransmission data, it sends the audio data blocks to the cloud server for speech-to-text processing; finally, it sends the speech-to-text result returned by the cloud server to the browser. Alternatively, it can achieve the following: First, it divides the collected recording data into multiple audio data blocks and stores them in a local database; it reads the audio data blocks from the local database and sends them to the business server; in response to determining that the audio data block transmission failed, it marks the audio data block as retransmission data and sends it to the business server again.

[0023] The voice data processing methods provided in the subsequent embodiments of this disclosure are generally executed by server 105, and correspondingly, the voice data processing device is generally also disposed in server 105. However, it should also be noted that when terminal devices 101, 102, and 103 also have sufficient computing power and resources, the voice data processing methods can also be executed by terminal devices 101, 102, and 103. Correspondingly, the voice data processing device can also be disposed in terminal devices 101, 102, and 103.

[0024] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, there can be any number of terminal devices, networks, and servers.

[0025] refer to Figure 2 , Figure 2 The flowchart of a voice data processing method according to an embodiment of the present disclosure is shown. This flowchart can be applied to a business server, which can be various application layer servers or middleware servers, or business applications, etc., without specific limitations. The flowchart 200 may include the following steps: Step 201: Receive audio data blocks sent by the browser.

[0026] This step is intended for the entity executing the speech data processing method (e.g., Figure 1 The server 105 or other devices in the browser receive audio data blocks sent by the browser.

[0027] In some implementations, audio data blocks may include user voice data collected by the browser. During recording in applications such as meetings, the browser can collect the user's voice data in real time using recording devices such as microphones or browser plugins, and send it to the service server in the form of audio data blocks. Each audio data block can be a continuous audio stream.

[0028] In some implementations, the browser can send audio data blocks to the service server based on RTMP (Real Time Messaging Protocol) or RTSP (Real Time Streaming Protocol).

[0029] Step 202: In response to the audio data block not being marked as retransmitted data, the audio data block is sent to the cloud server for speech transcription processing.

[0030] This step aims to determine whether the received audio data block is marked as retransmission data by the aforementioned executing entity. If it is not marked as retransmission data, the audio data block is sent to the cloud server. The cloud server can provide various ways to process the audio data, such as performing speech recognition or speech transcription on the speech data contained in the audio data block, thereby obtaining the speech transcription result, such as the target text corresponding to the speech data.

[0031] In some implementations, the business server can call the API (Application Programming Interface) of the speech recognition engine located on the cloud server to perform online speech-to-text processing on the speech data contained in the audio data block and generate the corresponding target text.

[0032] In other implementations, the business server can also perform offline speech transcription of audio data blocks using internally integrated or locally downloaded professional voice tools, such as a voice SDK (Software Development Kit).

[0033] Step 203: Send the speech-to-text results returned by the cloud server to the browser.

[0034] This step is intended for the aforementioned executing entity to send the speech-to-text result, i.e., the target text, returned by the cloud server to the browser.

[0035] In some implementations, the aforementioned execution entity can send the speech transcription results to the browser based on the RTMP or RTSP protocol.

[0036] The voice data processing method of the above embodiments of this disclosure first receives an audio data block sent by a browser; then, in response to the audio data block not being marked as retransmission data, the audio data block is sent to a cloud server for speech-to-text processing; finally, the speech-to-text result returned by the cloud server is sent to the browser. This method, by determining whether the received audio data block sent by the browser is marked as retransmission data, can confirm whether the audio data block is real-time transmitted data, and then send the real-time transmitted audio data block to the cloud server for speech-to-text processing, thereby ensuring that the audio data block sent to the cloud server is real-time transmitted, meeting the user's real-time speech-to-text needs.

[0037] Continue to refer to Figure 3 , Figure 3 A flowchart of a voice data processing method according to another embodiment of this disclosure is shown, which can be applied to a browser. For example... Figure 3 As shown, the process 300 may include the following steps: Step 301: Divide the collected recording data into multiple audio data blocks and store them in the local database.

[0038] This step is intended for the entity executing the speech data processing method (e.g., Figure 1 The server (105) collects the user's voice input during the recording process to obtain recording data, divides the recording data into multiple audio data blocks, and stores the audio data blocks in a local database. For example, a local file can be created for this recording task, and the audio data blocks for this recording task can be stored in this local file.

[0039] In some implementations, the aforementioned execution entity may perform time-stamping processing on each audio data block so that the audio data block carries corresponding time-stamping information, such as the start timestamp of the audio data block.

[0040] In some alternative implementations, the aforementioned execution entity may also associate audio data blocks with corresponding time stamp information and store them in a local file.

[0041] Step 302: Read audio data blocks from the local database and send them to the business server.

[0042] In this embodiment, the aforementioned execution entity reads the audio data block to be processed from the local database and sends it to the business server.

[0043] In some implementations, the aforementioned execution entity can read audio data blocks from the local database in chronological order based on multiple audio data blocks and corresponding time stamp information stored in the local database, and can send them to the service server based on the RTMP or RTSP protocol.

[0044] Step 303: In response to the determination that the audio data block failed to be sent, the audio data block is marked as retransmission data and sent to the service server again.

[0045] In this embodiment, when the execution entity determines that the audio data block has failed to be sent, it marks the failed audio data block as retransmission data and can send it to the service server again immediately or after waiting for a certain time interval.

[0046] In some implementations, the aforementioned executing entity can determine that the audio data block to be sent within a predetermined time window corresponding to the event has failed to be transmitted when it detects events such as abnormal network connection or physical layer failure.

[0047] The voice data processing method provided in the above embodiments of this disclosure first divides the collected recording data into multiple audio data blocks and stores them in a local database; then, it reads the audio data blocks from the local database and sends them to the service server; finally, in response to determining that the audio data block transmission failed, it marks the audio data block as retransmission data and sends it to the service server again. In related technologies, audio data collected during recording is usually only temporarily stored in the browser's memory. When the browser tab is closed, the page is refreshed, or the browser itself crashes due to a malfunction, the audio data in memory will be permanently lost. This method, by storing the audio data blocks in a local database, can effectively avoid the risk of data loss and ensure the continuity and integrity of the original audio data. In addition, when it is determined that the audio data block sent by the browser failed to be transmitted, the failed audio data block can be marked as retransmission data, and the original audio data stored in the local database can be used for backtracking. For example, the audio data block can be found based on its timestamp and sent to the service server again, thereby improving the continuity and reliability of audio data block transmission.

[0048] Further reference Figure 4 , Figure 4 A flowchart of a voice data processing method according to another embodiment of this disclosure is shown. This flowchart can be applied to a system that may include a browser and a service server. Figure 4 As shown, the process 400 may include the following steps: Step 401: The browser divides the collected recording data into multiple audio data blocks and stores them in the local database.

[0049] In this step, the browser divides the recording data collected during the recording process into multiple continuous audio data blocks and stores the audio data blocks in a local database. This avoids data loss due to network instability or the user closing the browser tab, and can meet the needs of data retrieval and retransmission.

[0050] Step 402: The browser reads audio data blocks from the local database and sends them to the business server.

[0051] In this step, the browser can read the audio data block and its corresponding timestamp information from the local database according to the timestamp information of the audio data block, such as the start timestamp, and send it to the business server.

[0052] In some implementations, the browser can package the read audio data blocks and send them to the service server based on the RTMP or RTSP protocol.

[0053] As an exemplary implementation, after the browser sends the audio data block to the service server, it may further include: in response to determining that the audio data block failed to be sent, marking the audio data block as retransmission data and sending it to the service server again.

[0054] Step 403: The business server receives the audio data block sent by the browser.

[0055] In some implementations, the service server receives data packets sent by the browser, parses the data packets, and obtains audio data blocks.

[0056] Step 404: In response to the audio data block not being marked as retransmitted data, the business server sends the audio data block to the cloud server for speech transcription processing.

[0057] If the business server determines that the received audio data block has not been marked as retransmission data, it will send the audio data block to the cloud server for speech-to-text processing.

[0058] In some implementations, the service server can determine whether an audio data block is a retransmitted data block based on the timestamp information of the received audio data block. For example, if the timestamp information of the audio data block is earlier than that of other received audio data blocks and the difference between the timestamp information of the two exceeds a predetermined time threshold, then the audio data block can be determined to be a retransmitted data block.

[0059] Step 405: The business server sends the speech-to-text result returned by the cloud server to the browser.

[0060] In this step, when the business server receives the speech-to-text result returned by the cloud server, it directly sends the speech-to-text result to the browser.

[0061] The voice data processing method provided in the above embodiments of this disclosure enables direct recording and transcription on the browser side, thereby meeting the recording and transcription needs in various scenarios such as online collaboration and remote learning. During the recording process, the browser stores audio data blocks in a local database, effectively avoiding the risk of data loss and ensuring the continuity and integrity of the original audio data. The browser reads audio data blocks from the local database and sends them to the business server. If transmission fails, the audio data block is marked as retransmission data. The business server can determine whether the received audio data block is real-time transmitted data by checking whether it is marked as retransmission data. Then, the real-time transmitted audio data block is sent to the cloud server for voice transcription processing, thus ensuring that the audio data block sent to the cloud server is real-time transmitted, meeting the user's real-time voice transcription needs.

[0062] Continue to refer to Figure 5 , Figure 5 The flowchart of a voice data processing method applied to a browser according to another embodiment of this disclosure is shown. Figure 5 As shown, the process 500 may include the following steps: Step 501: Divide the collected recording data into multiple audio data blocks and mark each audio data block with a corresponding identifier number.

[0063] The browser divides the voice data collected during recording into multiple audio data blocks at predetermined time intervals and assigns a corresponding identifier number to each audio data block. For example, the identifier numbers can be incremented according to the order in which the audio data blocks were generated.

[0064] Step 502: Store the audio data block and its corresponding identifier number in the local database.

[0065] As an example implementation, the browser can create a corresponding local database for the recording task when the recording operation is started, and store the audio data blocks of the same recording task together with the identification sequence number in the local database.

[0066] In some exemplary implementations, the browser can establish a WebSocket connection with the service server when recording is initiated. A WebSocket connection is a long-term, bidirectional, real-time communication channel established between a client (such as a browser) and a server (such as a service server). It can solve problems such as HTTP's inability to achieve server-initiated push and continuous low-latency communication.

[0067] Step 503: Read the audio data block and its corresponding identifier from the local database and send it to the business server.

[0068] As an exemplary implementation, the browser reads audio data blocks from the local database in ascending order of the identifier number, and encapsulates at least one audio data block into a data frame, which includes at least one audio data block and its corresponding identifier number.

[0069] Accordingly, when the business server successfully receives a data frame, it can send an acknowledgment signal to the browser, for example, by sending an acknowledgment frame, which includes the identifier sequence number in the data frame.

[0070] Step 504: In response to not receiving an acknowledgment signal and identification sequence number from the service server within the preset transmission time, it is determined that the audio data block transmission has failed.

[0071] As an example implementation, the browser can start a timer while sending a data frame. If no acknowledgment signal and identification number are received from the service server within the preset transmission time, it can be determined that the audio data block in the data frame has failed to be sent.

[0072] Step 505: Mark the audio data block as retransmission data and send it to the service server again.

[0073] As an example implementation, when the browser determines that the audio data block has failed to be sent, it can mark the audio data block as retransmitted data and send it again immediately or after a predetermined time interval.

[0074] In some implementations, if it is determined that the communication link between the browser and the service server is broken or abnormal, such as failing to receive an acknowledgment signal after several retries, the retransmission is abandoned and marked as an error, and the transmission can be attempted again after the communication link is restored.

[0075] Next reference Figure 6A , Figure 6A The flowchart illustrates the voice data processing method applied to the business server. For example... Figure 6AAs shown, the process 600 may include the following steps: Step 601: Receive audio data blocks sent by the browser.

[0076] Step 602: Determine whether the audio data block is marked as retransmitted data. If it is determined that it is marked as retransmitted data, continue to step 603; if it is not marked as retransmitted data, continue to step 603 and then proceed to step 607.

[0077] In this embodiment, when the service server determines that the audio data block has not been marked as retransmission data, it will no longer send it to the cloud server for speech transcription processing, thereby ensuring that the data sent to the cloud server is real-time data.

[0078] Step 603: Determine whether it corresponds to the same speech processing task as the stored data object. If yes, proceed to step 604; otherwise, proceed to step 605.

[0079] In this embodiment, the service server will append storage to the received audio data blocks, for example, it can store them as local data objects, or it can store them remotely, without any specific restrictions.

[0080] In some implementations, the browser may include a recording identifier when sending audio data blocks. This identifier indicates the speech processing task corresponding to the current audio data block. The service server can determine whether the audio data block corresponds to the same speech processing task as a stored data object based on the recording identifier.

[0081] Step 604: Append the audio data block to the data object.

[0082] If the business server determines that the audio data block corresponds to the same voice processing task as the stored data object, it will append and save the audio data block to the data object.

[0083] Step 605: Store the audio data block into the newly created data object.

[0084] If the business server cannot find a stored data object corresponding to the same voice processing task, it can create a data object based on the recording identifier of the audio data block and store the audio data block in the newly created data object.

[0085] Step 606: Transfer the data object to the target user's cloud storage space.

[0086] As an example implementation, the service server may, after appending and saving the received audio data block to the data object, or after determining that the current voice processing task has been completed, transfer the entire stored data object to the target user's cloud storage space, such as the user's personal cloud storage device, where the target user includes at least the user who initiated the voice processing task.

[0087] Step 607: Send the audio data block to the cloud server for speech transcription processing.

[0088] For audio data blocks not marked as retransmission data, the business server can directly send them to the cloud server for speech-to-text processing. For example, they can be sent to the cloud-based ASR (Automatic Speech Recognition) engine for speech-to-text processing.

[0089] In an exemplary implementation, the service server can also preprocess the audio data blocks before sending them to the cloud server for speech-to-text processing. For example, noise reduction, sentence segmentation detection, and speech segment aggregation can be performed on the audio data blocks to effectively filter out silence and noise, and intelligent sentence segmentation can be performed based on speech activity, thereby significantly improving the accuracy of the speech-to-text results and the readability of the final document. The specific implementation of preprocessing will be described in further detail below.

[0090] Step 608: Send the speech-to-text result returned by the cloud server to the browser.

[0091] In this embodiment, the service server only sends audio data blocks that are not marked as retransmission data to the cloud server for transcription processing, while audio data blocks that are not marked as retransmission data are not sent again. This can better meet the real-time requirements of speech transcription and improve the user experience.

[0092] Combination Figure 6B As shown, Figure 6B The processing flow of step 601 described above according to an exemplary embodiment is illustrated. In this exemplary embodiment, step 601 may further include the following steps: Step 611: Receive the audio data block and corresponding identifier number sent by the browser through the first communication link.

[0093] The service server can receive audio data blocks and corresponding identifiers (e.g., data frames) sent by the browser through the first communication link with the browser, such as a WebSocket connection.

[0094] A business server can typically run multiple service instances. The browser can establish a first communication link with the business server through the first service instance running on the business server.

[0095] Step 612: In response to confirming successful reception of the audio data block, an acknowledgment signal and identification number are sent to the browser via the first communication link.

[0096] As an exemplary implementation, the service server parses at least one audio data block and its corresponding identifier from the data frame, and sends a confirmation frame to the browser through the first communication link. The confirmation frame includes the corresponding identifier.

[0097] Step 613: In response to the pending update of the first service instance, an update notification is sent to the browser.

[0098] As an exemplary implementation, when the business server needs to perform rolling updates on the service instance, it sends an update notification to the browser. For example, it can send a control frame to notify the browser to exit the first communication link of the current connection.

[0099] Step 614: In response to the first communication link being disconnected and all currently received audio data blocks having been stored, exit and update the first service instance.

[0100] After the business server notifies the browser to disconnect the first communication link, it stores all currently received audio data blocks in the cache, for example, by appending and saving them to the already stored data object. After the storage is completed, it exits the first service instance and performs an update operation on the first service instance.

[0101] Step 615: In response to receiving a reconnection request from the browser, establish a second communication link with the browser through the second service instance that has not logged out.

[0102] The business server establishes a new connection with the browser through a second service instance that has not logged out, i.e., a second communication link, based on the reconnection request initiated by the browser.

[0103] Step 616: Receive the audio data block and corresponding identifier number sent by the browser through the second communication link.

[0104] After the business server establishes a second communication link with the browser through the second service instance that has not been logged out, it can continue to receive data frames sent by the browser through the second communication link.

[0105] Combination Figure 6C As shown, Figure 6C The preprocessing flow in step 607 above, according to an exemplary embodiment, is illustrated. In this exemplary embodiment, step 607 may further include the following steps: Step 671: Perform speech activity detection on the audio data block to determine the start and end points of valid speech segments.

[0106] As mentioned above, before sending audio data blocks to the cloud server, the business server can perform intelligent sentence segmentation, noise reduction, and speech segment aggregation on the audio data blocks, so that the data sent to the cloud server is optimized and effective speech data, thereby achieving a leap in transcription quality.

[0107] As an example implementation, a voice SDK including a VAD (Voice Activity Detection) model can be integrated into the business server. Before sending audio data blocks to the ASR engine in the cloud, the VAD model in the voice SDK is used to analyze and determine the start and end points of valid voice segments, such as the start point of the voice segment and the silence pause point.

[0108] Step 672: Based on the determined start and end points of the valid speech segments, filter out invalid speech segments in the audio data block to obtain valid speech segments.

[0109] Based on the start and end points of the valid speech segments identified in step 671, invalid speech segments (such as silent segments and noise segments) and valid speech segments in the audio stream can be distinguished. By filtering out the invalid speech segments, the valid speech segments are obtained.

[0110] Step 673: The valid speech segments are aggregated sequentially to obtain the aggregated speech data.

[0111] The valid speech segments obtained in step 672 are aggregated and merged into complete speech segments to ensure the coherence and semantic integrity of the speech data output to the cloud server.

[0112] Step 674: Send the aggregated voice data to the cloud server for voice transcription processing.

[0113] The aggregated voice data obtained in step 673 is sent to a cloud server, such as an ASR engine in the cloud, for voice transcription processing.

[0114] The voice data processing method provided in the above embodiments of this disclosure allows the service server to back up the recorded data by appending the received audio data blocks to the stored data object, thus ensuring the security and integrity of the recorded data. When the service server needs to update the running service instance, it can send a control frame to the browser through a graceful exit protocol to notify the browser to disconnect the currently connected first service instance and re-establish the connection through the second service instance that has not yet exited. This ensures that the update of the service server will not affect the user's real-time transcription needs. In addition, by preprocessing the received audio data blocks on the service server, the voice data sent to the cloud ASR engine is optimized, which not only improves the processing efficiency of the ASR engine but also significantly improves the quality of speech transcription and saves network bandwidth occupied by transmitting invalid voice segments, thereby further reducing latency.

[0115] The following section will further elaborate on the voice data processing method provided in this disclosure, taking into account specific application scenarios.

[0116] refer to Figure 7 , Figure 7 The flowchart illustrates a voice data processing method in an exemplary application scenario. The system architecture in this application scenario may include a browser-based client and a business server-side client.

[0117] The browser can be a client, which may include a recording module 710, an audio processing module 711, and a local database 712. The local database could be, for example, IndexedDB (Indexed Database), which allows for the storage of large amounts of structured data within the browser and enables high-performance retrieval and read / write operations.

[0118] The business server can reside in the middle service layer. The business server includes a data transmission module 720, a voice SDK 721, and an object storage module 722. The data transmission module 720 can be a WebSocket endpoint established with the browser, receiving and parsing audio data blocks sent by the browser. The object storage module 722 can store data locally as data objects or store the data in the cloud. For example, the object storage module 722 can be a cloud object storage module, providing cloud-based unstructured data storage services.

[0119] Combination Figure 7 As shown, in this application scenario, the voice data processing method flow 700 may include the following steps: Step 1001: Collect audio data and process it in blocks.

[0120] The client initiates recording via the recording module 710 and establishes a WebSocket connection with the business server. The client also creates a persistent local database for this recording using IndexedDB.

[0121] During the recording process, the audio processing module 711 can collect recording data by calling the MediaRecorder API, divide the recording data into audio data chunks at fixed time intervals (e.g., 160ms), and assign a corresponding identifier number (chunk_id) to each audio data chunk. For example, chunk_ids can be assigned starting from 0 and incrementing.

[0122] Step 1002: Store the audio data block in the local database.

[0123] The client immediately writes the segmented audio data blocks and their corresponding chunk_ids into the IndexedDB "pending transmission queue" for persistent storage.

[0124] Step 1003: Read audio data blocks from the local database and send them to the business server.

[0125] The client reads the audio data block with the smallest chunk_id that has not yet received an acknowledgment signal from IndexedDB, encapsulates it in a custom data frame, and sends it. This data frame may include metadata such as session_id (recording identifier) ​​and chunk_id. Then, the encapsulated data frame is sent to the business server's WebSocket Endpoint through the WebSocket connection established with the business server.

[0126] As an exemplary implementation, the client can start a timeout timer (e.g., 5 seconds) corresponding to the chunk_id when sending a data frame. Upon receiving an acknowledgment signal (e.g., an acknowledgment frame) from the service server, the client can clear the corresponding timeout timer. The audio data block can then be marked as "acknowledged" in the IndexedDB, and the client can continue sending the next audio data block. If the timeout timer expires without receiving an acknowledgment frame, the audio data block can be marked as "retransmitted data" in the IndexedDB, and the corresponding audio data block can be retransmitted to the service server according to the preset maximum number of retransmissions.

[0127] In some implementations, if a WebSocket connection is unexpectedly dropped, the client can attempt to reconnect by sending a reconnection request. Once the reconnection is successful, the client will query IndexedDB and, starting with the audio data block with the smallest chunk_id marked as "retransmit data," will retransmit all unacknowledged data blocks sequentially, thus achieving seamless resume functionality.

[0128] Step 1004: The business server sends the audio data block to the voice SDK for preprocessing.

[0129] When the data transmission unit on the business server receives a data frame, it parses it. If the parsing is successful, the parsed audio data block is sent to the internally integrated voice SDK for preprocessing. For example, noise reduction, intelligent sentence segmentation, and speech segment aggregation are performed using the VAD model integrated in the voice SDK to obtain preprocessed voice data.

[0130] Step 1005: The business server appends the audio data block to the data object.

[0131] When the data transmission module successfully parses the audio data block, it can also call the AppendObject API of the object storage module to append all the original audio data blocks received during this period to the stored data object at a fixed high frequency (such as every 4 seconds), thereby achieving near real-time cloud backup of the recording data and providing core protection for data security.

[0132] Step 1006: The business server sends an acknowledgment frame to the browser.

[0133] After successfully parsing the audio data block and completing the append writing of the audio data block, the data transmission module can send an ACK (acknowledgment frame) to the browser, and the acknowledgment frame includes the identifier number (chunk_id) corresponding to the acknowledged audio data block.

[0134] Step 1007: Send the preprocessed voice data to the cloud-based ASR engine for voice transcription processing.

[0135] The audio data blocks are preprocessed by the VAD model in the speech SDK to filter out invalid silence and noise segments, and continuous speech segments are aggregated into complete "speech units". The preprocessed high-quality "speech units" are sent to the cloud ASR engine to generate the corresponding target text.

[0136] Step 1008: Send the speech-to-text results returned by the cloud-based ASR engine to the browser.

[0137] After receiving the target text generated by the ASR engine, the business server returns it to the client in real time via a WebSocket connection for rendering, thus obtaining the final transcribed text.

[0138] Optionally, upon determining the end of the recording process, the business server may asynchronously transcode and transfer the data object stored in the object storage module, i.e., the complete original audio file, to the target user's cloud storage space, where the target user includes at least the user who initiated the recording.

[0139] The voice data processing method provided in the above embodiments of this disclosure has the following characteristics: First, it achieves high reliability and integrity in data transmission.

[0140] Building upon WebSocket connections, an innovative application-layer protocol with ACK confirmation, timeout retransmission, and breakpoint resumption mechanisms was designed and implemented. This upgrades the WebSocket connection from a fire-and-forget, best-effort "stream" channel to a reliable, acknowledgment- and retryable "message" channel. This ensures zero data loss in the client-server transmission link under harsh conditions such as network jitter or brief interruptions, fundamentally guaranteeing 100% integrity of cloud-recorded data.

[0141] Second, it significantly improves transcription accuracy and text quality.

[0142] By introducing an intermediate service layer with intelligent processing capabilities into the core architecture, the "black box" dependency of the general-purpose speech engine API that directly connects to the cloud in traditional browser-side solutions is eliminated.

[0143] In the traditional architecture of directly connecting to a general cloud-based ASR API, the application service layer only acts as a data proxy and cannot preprocess or optimize the audio stream. This results in non-speech segments such as silence and ambient noise being indiscriminately sent to the ASR engine, wasting expensive recognition computing power and easily causing the engine to misidentify noise as text or incorrectly segment sentences at natural pauses in speech, severely affecting the accuracy and fluency of the final transcript.

[0144] In contrast, this disclosure, through deep integration of a speech SDK with a VAD model on the business server side, enables precise preprocessing of the audio stream, effectively filtering out silence and noise, and intelligently segmenting sentences based on speech activity. This ensures that the audio segments fed into the ASR engine are "cleaner" and more in line with natural language units, thereby significantly improving the accuracy of the transcription results and the readability of the final document.

[0145] Third, a dual data security system has been built, from the client to the cloud.

[0146] On the one hand, local persistence via the client-side IndexedDB effectively combats unexpected events such as browser refreshes and crashes. On the other hand, frequent appending and writing of data objects on the business server enables near real-time cloud backup. This dual-insurance mechanism of "local pre-storage + real-time cloud backup" provides end-to-cloud dual persistence, thereby solving the fatal flaw of traditional solutions where data is only stored in memory and is extremely easy to lose, ensuring that users' recording data can be preserved to the maximum extent under any abnormal circumstances.

[0147] refer to Figure 8 As an implementation of the methods shown in the above figures, this disclosure provides corresponding voice data processing apparatus and system, which can be used as a... Figures 2 to 7 The voice data processing apparatus and system corresponding to the method embodiments or processing flow shown.

[0148] The voice data processing system 800 of this embodiment may include a browser and a service server. The browser is configured to segment the collected recording data into multiple audio data blocks and store them in a local database, read the audio data blocks from the local database and send them to the service server. In response to a failure to send an audio data block, the browser marks the audio data block as retransmission data and sends it to the service server again. The service server is configured to receive the audio data blocks sent by the browser, and in response to an audio data block not being marked as retransmission data, send the audio data block to a cloud server for speech-to-text processing, and send the speech-to-text result returned by the cloud server back to the browser.

[0149] As an exemplary embodiment, the voice data processing system 800 may further include a first voice data processing device 810 and a second voice data processing device 820. The first voice data processing device 810 and the second voice data processing device 820 can be applied to browsers, servers, terminal devices, or various other electronic devices.

[0150] As an exemplary implementation, the first voice processing device 810 can be applied to a browser and includes a data acquisition module 811, a first storage module 812, and a first processing module 813. The data acquisition module 810 is configured to divide the acquired recording data into multiple audio data blocks; the first storage module 811 is configured to store the audio data blocks in a local database; the first processing module 812 is configured to read the audio data blocks from the local database and send them to a service server, and in response to determining that the audio data block transmission failed, to mark the audio data block as retransmission data and send it to the service server again.

[0151] In some optional implementations of this embodiment, the acquisition module 811 is further configured to: divide the acquired recording data into multiple audio data blocks, and mark each audio data block with a corresponding identifier number. The first storage module 811 is further configured to store the audio data blocks and their corresponding identifier numbers in a local database. The first processing module 812 is further configured to read the audio data blocks and their corresponding identifier numbers from the local database and send them to the service server; in response to not receiving an acknowledgment signal and identifier number from the service server within a preset transmission time, it is determined that the audio data block transmission has failed.

[0152] In some optional implementations of this embodiment, the first processing module 812 is further configured to: in response to receiving an update notification from the service server, disconnect the first communication link currently running with the first service instance of the service server; send a reconnection request to the service server to establish a second communication link with the second service instance of the service server that has not exited; and send an audio data block and the corresponding identification number to the service server based on the second communication link.

[0153] As an exemplary implementation, the second voice processing device 820 can be applied to a business server, including a second processing module 821 and a second storage module 822. The second processing module 821 is configured to receive audio data blocks sent by a browser; in response to the audio data block not being marked as retransmission data, send the audio data block to a cloud server for speech-to-text processing; and send the speech-to-text result returned by the cloud server to the browser. The second storage module 822 is configured to append and save the audio data block to the data object in response to the audio data block corresponding to the same voice processing task as a stored data object.

[0154] In some optional implementations of this embodiment, the second processing module 821 is further configured to receive audio data blocks sent by the browser in the following manner: receiving audio data blocks and corresponding identification numbers sent by the browser through a first communication link; in response to determining that the audio data blocks have been successfully received, sending an acknowledgment signal and identification number to the browser through the first communication link.

[0155] In some optional implementations of this embodiment, the business server runs a first service instance and a second service instance, and a first communication link connects the browser and the first service instance of the business server. The second processing module 821 is further configured to receive audio data blocks sent by the browser in the following manner: in response to an update of the first service instance, it sends an update notification to the browser; in response to the first communication link being disconnected and all currently received audio data blocks having been stored, it exits and updates the first service instance; in response to receiving a reconnection request from the browser, it establishes a second communication link with the browser through the second service instance that has not exited; and it receives audio data blocks and corresponding identifiers sent by the browser through the second communication link.

[0156] In some optional implementations of this embodiment, the second storage module 822 is further configured to transfer the data object to the cloud storage space of the target user, including the user who initiated the voice processing task.

[0157] In some optional implementations of this embodiment, the second processing module 821 is further configured to: perform voice activity detection on the audio data block to determine the start and end points of valid voice segments; filter out invalid voice segments in the audio data block according to the determined start and end points of valid voice segments to obtain valid voice segments; aggregate the valid voice segments in sequence to obtain aggregated voice data; and send the aggregated voice data to the cloud server for speech transcription processing.

[0158] In this embodiment, the other features and their technical effects of the acquisition module 811, the first storage module 812, and the first processing module 813 of the first voice processing device 810, and the second processing module 821 and the second storage module 822 of the second voice processing device 820 can be referred to respectively. Figures 2 to 7 The relevant descriptions of the method embodiments or processing flows shown are not repeated here.

[0159] This embodiment exists as an apparatus and system embodiment corresponding to the above-described voice data processing method embodiment. By storing audio data blocks in a local database, the risk of data loss can be effectively avoided, ensuring the continuity and integrity of the original audio data. When it is determined that the audio data block sent by the browser fails to be transmitted, the failed audio data block can be marked as retransmitted data, and the original audio data stored in the local database can be sent to the service server again, thereby improving the continuity and reliability of audio data block transmission. By determining whether the received audio data block is marked as retransmitted data, the service server can confirm whether the audio data block is real-time transmitted data, and then send the real-time transmitted audio data block to the cloud server for speech-to-text processing, thereby ensuring that the audio data block sent to the cloud server is real-time transmitted, meeting the user's real-time speech-to-text needs.

[0160] According to embodiments of the present disclosure, the present disclosure also provides an electronic device, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement the voice data processing method described in any of the above embodiments when executed.

[0161] According to embodiments of this disclosure, this disclosure also provides a readable storage medium storing computer instructions that enable a computer to implement the voice data processing method described in any of the above embodiments when executed.

[0162] Next reference Figure 9 , Figure 9 A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0163] like Figure 9As shown, device 900 includes a computing unit 901, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 902 or a computer program loaded from storage unit 908 into random access memory (RAM) 903. RAM 903 may also store various programs and data required for the operation of device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via bus 904. Input / output (I / O) interface 905 is also connected to bus 904.

[0164] Multiple components in device 900 are connected to I / O interface 905, including: input unit 906, such as keyboard, mouse, etc.; output unit 907, such as various types of monitors, speakers, etc.; storage unit 908, such as disk, optical disk, etc.; and communication unit 909, such as network card, modem, wireless transceiver, etc. Communication unit 909 allows device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0165] The computing unit 901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as voice data processing methods. For example, in some embodiments, the voice data processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the voice data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the voice data processing method by any other suitable means (e.g., by means of firmware).

[0166] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0167] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0168] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0169] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0170] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0171] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and Virtual Private Server (VPS) services, such as high management difficulty and weak business scalability.

[0172] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0173] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A voice data processing method, applied to a business server, comprising: Receive audio data blocks sent from the browser; In response to the fact that the audio data block is not marked as retransmission data, the audio data block is sent to the cloud server for speech transcription processing; The speech-to-text result returned by the cloud server is sent to the browser.

2. The method according to claim 1, further comprising: In response to the audio data block corresponding to the same speech processing task as the stored data object, the audio data block is appended and saved to the data object.

3. The method according to claim 1, further comprising: In response to the audio data block being marked as retransmitted data and corresponding to the same speech processing task as the stored data object, the audio data block is directly appended and saved to the data object.

4. The method according to claim 2 or 3, wherein, The audio data block received from the browser includes: The browser sends audio data blocks and corresponding identifiers via the first communication link. In response to the confirmation that the audio data block has been successfully received, an acknowledgment signal and the identification number are sent to the browser via the first communication link.

5. The method according to claim 4, wherein, The business server runs a first service instance and a second service instance, and the first communication link connects the browser client and the first service instance of the business server. The audio data block received from the browser also includes: In response to the need to update the first service instance, an update notification is sent to the browser. In response to the first communication link being disconnected and all currently received audio data blocks having been stored, exit and update the first service instance; In response to receiving a reconnection request from the browser, a second communication link is established with the browser through a second service instance that has not been logged out; The second communication link receives audio data blocks and corresponding identifiers sent by the browser.

6. The method according to claim 2 or 3, further comprising: The data object is transferred to the cloud storage space of the target user, including the user who initiated the voice processing task.

7. The method according to claim 1, wherein, Sending the audio data block to the cloud server for speech-to-text processing includes: Speech activity detection is performed on the audio data block to determine the start and end points of valid speech segments; Based on the determined start and end points of the valid speech segments, invalid speech segments in the audio data block are filtered out to obtain valid speech segments; The effective speech segments are aggregated sequentially to obtain aggregated speech data; The aggregated voice data is sent to a cloud server for speech-to-text processing.

8. A voice data processing method, applied to a browser, comprising: The collected audio data is divided into multiple audio data blocks and stored in a local database; Read the audio data block from the local database and send it to the service server; In response to determining that the audio data block failed to be sent, the audio data block is marked as retransmission data and sent to the service server again.

9. The method according to claim 8, wherein, The process of dividing the collected recording data into multiple audio data blocks and storing them in a local database includes: The collected audio data is divided into multiple audio data blocks, and each audio data block is marked with a corresponding identifier number; Store the audio data block and its corresponding identifier in a local database; The step of reading the audio data block from the local database and sending it to the service server includes: Read the audio data block and its corresponding identifier from the local database and send it to the service server; If no confirmation signal and the identification number are received from the service server within the preset transmission time, it is determined that the audio data block transmission has failed.

10. The method according to claim 8 or 9, further comprising: In response to receiving an update notification from the service server, disconnect the first communication link between the first service instance currently running on the service server; Send a reconnection request to the service server and establish a second communication link with the second service instance that has not logged out of the service server; The audio data block and its corresponding identifier are sent to the service server via the second communication link.

11. A voice data processing method, applied to a system including a browser and a business server, the method comprising: The browser divides the collected recording data into multiple audio data blocks and stores them in a local database. It then reads the audio data blocks from the local database and sends them to the service server. In response to determining that the audio data block failed to be sent, the browser marks the audio data block as retransmission data and sends it to the service server again. The service server receives the audio data block sent by the browser client. In response to the fact that the audio data block is not marked as retransmission data, it sends the audio data block to the cloud server for speech-to-text processing, and sends the speech-to-text result returned by the cloud server to the browser client.

12. A voice data processing device, applied to a browser, comprising: The acquisition module is configured to divide the acquired recording data into multiple audio data blocks; The first storage module is configured to store the audio data blocks in a local database; The first processing module is configured to read the audio data block from the local database and send it to the service server; in response to determining that the audio data block failed to be sent, the audio data block is marked as retransmission data and sent to the service server again.

13. A voice data processing device, applied to a business server, comprising: The second processing module is configured to receive audio data blocks sent by the browser. In response to the audio data block not being marked as retransmission data, the audio data block is sent to the cloud server for speech-to-text processing; and the speech-to-text result returned by the cloud server is sent to the browser. The second storage module is configured to append and save the audio data block to the data object in response to the audio data block corresponding to the same speech processing task as the stored data object.

14. A voice data processing system, comprising a browser and a service server, wherein, The browser is configured to divide the collected recording data into multiple audio data blocks and store them in a local database, and read the audio data blocks from the local database and send them to the service server. In response to determining that the audio data block failed to be sent, the audio data block is marked as retransmission data and sent to the service server again. The service server is configured to receive audio data blocks sent by the browser, and in response to the audio data blocks not being marked as retransmission data, send the audio data blocks to the cloud server for speech-to-text processing, and send the speech-to-text result returned by the cloud server to the browser.

15. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-11.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-11.

Images