A distributed speech-to-text processing system, method, device and medium

Abstract

The present invention discloses a distributed speech-to-text processing system, including a translation assistant end, a server and a client end electrically connected in sequence; the translation assistant end includes a data processing cluster module and a remote data collector, a load distributor, a socket cluster module, a replaceable translation module, a replaceable decoder and an API controllable IP valve electrically connected to the data processing cluster module; it also includes a k8s module, and the server includes a cloud server and a local server. The present invention supports independent deployment and flexible configuration on local servers and the cloud; it automatically increases or decreases computing nodes according to the load distributor and the k8s module to balance the load; a replaceable translation module and a replaceable decoder are provided to flexibly select translation products according to the translation accuracy of different service providers; the translation results can be received in real time through the socket cluster module, which can be stable and efficient, and can process speech-to-text tasks in large quantities in real time.

Description

Technical Field

[0001] The present invention relates to the field of audio processing technology, and in particular to a distributed speech-to-text processing system, method, device and medium. Background Art

[0002] In the existing technology, in the field of speech-to-text, there are many mature technical solutions applied to customer service scenarios: Avaya, as a world-renowned enterprise communication solution provider, focuses on the construction of communication infrastructure with its core technology, such as providing high-quality voice calls, video conferencing, messaging and other functions. Although it does not directly provide speech-to-text services, its platform can be used in conjunction with other speech recognition services to indirectly realize the text recording of call content. Baidu's speech-to-text service relies on Baidu's deep accumulation in the field of artificial intelligence, providing basic speech recognition APIs, allowing developers to integrate it into the call center system and convert telephone call content into text in real time. Although a single API service is relatively independent, it requires developers to deeply customize and interface docking to integrate into the entire customer service process. iFlytek also occupies an important position in the domestic market, and its speech-to-text technology is widely used in various scenarios, including but not limited to customer service systems. By providing a standardized speech recognition interface, customers can call its service for real-time speech transcription, but it also faces the same problem as Baidu, that is, it requires additional technical docking and maintenance.

[0003] The problems of existing technologies mainly include: (1) Lack of customization and secondary development capabilities: Although enterprise communication solutions such as Avaya are widely used, they are not open source systems and cannot be easily customized to meet the needs of specific enterprises for speech-to-text services; (2) Incomplete functions and complex interface docking: Service providers such as Baidu Speech-to-Text and iFlytek only provide basic speech-to-text services, but have not formed a complete solution that covers all necessary links and is easy to integrate. Developers need to dock with the interface by themselves when using it, which increases the complexity and risk of the project, and the technical level of programmers directly affects the stability of the translation service; (3) There is no one-stop service in the market: There is no unified and complete speech-to-text service product on the market, and it is impossible to achieve seamless docking from speech data capture, processing, translation to final presentation. Especially in the case of large-scale real-time translation, there are deficiencies in performance and availability. Summary of the invention

[0004] The present invention provides a distributed speech-to-text processing system, method, device and medium, which support independent deployment and flexible configuration on local servers and the cloud; automatically increase or decrease computing nodes according to a load distributor and a k8s module to balance the load; a replaceable translation module and a replaceable decoder are provided to flexibly select translation products according to the translation accuracy of different service providers; and the translation results can be received in real time through a socket cluster module, so that the system can be stable and efficient, and speech-to-text tasks can be processed in real time and in large quantities.

[0005] The present invention provides a distributed speech-to-text processing system, comprising a translation assistant terminal, a server and a client terminal which are electrically connected in sequence;

[0006] The translation assistant end includes a data processing cluster module and a remote data collector, a load distributor, a socket cluster module, a replaceable translation module, a replaceable decoder and an API controllable IP valve, all of which are electrically connected to the data processing cluster module.

[0007] Preferably, a k8s module is also included, and the server includes a cloud server and a local server;

[0008] Both the cloud server and the local server communicate with the translation assistant through the k8s module;

[0009] The client includes an Internet phone and a computer; the cloud server and the local server are respectively connected to the Internet phone and the computer for communication.

[0010] A distributed speech-to-text processing method, applied to the distributed speech-to-text processing system, comprises:

[0011] The remote data collector on the translation assistant collects the remote data and local data of the network card, identifies and intercepts the audio data packets;

[0012] The API-controllable IP valve on the translation assistant provides additional configuration for the remote data collector to filter out valid audio data;

[0013] The load distributor distributes the valid audio data to different nodes in the data processing cluster module for processing according to the load condition of the data processing cluster module;

[0014] The interchangeable decoder at the translation assistant end decompresses the valid audio data to restore the audio to obtain clear audio segments, and merges the clear audio segments to obtain coherent audio data;

[0015] The interchangeable translation module on the translation assistant side creates an audio processor instance and assigns the audio processor to the coherent audio data for translation, and sends the translation result to the data processing cluster module;

[0016] The socket cluster module on the translation assistant side receives the translation results obtained through the message middleware.

[0017] Preferably, the collecting of remote data and local data of the network card and identifying and intercepting audio data packets are specifically as follows:

[0018] The translation assistant captures the remote data and local data of the network card in real time through the remote data collector, and identifies and intercepts the audio data packets related to the telephone calls contained therein.

[0019] Preferably, the decompressing and restoring the valid audio data to obtain clear audio segments, and merging the clear audio segments to obtain coherent audio data specifically comprises:

[0020] For the filtered valid audio data, the decoder can be used to perform real-time decompression and restoration to obtain clear audio clips without translation;

[0021] At the appropriate stage, clear audio clips from different seats or terminals are merged in an orderly manner to obtain coherent audio data.

[0022] Preferably, the step of creating an audio processor instance and assigning an audio processor to translate the coherent audio data is as follows:

[0023] According to the characteristics of the coherent audio data and the system resources, the convertible translation module creates an appropriate audio processor instance and allocates an independent processor to each or multiple audio channels for parallel processing and real-time translation;

[0024] The audio processor feeds coherent audio data into the translation module in real time, and performs speech-to-text operations through the integrated Baidu and Huawei cloud translation service interfaces.

[0025] Preferably, the receiving of the translation result obtained through the message middleware is specifically:

[0026] After the data processing cluster module receives the real-time audio translation, it encapsulates the translation result in a structured form and publishes the message through the message queue middleware.

[0027] Preferably, the translated text data is received via the Socket cluster module, and is used to store in the cloud server and the local server and push to the Internet phone and the computer.

[0028] A device comprises a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the distributed speech-to-text processing method when executing the computer program.

[0029] A computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the distributed speech-to-text processing method.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] The present invention discloses a distributed speech-to-text processing system, method, device and medium, which support independent deployment and flexible configuration on local servers and the cloud; automatically increase or decrease computing nodes according to a load distributor and a k8s module to balance the load; provide a replaceable translation module and a replaceable decoder to flexibly select translation products according to the translation accuracy of different service providers; receive translation results in real time through a socket cluster module, achieve stability and efficiency, and process speech-to-text tasks in real time and in large quantities. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of a distributed speech-to-text processing system module provided by an embodiment of the present invention;

[0033] Figure 2 It is a flow chart of a distributed speech-to-text processing method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] A distributed speech-to-text processing system comprises a translation assistant terminal, a server and a client terminal which are electrically connected in sequence;

[0036] The translation assistant end includes a data processing cluster module and a remote data collector, a load distributor, a socket cluster module, a replaceable translation module, a replaceable decoder and an API controllable IP valve, all of which are electrically connected to the data processing cluster module.

[0037] Preferably, a k8s module is also included, and the server includes a cloud server and a local server;

[0038] Both the cloud server and the local server communicate with the translation assistant through the k8s module;

[0039] The client includes an Internet phone and a computer; the cloud server and the local server are respectively connected to the Internet phone and the computer for communication.

[0040] In a preferred embodiment, the data processing cluster module, the core component of the translation assistant end, is responsible for the coordination and management of the entire system; through the remote data collector, the device can capture the original voice data in real time from the call data stream generated by the telephone agent through the Internet phone; the communication data between the telephone agent and the Internet phone is effectively managed and screened through the API-controllable IP valve to ensure that only the target voice data flows into the data processing cluster.

[0041] The load distributor is located inside the data processing cluster module and is responsible for distributing voice data to different nodes in the data processing cluster for processing according to the current system load. When the data traffic is too large, the k8s module will automatically intervene and dynamically expand resources between the local server and the cloud server according to the pre-set strategy to accommodate more data processing units and ensure that the system is always in a high-availability state; the data processing cluster establishes a real-time communication link with the front-end display system through the socket cluster module, and instantly pushes the translated text data to the HTML page used by the customer service staff for display.

[0042] During the data processing process, the device adopts the design of interchangeable translation modules and interchangeable decoders. The decoder determines and decompresses the voice data according to the compression method of the data packet, and then sends the decompressed audio data to third-party translation services such as Baidu Translate, Huawei Cloud, and iFlytek through the interchangeable translation module for real-time translation. The device can flexibly switch the translation service capabilities according to the quality and characteristics of different service providers, thereby ensuring the accuracy and stability of the translation results.

[0043] A distributed speech-to-text processing method, applied to the distributed speech-to-text processing system, comprises:

[0044] S1: The remote data collector on the translation assistant collects the remote data and local data of the network card, identifies and intercepts the audio data packets;

[0045] S2: The API-controllable IP valve on the translation assistant provides additional configuration for the remote data collector to filter out valid audio data;

[0046] S3: The load distributor distributes the valid audio data to different nodes in the data processing cluster module for processing according to the load of the data processing cluster module;

[0047] S4: The interchangeable decoder at the translation assistant end decompresses the valid audio data to restore the audio to obtain clear audio segments, and merges the clear audio segments to obtain coherent audio data;

[0048] S5: The interchangeable translation module at the translation assistant end creates an audio processor instance and allocates the audio processor to the coherent audio data for translation, and sends the translation result to the data processing cluster module;

[0049] S6: The socket cluster module on the translation assistant side receives the translation result obtained through the message middleware.

[0050] In the above solution, the workflow of the translation assistant covers all aspects from raw data capture to final front-end display, ensuring the real-time and high availability of the speech-to-text process. With the elastic expansion capability achieved by the load distributor and k8s module, it can adapt to the speech-to-text needs of large-scale telephone agent systems. Through the capture, filtering, decoding, and translation of voice data, and then to the intelligent distribution, transmission, and display of results, a complete, efficient, and elastically scalable speech-to-text processing closed loop is formed. It is especially suitable for handling the real-time speech-to-text needs of large-scale telephone agent systems, greatly improving the work efficiency and customer service quality of customer service personnel.

[0051] Preferably, in step S1, the collector collects remote data and local data of the network card, and identifies and intercepts audio data packets specifically as follows:

[0052] The translation assistant captures the remote data and local data of the network card in real time through the remote data collector, and identifies and intercepts the audio data packets related to the telephone calls contained therein.

[0053] Preferably, in step S4, the decompression and restoration of the valid audio data to obtain clear audio segments, and merging the clear audio segments to obtain coherent audio data are specifically as follows:

[0054] For the filtered valid audio data, the decoder can be used to perform real-time decompression and restoration to obtain clear audio clips without translation;

[0055] At the appropriate stage, clear audio clips from different seats or terminals are merged in an orderly manner to obtain coherent audio data.

[0056] Preferably, in step S5, creating an audio processor instance and assigning an audio processor to the coherent audio data for translation is specifically as follows:

[0057] According to the characteristics of the coherent audio data and the system resources, the convertible translation module creates an appropriate audio processor instance and allocates an independent processor to each or multiple audio channels for parallel processing and real-time translation;

[0058] The audio processor feeds coherent audio data into the translation module in real time, and performs speech-to-text operations through the integrated Baidu and Huawei cloud translation service interfaces.

[0059] Preferably, in step S6, the receiving of the translation result obtained through the message middleware is specifically:

[0060] After the data processing cluster module receives the real-time audio translation, it encapsulates the translation result in a structured form and publishes the message through the message queue middleware.

[0061] Preferably, the translated text data is received via the Socket cluster module, and is used to store in the cloud server and the local server and push to the Internet phone and the computer.

[0062] In the above scheme, the device supports completely independent deployment, and can be flexibly configured on a local server or in the cloud to meet the deployment requirements of different scales and environments. At the same time, through the built-in load distributor and the container orchestration technology of the k8s module, high availability is achieved, and computing nodes can be automatically increased or decreased to balance the load, and tasks can be quickly migrated to other normal nodes when some nodes fail, ensuring the continuous and stable operation of the translation service; it particularly emphasizes real-time and batch processing capabilities, and can handle the voice-to-text requirements of a large number of seats at the same time, significantly improving customer service efficiency, especially facing the huge call volume of medium and large telephone seat systems; the interchangeable translation module and the interchangeable decoder module in the translation assistant terminal are replaceable, and users can flexibly select the optimal translation product according to the translation accuracy and stability of different service providers, ensuring the quality and diversity of the translation results; compared with the existing technology that requires programmers to additionally connect to the translation service interface, the device can achieve real-time reception of translation results by simply accessing the socket cluster module, reducing the integration difficulty and dependence on the developer's technical level, thereby enhancing the ease of use and stability of the system.

[0063] A device comprises a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the distributed speech-to-text processing method when executing the computer program.

[0064] A computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the distributed speech-to-text processing method.

[0065] Preferably, the computer program can be divided into one or more modules / units (such as computer program, computer program), and the one or more modules / units are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program in the terminal device.

[0066] The processor may be a central processing unit (CPU), or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor. The processor is the control center of the terminal device, and various parts of the terminal device are connected using various interfaces and lines.

[0067] The memory mainly includes a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function, etc., and the data storage area can store related data, etc. In addition, the memory can be a high-speed random access memory, or a non-volatile memory, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, and a flash card (Flash Card), etc., or the memory can also be other volatile solid-state storage devices.

[0068] It should be noted that the above-mentioned terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that the above-mentioned terminal device is merely an example and does not constitute a limitation on the terminal device. It may include more or fewer components, or a combination of certain components, or different components.

[0069] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A distributed speech-to-text processing system, characterized in that: include A translation assistant terminal, a server, and a client terminal electrically connected in sequence; The translation assistant end includes a data processing cluster module and a remote data collector, a load distributor, a socket cluster module, a replaceable translation module, a replaceable decoder and an API controllable IP valve, all of which are electrically connected to the data processing cluster module; It also includes a k8s module, and the server includes a cloud server and a local server; Both the cloud server and the local server communicate with the translation assistant through the k8s module; The client includes an Internet phone and a computer; the cloud server and the local server are respectively connected to the Internet phone and the computer for communication; The load distributor is located inside the data processing cluster module and is responsible for distributing voice data to different nodes in the data processing cluster for processing according to the current system load. When the data traffic is too large, the k8s module will automatically intervene and dynamically expand resources between the local server and the cloud server according to the pre-set strategy. The interchangeable translation module and interchangeable decoder module in the translation assistant are replaceable, and users can flexibly choose the optimal translation product based on the translation accuracy and stability of different service providers.

2. A distributed speech-to-text processing method, applied to the distributed speech-to-text processing system of claim 1, characterized in that: include: The remote data collector on the translation assistant collects the remote data and local data of the network card, identifies and intercepts the audio data packets; The API-controllable IP valve on the translation assistant provides additional configuration for the remote data collector to filter out valid audio data; The load distributor distributes the valid audio data to different nodes in the data processing cluster module for processing according to the load condition of the data processing cluster module; The interchangeable decoder at the translation assistant end decompresses the valid audio data to restore the audio to obtain clear audio segments, and merges the clear audio segments to obtain coherent audio data; The interchangeable translation module on the translation assistant side creates an audio processor instance and assigns the audio processor to the coherent audio data for translation, and sends the translation result to the data processing cluster module; The socket cluster module on the translation assistant side receives the translation results obtained through the message middleware.

3. A distributed speech-to-text processing method according to claim 2, characterized in that: The acquisition of remote data and local data of the network card and identification and interception of audio data packets are specifically as follows: The translation assistant captures the original data stream of the network card in real time through a remote data collector, and identifies and intercepts the audio data packets related to the telephone call contained therein.

4. A distributed speech-to-text processing method according to claim 3, characterized in that: The decompression and restoration of the valid audio data to obtain clear audio segments, and merging the clear audio segments to obtain coherent audio data are specifically as follows: For the filtered valid audio data, the decoder can be used to perform real-time decompression and restoration to obtain clear audio clips; At the appropriate stage, clear audio clips from different seats or terminals are merged in an orderly manner to obtain coherent audio data.

5. A distributed speech-to-text processing method according to claim 4, characterized in that: The steps of creating an audio processor instance and assigning the audio processor to translate the coherent audio data are as follows: According to the characteristics of the coherent audio data and the system resources, the convertible translation module creates an appropriate audio processor instance and allocates an independent processor to each or multiple audio channels for parallel processing and real-time translation; The audio processor feeds coherent audio data into the translation module in real time, and performs speech-to-text operations through the integrated Baidu and Huawei cloud translation service interfaces.

6. A distributed speech-to-text processing method according to claim 5, characterized in that: The translation result received through the message middleware is specifically: After the data processing cluster module receives the real-time audio translation, it encapsulates the translation result in a structured form and publishes the message through the message queue middleware.

7. A distributed speech-to-text processing method according to claim 6, characterized in that: The translated text data is received through the Socket cluster module and is stored in the cloud server and local server and pushed to the Internet phone and computer.

8. A device, characterized in that The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, a distributed speech-to-text processing method as described in any one of claims 2 to 7 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute a distributed speech-to-text processing method as described in any one of claims 2 to 7.

Images