Speech recognition method, device, equipment and storage medium based on artificial intelligence

Abstract

The present application relates to artificial intelligence technology, and proposes a speech recognition method, device, equipment and storage medium based on artificial intelligence. The method includes: if a recording start instruction is received, a storage area is allocated and monitored; the audio data generated during the recording process is temporarily stored in the storage area; if audio data is stored in the storage area, a communication link with the server is established; if the communication link is successfully established, the audio data in the storage area is uploaded to the server; and the text data returned by the server is received and displayed. The present application only controls the opening and closing of the recording through the user's recording operation, and the establishment of the communication link is driven by the presence or absence of audio data. It ensures that the user can press the button to record at any time, reduce the loss of recording information, and realize real-time speech recognition. At the same time, there will be no waste of server resources caused by idle links, which effectively reduces the pressure on the server.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a speech recognition method, apparatus, device and storage medium based on artificial intelligence. Background Art

[0002] Currently, speech-to-text conversion usually adopts the interactive mode of voice input, where the user presses a button to speak, and then the recorded speech content is recognized by speech-to-text conversion. The main recognition schemes are as follows:

[0003] The first method is to record the audio first, generate an audio file after the recording is completed, and send it to the back-end server for recognition. The advantage of this method is that it is relatively stable and the burden on the server is lower, but it cannot achieve real-time speech recognition.

[0004] The second solution is to continuously send the audio stream to the backend server for recognition, and then splice the returned results, so that the user can see the transcribed text results while speaking. However, the problem with this solution is that when the user's mobile phone environment is unstable, it takes a certain amount of time to establish a link. If the link is not established when the user presses the button to start speaking, part of the content at the beginning of the voice will be lost. To solve this problem, the link can be established before the user presses the button, but this solution will exist regardless of whether the user is actually recording, which will bring additional server overhead. If the number of users is large, a large number of idle communication links will cause a waste of server resources. Summary of the invention

[0005] In order to solve the technical problem that real-time speech recognition cannot be achieved in the prior art or that real-time speech recognition brings a large overhead to the server, the present application provides a speech recognition method, device, equipment and storage medium based on artificial intelligence, the main purpose of which is to achieve real-time speech recognition while effectively reducing the pressure on the server.

[0006] To achieve the above object, the present application provides a real-time speech recognition method, which comprises:

[0007] If a first recording start instruction is received, a first storage area is allocated and monitored;

[0008] temporarily storing the audio data generated in the first recording process in the first storage area;

[0009] If it is monitored that the first storage area stores audio data, a first communication link with the server is established;

[0010] If the first communication link is successfully established, the audio data in the first storage area is uploaded to the server, and the server is used to convert the received audio data into corresponding text data in real time and return it to the terminal device;

[0011] The first text data returned by the server is received and displayed, where the first text data is text data corresponding to the audio data in the first storage area.

[0012] In addition, to achieve the above purpose, the present application also provides a real-time speech recognition device, which includes:

[0013] A first cache creation module, configured to allocate and monitor a first storage area if a first recording start instruction is received;

[0014] A first cache module, used for temporarily storing the audio data generated in the first recording process in a first storage area;

[0015] A first link establishing module, configured to establish a first communication link with a server end if it is detected that audio data is stored in the first storage area;

[0016] A first data uploading module, configured to upload the audio data in the first storage area to a server if the first communication link is successfully established, wherein the server is configured to convert the received audio data into corresponding text data in real time and return the text data to the terminal device;

[0017] The receiving and displaying module is used to receive and display the first text data returned by the server, wherein the first text data is the text data corresponding to the audio data in the first storage area.

[0018] To achieve the above objectives, the present application also provides a computer device, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, the steps of the real-time speech recognition method as described in any of the above items are performed.

[0019] To achieve the above objectives, the present application also provides a computer-readable storage medium, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor, the processor executes the steps of any of the above-mentioned real-time speech recognition methods.

[0020] The speech recognition method, device, equipment and storage medium based on artificial intelligence proposed in this application decouple the user's recording behavior from the establishment of the link. The user's recording operation only controls the opening and closing of the recording, and the establishment of the communication link is driven by the presence or absence of audio data. The communication link is established only when there is audio data, which ensures that the user can press the button to record at any time without information loss, and there will be no idle links, causing waste of the server. While achieving real-time speech recognition, it is not affected by the network and loses information, and can effectively reduce the pressure on the server. The user's operation does not depend on the network, and the operation experience will be smoother. At the same time, it will be simpler and more stable to deal with certain boundary problems. For example, in the scenario where the user repeatedly clicks the recording button without speaking, since the audio data is empty, there is no need to deal with the conflict of repeatedly establishing and disconnecting long links in the case of multiple invalid recordings, which improves the stability of the application, reduces the complexity of the problems in the development process, and improves the user's actual application experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is an application scenario diagram of the real-time speech recognition method in one embodiment of the present application;

[0022] Figure 2 This is a flowchart of a real-time speech recognition method in an embodiment of the present application;

[0023] Figure 3 This is a structural block diagram of a real-time speech recognition device in an embodiment of the present application;

[0024] Figure 4 This is a block diagram of the internal structure of a computer device in one embodiment of the present application.

[0025] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of this application. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0027] The real-time speech recognition method provided in this application can be applied in Figure 1In a real-time speech recognition system. The real-time speech recognition system includes a terminal device 110 and a server end 120 communicating through a network. Among them, the terminal device 110 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The server end 120 can be implemented with an independent server or a server cluster composed of multiple servers. An application is installed in the terminal device 110. If the terminal device 110 receives a first recording start instruction, it allocates and monitors a first storage area; temporarily stores the audio data generated by the first recording process in the first storage area; if it is monitored that the first storage area stores audio data, a first communication link with the server end 120 is established; if the first communication link is successfully established, the audio data in the first storage area is uploaded to the server end 120, and the server end 120 is used to convert the received audio data into corresponding text data in real time and return it to the terminal device 110; receive and display the first text data returned by the server 120, and the first text data is the text data corresponding to the audio data in the first storage area.

[0028] Figure 2 Schematic diagram of the flow of a real-time speech recognition method in an embodiment of the present application. Figure 2 , this method is applied in Figure 1 The real-time speech recognition method includes the following steps S100-S500.

[0029] S100: If a first recording start instruction is received, a first storage area is allocated and monitored.

[0030] Specifically, this embodiment is specifically applied to a terminal device with an application installed. The application may be a small program or an application App on the application, but is not limited thereto. The application has a voice call function. When the user presses the voice recording button on the application, a recording start instruction is generated. For example, on the WeChat App, when the user needs to send a voice, he must first long press the "Press and Talk" button, and the voice module of the terminal device receives and records the user's voice.

[0031] The first storage area may be allocated from a memory of the terminal device.

[0032] S200: Temporarily storing the audio data generated in the first recording process in the first storage area.

[0033] Specifically, the first recording process is a recording process from receiving the first recording start instruction to receiving the first recording stop instruction. The audio data is specifically an audio stream, more specifically a binary data stream. The terminal device receives the user's voice and converts the received user voice into an audio stream. Since the corresponding data cache is quickly established after receiving the recording start instruction, the audio data is cached, because the audio data will not be lost regardless of whether the communication link is established.

[0034] After receiving the recording start instruction, this embodiment quickly creates a corresponding data cache, which can store the generated audio data in time and reduce the loss of audio data.

[0035] S300: If it is detected that audio data is stored in the first storage area, a first communication link with the server is established.

[0036] Specifically, in this embodiment, before receiving the first recording start instruction, the terminal device does not establish a communication link for voice recognition with the server side, but establishes the first communication link only when it monitors that audio data is stored in the first storage area. The first communication link is used for voice recognition between the terminal device and the server side. The first communication link is a long link, which can specifically be a websocket link based on the websocket protocol. In this embodiment, a communication link is established only when audio data is generated, which can reduce the occupation and consumption of network resources by idle communication links.

[0037] S400: If the first communication link is successfully established, the audio data in the first storage area is uploaded to the server, wherein the server is used to convert the received audio data into corresponding text data in real time and return it to the terminal device.

[0038] Specifically, the first storage area follows a first-in-first-out strategy, and the audio data that first enters the first storage area is uploaded to the server side first. In this embodiment, the audio data in the first storage area is uploaded to the server side in real time through the communication channel corresponding to the first communication link. The server side converts the received audio data into corresponding text data according to the speech-to-text technology, and returns the converted text data to the terminal device. This ensures that the first generated voice is recognized first, and the returned text data can be displayed in order according to the voice sequence.

[0039] S500: Receive and display first text data returned by the server, wherein the first text data is text data corresponding to the audio data in the first storage area.

[0040] Specifically, the corresponding text data is displayed in order below or beside the voice corresponding to the application program in the order in which the voice is emitted, so that the complete text sentence corresponding to a complete voice is smooth and organized.

[0041] The present application decouples the user's recording behavior from the establishment of a link. The user's recording operation only controls the opening and closing of the recording, while the establishment of the communication link is driven by the presence or absence of audio data. The communication link is established only when there is audio data, which not only ensures that the user can press the button to record at any time, but also creates a data cache while recording. Therefore, no matter whether the communication link is established or not, there will be no loss of voice data. This overcomes the problem that the audio data generated in the prior art must be sent out immediately through the communication link due to the lack of cache, resulting in the loss of voice data before the communication link is successfully established or when the network is unstable, and effectively ensures the integrity of the voice data. Since the communication link is established only when there is audio data, and the communication link will not be established before there is no audio data, there will be no idle link, which reduces the waste of server resources and network resources. This embodiment can achieve real-time voice recognition without losing information due to network influence, and can effectively reduce the pressure on the server. The user's operation does not rely on the network, and the operation experience will be smoother. At the same time, it will be simpler and more stable to handle certain boundary issues. For example, in a scenario where the user repeatedly clicks the recording button without speaking, since the audio data is empty, there is no need to deal with the conflict of repeatedly establishing and disconnecting long links in the case of multiple invalid recordings. This improves the stability of the application, reduces the complexity of problems in the development process, and improves the user's actual application experience.

[0042] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. For example, in the embodiments of the present application, the server converts speech into text using artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0043] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0044] In one embodiment, after step S500, the method further includes:

[0045] If the first communication link is in an idle state, the first communication link is disconnected.

[0046] Specifically, if there is no data transmission, the first communication link is disconnected, and the data includes audio data and text data. If all the audio data in the first storage area is uploaded to the server side, and all the text data returned by the server side is received, and there is no other data cache that needs to upload audio data or receive text data through the communication channel corresponding to the first communication link, then it is judged that the first communication link is in an idle state and can be disconnected. Disconnecting the link in time when the communication link is in an idle state can effectively reduce the occupation and waste of server resources and reduce the additional invalid overhead of the server.

[0047] In one embodiment, the method further includes: removing the audio data in the first storage area that has been successfully uploaded to the server from the head of the first storage area.

[0048] Specifically, in the process of uploading the audio data in the first storage area to the server, the terminal device will also receive a response from the server confirming the receipt of the data packet. Therefore, through the confirmation response, it can be determined which audio data has been successfully received, and then the successfully received audio data can be removed from the cache to reduce memory occupancy.

[0049] In one embodiment, after step S200, the method further includes:

[0050] If a second recording start instruction is received after the first recording ends, a second storage area is allocated and monitored;

[0051] temporarily storing the audio data generated by the second recording process in the second storage area;

[0052] If the first communication link is disconnected, then if it is detected that the second storage area stores audio data, a second communication link with the server is established.

[0053] If the second communication link is successfully established, the audio data in the second storage area is uploaded to the server via the second communication link.

[0054] receiving and displaying second text data returned by the server through the second communication link, wherein the second text data is text data corresponding to the audio data in the second storage area;

[0055] If the first communication link is not disconnected, the audio data in the second storage area is uploaded to the server through the first communication link,

[0056] The second text data returned by the server is received and displayed through the first communication link.

[0057] Specifically, in voice chat or communication, a continuous voice is often time-limited, so the user needs to send multiple voices at different sending times. Each voice is initiated by a recording start command and ended by a corresponding recording stop command. In this embodiment, each time a recording start command is received, a corresponding data cache is allocated and monitored; the audio data generated during the recording process is temporarily stored in the corresponding data cache; if it is monitored that audio data is stored in the data cache and a communication link with the server is not established, a communication link with the server is established; if the communication link is successfully established, the audio data in the data cache is uploaded to the server to convert the audio data into corresponding text data. If a communication link with the server has been established, the audio data in the data cache is directly uploaded to the server through the communication channel corresponding to the communication link without establishing a new communication link.

[0058] This embodiment solves the problem of storing audio data generated in different recording processes, and the audio data generated by each newly started recording process is stored in the corresponding newly created data cache, thereby solving the problem of the long interval between two adjacent recording processes causing the data cache to be occupied.

[0059] In addition, if the interval between two adjacent recording processes is relatively short, the previously established communication link may not be disconnected. In this case, there is no need to re-establish a new communication link, but the undisconnected communication link is used to transmit data. If the communication link is disconnected, a new communication link needs to be re-established to transmit audio data to the server and receive text data obtained by speech recognition on the server.

[0060] Similarly, if there is no data transmission, the second communication link is disconnected, and the data includes audio data and text data. If all the audio data in the second storage area is uploaded to the server side, and all the text data returned by the server side is received, and there is no other data cache that needs to upload audio data or receive text data through the communication channel corresponding to the second communication link, then it is judged that the second communication link is in an idle state and can be disconnected. Disconnecting the link in time when the communication link is in an idle state can effectively reduce the occupation and waste of server resources and reduce the additional invalid overhead of the server.

[0061] Similarly, the method further includes: removing the audio data in the second storage area that has been successfully uploaded to the server from the head of the second storage area.

[0062] Specifically, in the process of uploading the audio data in the second storage area to the server, the terminal device will also receive a response from the server confirming the receipt of the data packet. Therefore, through the confirmation response, it can be determined which audio data has been successfully received, and then the successfully received audio data can be removed from the cache to reduce memory occupancy.

[0063] In one embodiment, after step S200, the method further includes:

[0064] If a second recording start instruction is received within a preset time period after the first recording is completed, the audio data generated during the second recording process is temporarily stored in the first storage area.

[0065] Specifically, after receiving the recording start instruction and recording for a period of time, the user will send a recording stop instruction, for example, releasing the recording button, and then a recording stop instruction is generated.

[0066] After the first recording stops, the user can also start a new second recording.

[0067] If the time between the start time of the second recording and the end time of the first recording is less than the preset time, the audio data generated by the second recording process will continue to be cached in the first storage area. At this time, the first storage area may still contain the audio data generated by the first recording, or it may be empty, but the first storage area still exists and has not been removed, and because the server needs to return text data through the first communication link and / or the audio data in the first storage area has not been fully uploaded to the server, the first communication link has not been disconnected, that is, the first communication link has not been disconnected within the preset time.

[0068] The audio data newly added to the first storage area will be uploaded to the server through the communication channel corresponding to the first communication link, and the server will also return the converted text data to the terminal device according to the communication channel corresponding to the first communication link.

[0069] Under certain conditions, this embodiment can realize that audio data generated by different recording processes are stored in the same storage area, while in the previous embodiment of this embodiment, audio data generated by different recording processes are independently stored in different storage areas, that is, audio data generated by different recording processes do not coexist in the same storage area. Either this embodiment or the previous embodiment of this embodiment is executed.

[0070] In one embodiment, the method further comprises:

[0071] If a second recording start instruction is received after a preset time period after the first recording ends, a second storage area is allocated and monitored;

[0072] temporarily storing the audio data generated by the second recording process in the second storage area;

[0073] If the first communication link is not disconnected, the audio data in the second storage area is uploaded to the server through the first communication link, wherein the server is used to convert the received audio data into corresponding text data in real time and return it to the terminal device,

[0074] receiving and displaying second text data returned by the server through the first communication link, wherein the second text data is text data corresponding to the audio data in the second storage area;

[0075] If the first communication link is disconnected, then if it is detected that the second storage area stores audio data, a second communication link with the server is established.

[0076] If the second communication link is successfully established, the audio data in the second storage area is uploaded to the server through the second communication link, and the server is used to convert the received audio data into corresponding text data in real time and return it to the terminal device.

[0077] The second text data returned by the server is received and displayed through the second communication link.

[0078] Specifically, if after the first recording ends, no second recording start instruction is received within a preset time period, and within the preset time period after the first recording ends, all audio data in the first storage area has been uploaded to the server, the first storage area will be cleared and destroyed or removed. Therefore, after the preset time period, if a second recording start instruction is received, a new second storage area will be created, and the second storage area will be monitored for audio data.

[0079] After the preset time, the first communication link may not be disconnected or may be disconnected. If the server has not returned all the text data to the terminal device when the second recording start instruction is received, the first communication link will not be disconnected. The audio data in the second storage area can continue to be uploaded to the server through the first communication link, and the server will also return the corresponding text data through the first communication link.

[0080] If it is detected that the audio data has started to be stored in the second storage area and the first communication link has been disconnected, it is necessary to re-establish the second communication link between the terminal device and the server. At the same time, only one of the first communication link and the second communication link can exist at most.

[0081] This embodiment minimizes the ineffective occupation of memory when the time interval between two adjacent voice segments is long, and at the same time disconnects the communication link according to the completion of the actual task. When a new task is added, the undisconnected communication link is used or a new communication link is created, thereby achieving full utilization of network resources and server resources and reducing resource waste.

[0082] In this embodiment, the audio data generated by the newly started recording process after the preset time period is exceeded is stored in the newly allocated storage area. The difference between this embodiment and the previous embodiment of this embodiment is whether the audio data generated by the newly started recording process at different time points can coexist in the same storage area. This embodiment and the previous embodiment of this embodiment are executed one by one.

[0083] In one embodiment, step S200 specifically includes:

[0084] Whenever the accumulated data volume of the newly generated audio data during the recording process reaches the preset callback data volume, the recording interface is called to perform a callback on the newly generated audio data reaching the callback data volume and temporarily store it in the first storage area.

[0085] Specifically, the callback data volume is pre-configured in kb. The recording interface is provided by the terminal device, such as the recording interface provided by WeChat, which is used to receive the user's voice and generate audio data. Whenever the audio data accumulates to the callback data volume, an audio stream callback is performed. The purpose of the callback is to store the generated audio data in the data cache. After the callback, the newly generated audio data is accumulated again until the callback data volume is reached and is called back and stored in the data cache. Of course, if the last accumulated audio data does not reach the callback data volume and the recording stop command is received, the recording interface is directly called to execute the data callback.

[0086] In one embodiment, establishing a first communication link with the server in step S300 specifically includes:

[0087] The preset voice conversion rules, authentication information and user identity information are encrypted and sent to the server to establish a first communication link with the server.

[0088] Specifically, the purpose of establishing a communication link is to authenticate permissions and user identities, as well as to reach a consensus on conversion, so that the server can convert the audio data into text data in the format required by the terminal device according to the preset voice conversion rules. After the communication link is established, the server can know which user's terminal device sent the audio data, and also know which user's terminal device the converted text data should be sent back. Subsequent use of the communication channel corresponding to the communication link to send audio data does not require user identity information every time, nor does it require authentication every time.

[0089] Of course, in this embodiment, the method of this embodiment is used whether the terminal device and the server establish the first communication link or the second communication link.

[0090] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0091] Figure 3 This is a structural block diagram of a real-time speech recognition device in an embodiment of the present application. Figure 3 , the device comprises:

[0092] A first cache creation module 100 is used to allocate and monitor a first storage area if a first recording start instruction is received;

[0093] A first buffer module 200, used to temporarily store the audio data generated in the first recording process in a first storage area;

[0094] The first link establishing module 300 is used to establish a first communication link with the server side if it is monitored that the first storage area stores audio data;

[0095] The first data uploading module 400 is used to upload the audio data in the first storage area to the server side if the first communication link is successfully established, wherein the server side is used to convert the received audio data into corresponding text data in real time and return it to the terminal device;

[0096] The receiving and displaying module 500 is used to receive and display the first text data returned by the server, wherein the first text data is the text data corresponding to the audio data in the first storage area.

[0097] In one embodiment, the apparatus further comprises:

[0098] The link disconnection module is used to disconnect the first communication link if the first communication link is in an idle state.

[0099] In one embodiment, the apparatus further comprises:

[0100] A second cache creation module is used to allocate and monitor the second storage area if a second recording start instruction is received after the first recording ends;

[0101] A second cache module, used for temporarily storing the audio data generated in the second recording process in a second storage area;

[0102] The second link establishment module is used to establish a second communication link with the server side if the first communication link is disconnected and if the second storage area is monitored to store audio data,

[0103] The second data uploading module is used to upload the audio data in the second storage area to the server through the second communication link if the second communication link is successfully established.

[0104] The receiving and displaying module 500 is further used to receive and display the second text data returned by the server through the second communication link, wherein the second text data is the text data corresponding to the audio data in the second storage area;

[0105] The first data uploading module 400 is further used to upload the audio data in the second storage area to the server through the first communication link if the first communication link is not disconnected.

[0106] The receiving and displaying module 500 is further configured to receive and display the second text data returned by the server through the first communication link.

[0107] In one embodiment, the first cache module 200 is further used to temporarily store the audio data generated by the second recording process in the first storage area if a second recording start instruction is received within a preset time period after the first recording ends.

[0108] In one embodiment, the apparatus further comprises:

[0109] A second cache creation module is used to allocate and monitor the second storage area if a second recording start instruction is received after a preset time after the first recording ends;

[0110] A second cache module, used for temporarily storing the audio data generated in the second recording process in a second storage area;

[0111] The first data uploading module is further used for uploading the audio data in the second storage area to the server through the first communication link if the first communication link is not disconnected, and the server is used for converting the received audio data into corresponding text data in real time and returning it to the terminal device,

[0112] The receiving and displaying module 500 is further used to receive and display the second text data returned by the server through the first communication link, wherein the second text data is the text data corresponding to the audio data in the second storage area;

[0113] The second link establishment module is used to establish a second communication link with the server side if the first communication link is disconnected and if it is monitored that there is audio data stored in the second storage area,

[0114] A second data uploading module is used to upload the audio data in the second storage area to the server through the second communication link if the second communication link is successfully established, and the server is used to convert the received audio data into corresponding text data in real time and return it to the terminal device;

[0115] The receiving and displaying module 500 is further used to receive and display the second text data returned by the server through the second communication link.

[0116] In one embodiment, the first cache module 200 is specifically used to: whenever the cumulative data volume of newly generated audio data during the recording process reaches a preset callback data volume, call the recording interface to perform a callback on the newly generated audio data that reaches the callback data volume and temporarily store it in the first storage area.

[0117] In one embodiment, the first link establishing module 300 is specifically used to encrypt the preset voice conversion rules, authentication information and user identity information and send them to the server to establish a first communication link with the server.

[0118] The meaning of "first" and "second" in the above modules / units is only to distinguish different modules / units, and is not used to define which module / unit has a higher priority or other limiting meanings. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The division of modules in this application is only a logical division, and there may be other division methods when implemented in actual applications.

[0119] For the specific definition of the real-time speech recognition device, please refer to the definition of the real-time speech recognition method above, which will not be repeated here. Each module in the above-mentioned real-time speech recognition device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0120] Figure 4 This is a block diagram of the internal structure of a computer device in one embodiment of the present application. The computer device may specifically be Figure 1 The terminal device in. Figure 4As shown, the computer device includes a processor, a memory, a network interface, an input device and a display screen connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory includes a storage medium and an internal memory. The storage medium can be a non-volatile storage medium or a volatile storage medium. The storage medium stores an operating system and can also store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can implement a real-time speech recognition method. The internal memory provides an environment for the operation of the operating system and computer-readable instructions in the storage medium. The internal memory can also store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute a real-time speech recognition method. The network interface of the computer device is used to communicate with an external server through a network connection. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse.

[0121] In one embodiment, a computer device is provided, including a memory, a processor, and computer-readable instructions (such as a computer program) stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, the steps of the real-time speech recognition method in the above embodiment are implemented, such as Figure 2 Alternatively, when the processor executes the computer-readable instructions, the functions of each module / unit of the real-time speech recognition device in the above embodiment are realized, for example, Figure 3 The functions of the modules 100 to 500 are shown in detail and will not be described again to avoid repetition.

[0122] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of a computer device, and uses various interfaces and lines to connect various parts of the entire computer device.

[0123] The memory can be used to store computer-readable instructions and / or modules. The processor implements various functions of the computer device by running or executing the computer-readable instructions and / or modules stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, video data, etc.), etc.

[0124] The memory may be integrated into the processor or provided separately from the processor.

[0125] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0126] In one embodiment, a computer-readable storage medium is provided on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor, the steps of the real-time speech recognition method in the above embodiment are implemented, such as Figure 2 Alternatively, when the computer-readable instructions are executed by the processor, the functions of each module / unit of the real-time speech recognition device in the above embodiment are realized, for example, Figure 3 The functions of the modules 100 to 500 are shown in detail and will not be described again to avoid repetition.

[0127] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the computer-readable instructions are executed, they can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0128] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, device, article or method including the element.

[0129] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments. Through the description of the above implementation modes, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware, but in many cases the former is a better implementation mode. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, including a number of instructions to enable a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0130] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A real-time speech recognition method, characterized in that: The method comprises: If a first recording start instruction is received, a first storage area is allocated and monitored; temporarily storing the audio data generated in the first recording process in the first storage area; If it is detected that the first storage area stores audio data, a first communication link with the server is established; If the first communication link is successfully established, the audio data in the first storage area is uploaded to the server, wherein the server is used to convert the received audio data into corresponding text data in real time and return it to the terminal device; Receiving and displaying first text data returned by the server, wherein the first text data is text data corresponding to the audio data in the first storage area; If a second recording start instruction is received after the first recording ends, a second storage area is allocated and monitored; temporarily storing the audio data generated in the second recording process in the second storage area; If the first communication link is disconnected, then if it is detected that the second storage area stores audio data, a second communication link with the server is established. If the second communication link is successfully established, the audio data in the second storage area is uploaded to the server via the second communication link. receiving and displaying second text data returned by the server through the second communication link, wherein the second text data is text data corresponding to the audio data in the second storage area; If the first communication link is not disconnected, uploading the audio data in the second storage area to the server via the first communication link, The second text data returned by the server is received and displayed through the first communication link.

2. The method according to claim 1, characterized in that After receiving and displaying the first text data returned by the server, the method further includes: If the first communication link is in an idle state, the first communication link is disconnected.

3. The method according to claim 1, characterized in that After temporarily storing the audio data generated in the first recording process in the first storage area, the method further includes: If a second recording start instruction is received within a preset time period after the first recording is completed, the audio data generated during the second recording process is temporarily stored in the first storage area.

4. The method according to claim 3, characterized in that The method further comprises: If a second recording start instruction is received after a preset time period after the first recording ends, a second storage area is allocated and monitored; temporarily storing the audio data generated in the second recording process in the second storage area; If the first communication link is not disconnected, uploading the audio data in the second storage area to the server via the first communication link, receiving and displaying second text data returned by the server through the first communication link, wherein the second text data is text data corresponding to the audio data in the second storage area; If the first communication link is disconnected, then if it is detected that the second storage area stores audio data, a second communication link with the server is established. If the second communication link is successfully established, the audio data in the second storage area is uploaded to the server through the second communication link, and the server is used to convert the received audio data into corresponding text data in real time and return it to the terminal device. The second text data returned by the server is received and displayed through the second communication link.

5. The method according to claim 1, characterized in that The step of temporarily storing the audio data generated in the first recording process in the first storage area includes: Whenever the accumulated data volume of the newly generated audio data during the recording process reaches the preset callback data volume, the recording interface is called to perform a callback on the newly generated audio data reaching the callback data volume and temporarily store it in the first storage area.

6. The method according to claim 1, characterized in that The step of establishing a first communication link with the server side includes: The preset voice conversion rules, authentication information and user identity information are encrypted and sent to the server to establish a first communication link with the server.

7. A real-time speech recognition device, characterized in that: The device comprises: A first cache creation module, configured to allocate and monitor a first storage area if a first recording start instruction is received; A first cache module, used for temporarily storing the audio data generated in the first recording process in the first storage area; A first link establishing module, configured to establish a first communication link with a server end if it is detected that audio data is stored in the first storage area; A first data uploading module, configured to upload the audio data in the first storage area to the server end if the first communication link is successfully established, wherein the server end is configured to convert the received audio data into corresponding text data in real time and return the text data to the terminal device; A receiving and displaying module, used for receiving and displaying first text data returned by the server, wherein the first text data is text data corresponding to the audio data in the first storage area; A second cache creation module is used to allocate and monitor the second storage area if a second recording start instruction is received after a preset time after the first recording ends; A second cache module, used for temporarily storing the audio data generated in the second recording process in a second storage area; The first data uploading module is further used for uploading the audio data in the second storage area to the server through the first communication link if the first communication link is not disconnected, and the server is used for converting the received audio data into corresponding text data in real time and returning it to the terminal device, The receiving and displaying module is further used to receive and display the second text data returned by the server through the first communication link, wherein the second text data is the text data corresponding to the audio data in the second storage area; The second link establishment module is used to establish a second communication link with the server side if the first communication link is disconnected and if it is monitored that there is audio data stored in the second storage area, A second data uploading module is used to upload the audio data in the second storage area to the server through the second communication link if the second communication link is successfully established, and the server is used to convert the received audio data into corresponding text data in real time and return it to the terminal device; The receiving and displaying module is further used to receive and display the second text data returned by the server through the second communication link.

8. A computer device comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, characterized in that: When the processor executes the computer-readable instructions, the processor performs the steps of the real-time speech recognition method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer-readable instructions stored thereon, characterized in that: When the computer-readable instructions are executed by a processor, the processor performs the steps of the real-time speech recognition method according to any one of claims 1 to 6.

Images