AI voice assistant implementation method and equipment based on cloud computer terminal and medium

Abstract

The invention discloses an AI voice assistant implementation method and equipment based on a cloud computer terminal and a medium, belongs to the technical field of cloud computers and voice assistants, and aims to solve the technical problems of how to apply an AI voice assistant to the cloud computer terminal and awaken the assistant by customizing an awakening word, fill the blank that the cloud computer terminal does not belong to a voice assistant of the cloud computer terminal and improve the voice assistant implementation efficiency. According to the technical scheme, voice wake-up and interaction based on the Android platform comprises the steps that a background thread decompresses a voice model, a microphone monitoring service is automatically started after decompression is completed, a voice recognition service continuously captures microphone input, and a recognition result is returned in real time; when a preset wake-up word is detected, popping up a floating window to enter an interaction mode, managing text display and recording states in an interaction process, and realizing timeout automatic processing through a delay task; speech recognition and interaction control based on streaming audio; performing asynchronous voice question-answer response; asynchronous speech synthesis and transmission; and playing the audio after the voice synthesis is successful.

Description

Technical Field

[0001] This invention relates to the field of cloud computing and voice assistant technology, specifically to a method, device, and medium for implementing an AI voice assistant based on a cloud computing terminal. Background Technology

[0002] With the continuous development of global artificial intelligence, the market size of AI products is currently unprecedentedly large. Simultaneously, due to further advancements in communication technology, AI customization has become indispensable, leading to wider industry applications and a continuously expanding range of application scenarios. In particular, the integration of cloud computing and artificial intelligence in cloud computing terminals holds a significant position in the market, with the most basic AI voice assistant's human-customized functionality playing a crucial role. Currently, there are no dedicated voice assistants for cloud computing terminals.

[0003] Therefore, how to apply AI voice assistants to cloud computer terminals, customize wake words to wake up the assistant, and fill the gap of cloud computer terminals not having their own voice assistants is a technical problem that urgently needs to be solved. Summary of the Invention

[0004] The technical objective of this invention is to provide a method, device, and medium for implementing an AI voice assistant based on a cloud computer terminal, in order to solve the problem of how to apply an AI voice assistant to a cloud computer terminal, to use a custom wake word to wake up the assistant, and to fill the gap in cloud computer terminals not having their own voice assistant.

[0005] The technical objective of this invention is achieved as follows: a method for implementing an AI voice assistant based on a cloud computer terminal, the specific method of which is as follows:

[0006] Voice wake-up and interaction based on the Android platform: The background thread decompresses the speech model, and automatically starts the microphone monitoring service after completion. The speech recognition service continuously captures microphone input and returns the recognition results in real time. When a preset wake word is detected, a floating window pops up to enter the interaction mode. During the interaction, the text display and recording status are managed, and timeout is automatically handled through delayed tasks.

[0007] Speech recognition and interactive control based on streaming audio: The background collects 5 seconds of audio data and transmits it to the speech recognition engine frame by frame. The speech recognition engine parses the audio frame by frame. When a complete sentence is obtained, a callback is triggered. The recognition results are then subjected to semantic filtering, including removing wake word prefixes. Then, an exit command is executed or a dialogue interaction is started. The floating window content is updated in real time, and the interaction timeout is controlled by a delayed task.

[0008] Asynchronous voice Q&A response: A separate thread is started through a thread pool to handle network requests, avoiding blocking of the main thread. The user's voice recognition results are encapsulated as form data and sent to the local server via HTTP POST. After the server returns JSON data, the client parses and extracts the core answer text, and passes the successfully parsed answer through a callback interface, or the error information is notified through a dedicated callback.

[0009] Asynchronous speech synthesis and transmission: The main thread submits speech synthesis tasks to the thread pool, executes the synthesis process asynchronously, generates a secure WebSocket connection address, creates a temporary storage file, streams speech data through WebSocket, synchronously writes it to an MP3 file, uses CountDownLatch to implement asynchronous operations to complete signal synchronization, supports timeout control, returns the file path on successful synthesis, and notifies the error through callback on failure.

[0010] Audio playback after successful speech synthesis: Reliable playback of the MP3 file resulting from speech synthesis is achieved through resource management that releases old resources and creates new instances, a secure file access mechanism (FileProvider), and thread-safe operations (UI thread switching).

[0011] As a preferred option, the voice wake-up and interaction based on the Android platform are as follows:

[0012] The `initModel()` method is used to initialize the model and perform background decompression: In a background thread, the `StorageService.unpack()` method decompresses the speech recognition model named "vosk-model-small-cn-0.22" to the local "model" directory. If decompression is successful, the speech recognition model instance is assigned to `this.model`, and `recognizeMicrophone()` is triggered to start the speech recognition service. If decompression fails, `setErrorState()` is used to set the error state and log the exception information.

[0013] The speech recognition service is managed through the `recognizeMicrophone()` method: This manages the lifecycle of the speech recognition service: if `speechService` already exists (not null), the current service is stopped and resources are released; if the service is not running, a `Recognizer` object (using a 16000Hz sampling rate) and the `SpeechService` are created, and microphone listening is started using `startListening()`; exception handling: IOException is captured and error logs are recorded, while error information is fed back using `setErrorState()`.

[0014] The wake word detection and response are performed through the onResult() callback method: triggered when the speech recognition engine returns a result, it checks whether the recognition result hypothesis contains a preset wake word; if the match is successful, a floating window is displayed on the main thread through showFloatingWindow(), triggering the subsequent interaction process;

[0015] The user interface is managed through the controlView() method: the floating window text is updated to "Please state your needs...", the history is cleared, the audio-to-text module (mRecordingToText.startRecording()) is started, the previously delayed delayedRunnable task is canceled, and a 20-second (20000 milliseconds) delay task is reset for timeout control or automatic termination of interaction.

[0016] As a preferred embodiment, the speech recognition and interactive control based on streaming audio are as follows:

[0017] The recording thread is initialized and data acquisition is implemented using the startRecordingThread() method: a Flowable is created. <bytebuffer>The audio data source asynchronously executes the recording task through a thread;

[0018] Configure the streaming speech recognition engine: build a RecognitionParam parameter object, and call the recognizer.streamCall() method to establish a streaming recognition connection. Subscribe to the recognition result stream: when the sentence end flag (isSentenceEnd()) is detected, extract the final transcribed text transResult; pass the recognition result through the callback interface mCallback.onRecognitionSuccess(), or trigger onRecognitionFailure() when it fails; among which, building the RecognitionParam parameter object specifically means: specifying the speech recognition model as "paraformer-realtime-v2", the audio format as PCM, the sampling rate as 16000Hz, and configuring the API Key (which needs to be replaced with a valid key and supports loading from environment variables);

[0019] Process the speech interaction result through the onRecognitionSuccess(String text) callback method: for special instructions: if the recognized text starts with "Are you there", continue to process after filtering the prefix; for pure "Are you there" or "Are you there.", directly return; when the "Exit" instruction is recognized, close the floating window; for the interaction control logic: cancel the previous delayed task delayedRunnable, and reset the 20-second timeout task (for automatically ending the interaction due to timeout); update the text of the floating window in the UI thread, and start the answering recording module RecordingAnswer to generate a response.

[0020] More preferably, create Flowable <bytebuffer>The audio data source executes recording tasks asynchronously via a thread, as detailed below:

[0021] Configure the audio format and start AudioRecord recording, then verify the recording status (make sure it enters RECORD STATE_RECORDING);

[0022] Use a 16KB buffer to read 5 seconds of audio data in a loop. After each read, push the data stream using `emitter.onNext()` and reset the buffer.

[0023] Threads sleep for 10 milliseconds to control CPU usage and achieve energy-efficient recording;

[0024] The data stream ends after 5 seconds by triggering emitter.onComplete().

[0025] As a preferred option, asynchronous voice question-and-answer response is as follows:

[0026] Asynchronous task execution: Create an elastic thread pool using Executors.newCachedThreadPool() to execute network request tasks asynchronously, avoiding blocking the main thread, and use Log.d to record request content within the thread task for easy debugging and tracing;

[0027] The HTTP request construction and sending process is as follows: Form data construction: Use FormBody.Builder to construct the POST request body, encapsulating the user's question as the value of the parameter "string"; Request object creation: Construct a POST request pointing to the local server endpoint http: / / 127.0.0.1:5269 / receive_string, carrying the constructed form data; Synchronous request execution: Send a synchronous HTTP request through client.newCall(request).execute() and obtain the server response;

[0028] Response processing and parsing: Read the response body string response.body().string(), obtain the raw JSON data, parse the raw JSON data, trigger the onAnswer Error callback to report parsing errors when JSONException is caught, print the stack trace when IOException is caught, and record network request exceptions;

[0029] Resource cleanup and log tracking: The `startAnswer:f inish!` block records task completion logs, ensuring execution regardless of success or failure.

[0030] More specifically, the parsing of the original JSON data is as follows:

[0031] Parse the response string into a JSONObject object;

[0032] Extract nested objects from the data field and retrieve the text field value (automatically handles Unicode encoding conversion);

[0033] The parsed result is passed to the caller via the callback interface mCallback.onAnswerReceived(textValue).

[0034] As a preferred method, asynchronous speech synthesis and transmission are as follows:

[0035] Asynchronous speech synthesis task scheduling: The sendToServer3 method is used to start asynchronous tasks in the thread pool via Executors.newCachedThreadPool() to avoid blocking the main thread. The parameter audioResponse is defined as the text content to be synthesized, and callback is used to pass the synthesis result or error status. When an exception is caught, the caller is notified of failure via mCallback.onTranslationFailure().

[0036] Speech is synthesized using the static method `startTransSpeech`: An authenticated WebSocket URL is generated using `getAuthUrl`, and the HTTPS protocol is replaced with WSS to establish a secure connection. A temporary MP3 file (prefixed "audio_") is generated in the public music directory to store the synthesized speech data. `CountDownLatch` is used to convert asynchronous to synchronous operation, ensuring the main thread waits until transmission is complete. Specifically, during initialization, `latch.countDown()` resets the state, and re-initializing it to 1 ensures subsequent blocking and waiting.

[0037] WebSocket transmission and file writing are specifically configured as follows: Asynchronous transmission configuration: The WebSocket connection is established by calling the `websocketWork` method, passing in the authenticated URL, the output stream `outputStream` (used to write voice binary data), and a custom callback interface. The custom callback interface includes `onCompleted()` and `onError()`. `onCompleted()` is used to trigger `latch.concurrentDown()` to release the waiting thread when transmission is complete; `onError()` is used to release the lock and throw a runtime exception when an exception occurs. Blocking and waiting with timeout handling: The current thread is blocked using `latch.await(2, TimeUnit.MINUTES)`, waiting for a maximum of 2 minutes; a `TimeoutException` is thrown upon timeout to avoid infinite waiting.

[0038] Result processing and resource management: After successful transmission, the absolute path of the temporary file is assigned to OUTPUT_FILE_PATH and returned. The FileOutputStream is automatically closed using try-with-resources to ensure resource release. The temporary file is stored in a public music directory for easy access or playback later.

[0039] As a preferred option, the audio playback after successful speech synthesis is as follows:

[0040] Thread switching and parameter validation: runOnUiThread() ensures that subsequent operations are executed on the UI main thread, guaranteeing the safety of interface operations; at the same time, it checks whether the passed file path filepath is not empty: if it is empty, an error log is recorded and the process is terminated.

[0041] Audio player resource management: First, call mAudioPlayer.releasePlayer() to release the old player instance to avoid resource leaks, and set the player reference to null. Then, create a new AudioPlayer instance, rebind the current class as a playback listener (to implement playback status callbacks), and execute initialization().

[0042] Securely obtain file URI: Use FileProvider.getUriForFile() to convert the file path into a secure URI, and resolve file access permission issues in Android 7.0 and above by specifying the application package name + ".fileprovider" authorization method; if the URI acquisition fails (e.g., the file does not exist), a Toast message "File does not exist" will pop up; if successful, playback will be triggered.

[0043] Playback control and status tracking: After successfully obtaining the URI, call mAudioPlayer.play(audioUri) to start playback. The status changes during playback (such as start, pause, finish) will be triggered by the listener callback bound to setPlaybackListener(this), and key nodes (such as successful path acquisition) will be recorded in Log.d for easy debugging and tracking.

[0044] An electronic device includes: a memory and at least one processor;

[0045] The memory contains computer programs;

[0046] The at least one processor executes the computer program stored in the memory, causing the at least one processor to perform the AI ​​voice assistant implementation method based on the cloud computer terminal as described above.

[0047] A computer-readable storage medium storing a computer program that can be executed by a processor to implement the AI ​​voice assistant implementation method based on a cloud computer terminal as described above.

[0048] The AI ​​voice assistant implementation method, device, and medium based on cloud computer terminals of the present invention have the following advantages:

[0049] (i) This invention interfaces with open-source APIs to realize the most basic voice functions of AI. Customers can customize wake words to enable users to receive voice answers from voice input, providing users with a more convenient experience and sensory experience. It can also be distinguished from other voice assistants on the market, thus achieving a unique voice assistant, which plays a crucial role in securing a place in the AI ​​market in the future.

[0050] (II) This invention wakes up the assistant by customizing the wake word and asks it some simple questions. It connects to voice platforms such as Baidu, Alibaba, and iFlytek, converts the speech into text and sends it to the platform again. The platform returns the answer to the question, converts the text into speech, and presents it to the user in the form of speech and text. The implementation of the voice assistant has played a crucial role in the development of cloud computers in the market.

[0051] (III) This invention combines an open-source voice assistant platform to realize your own AI voice assistant. You can use a custom wake word to wake up the assistant and ask the assistant some questions. The assistant can intelligently give the correct answer. You can also issue commands to the assistant. The assistant will execute the correct instructions according to the commands. For example, if you say "Open Himalaya", the voice assistant will open the Himalaya application after recognizing the command.

[0052] (iv) This invention realizes a complete closed loop from model loading, speech recognition, wake word detection to interactive control, and is applicable to scenarios such as intelligent voice assistants and voice wake-up. It ensures interface smoothness and background task efficiency through asynchronous threads and callback mechanisms.

[0053] (V) This invention achieves low-latency speech recognition through streaming processing, and combines thread management and callback mechanisms to ensure efficient system operation, making it suitable for scenarios such as intelligent voice assistants and real-time voice interaction.

[0054] (vi) This invention realizes closed-loop processing from speech recognition results to server-side question answering, and ensures system response efficiency through asynchronous threads and callback mechanisms. It is suitable for scenarios that require real-time interaction, such as intelligent voice assistants and conversational AI.

[0055] (vii) This invention balances efficiency and reliability through an asynchronous thread + synchronous lock mechanism, and is suitable for scenarios such as speech synthesis and text-to-speech that require background processing and return of results. At the same time, it ensures resource security and system stability through temporary files and timeout control.

[0056] (viii) This invention achieves reliable playback of speech synthesis results (MP3 files) through strict resource management (releasing old resources → creating new instances), a secure file access mechanism (FileProvider), and thread-safe operations (UI thread switching), making it suitable for intelligent interactive scenarios that require dynamic generation and playback of speech. Attached Figure Description

[0057] The invention will be further described below with reference to the accompanying drawings.

[0058] Appendix Figure 1 A flowchart illustrating the implementation method of an AI voice assistant based on a cloud computer terminal;

[0059] Appendix Figure 2 This is a diagram illustrating the usage process of an AI voice assistant based on a cloud computer terminal. Detailed Implementation

[0060] The following detailed description of the AI ​​voice assistant implementation method, device, and medium based on a cloud computer terminal of the present invention is provided with reference to the accompanying drawings and specific embodiments.

[0061] Example 1:

[0062] As attached Figure 1 As shown in the figure, this embodiment provides a method for implementing an AI voice assistant based on a cloud computer terminal. The method is as follows:

[0063] S1. Voice wake-up and interaction based on the Android platform: The background thread decompresses the speech model and automatically starts the microphone monitoring service after completion. The speech recognition service continuously captures microphone input and returns the recognition results in real time. When a preset wake-up word is detected, a floating window pops up to enter the interaction mode. During the interaction, the text display and recording status are managed, and timeout is automatically handled through delayed tasks.

[0064] S2. Speech recognition and interactive control based on streaming audio: The background collects 5 seconds of audio data and transmits it to the speech recognition engine frame by frame. The speech recognition engine parses the audio frame by frame. When a complete sentence is obtained, a callback is triggered. The recognition results are then subjected to semantic filtering, including removing wake word prefixes. Then, an exit command is executed or a dialogue interaction is started. The floating window content is updated in real time, and the interaction timeout is controlled by a delayed task.

[0065] S3, Asynchronous Voice Q&A Response: A separate thread is started through a thread pool to handle network requests, avoiding blocking of the main thread. The user's voice recognition results are encapsulated as form data and sent to the local server via HTTP POST. After the server returns JSON data, the client parses and extracts the core answer text, and passes the successfully parsed answer through a callback interface, or the error information is notified through a dedicated callback.

[0066] S4. Asynchronous speech synthesis and transmission: The main thread submits speech synthesis tasks to the thread pool, executes the synthesis process asynchronously, generates a secure WebSocket connection address, creates a temporary storage file, streams speech data through WebSocket, synchronously writes it to an MP3 file, uses CountDownLatch to achieve asynchronous operation to complete signal synchronization, supports timeout control, returns the file path on successful synthesis, and notifies the error through callback on failure.

[0067] S5. Audio playback after successful speech synthesis: Reliable playback of the MP3 file resulting from speech synthesis is achieved through resource management that releases old resources and creates new instances, a secure file access mechanism (FileProvider), and thread-safe operations (UI thread switching).

[0068] The voice wake-up and interaction based on the Android platform in step S1 of this embodiment are as follows:

[0069] S101. Model initialization and background decompression are implemented through the initModel() method: In a background thread, the speech recognition model named "vosk-model-small-cn-0.22" is decompressed to the local "model" directory using the StorageService.unpack() method. If decompression is successful, the speech recognition model instance is assigned to this.model, and recognizeMicrophone() is triggered to start the speech recognition service. If decompression fails, the error state is set using setErrorState() and the exception information is recorded.

[0070] S102. Manage the speech recognition service using the recognizeMicrophone() method: Manage the lifecycle of the speech recognition service: If the speechService already exists (not null), stop the current service and release resources; if the service is not running, create a Recognizer object (using a 16000Hz sampling rate) and the speechService, and start microphone listening using startListening(); Exception handling: Catch I / OExceptions and log errors, and simultaneously use setErrorState() to report error information.

[0071] S103. Wake-up word detection and response via the onResult() callback method: Triggered when the speech recognition engine returns a result, it checks whether the recognition result hypothesis contains a preset wake-up word. If the match is successful, a floating window is displayed on the main thread via showFloatingWindow(), triggering the subsequent interaction process.

[0072] S104. Manage the user interface through the controlView() method: Update the floating window text to "Please state your needs...", clear the history, start the audio-to-text module (mRecordingToText.startRecording()), cancel the previously delayed delayedRunnable task, and reset the 20-second (20000 milliseconds) delay task for timeout control or automatic termination of interaction.

[0073] The key code is as follows:

[0074]

[0075]

[0076]

[0077] The specific details of the voice recognition and interactive control based on streaming audio in step S2 of this embodiment are as follows:

[0078] S201. Initialize the recording thread and acquire data using the startRecordingThread() method: Create a Flowable <bytebuffer>The audio data source asynchronously executes the recording task through a thread;

[0079] S202. Configure the streaming speech recognition engine: Construct a RecognitionParam parameter object, and call the recognizer.streamCall() method to establish a streaming recognition connection and subscribe to the recognition result stream: When the sentence end flag (isSentenceEnd()) is detected, extract the final transcribed text transResult; Pass the recognition result through the callback interface mCallback.onRecognitionSuccess(), or trigger onRecognitionFailure() when it fails; Among them, constructing the RecognitionParam parameter object specifically means: Specify the speech recognition model as "paraformer-realtime-v2", the audio format as PCM, the sampling rate as 16000Hz, and configure the API Key (which needs to be replaced with a valid key and supports loading from environment variables);

[0080] S203. Process the voice interaction result through the onRecognitionSuccess(String text) callback method: For special instructions: If the recognized text starts with "Are you there", filter the prefix and continue to process; For pure "Are you there" or "Are you there.", directly return; When the "Exit" instruction is recognized, close the floating window; For the interaction control logic: Cancel the previous delayed task delayedRunnable, and reset the 20-second timeout task (used for automatic end of interaction due to timeout); Update the text of the floating window in the UI thread, and start the answering recording module RecordingAnswer to generate a response.

[0081] The creation of Flowable in step S201 of this embodiment <bytebuffer>The audio data source executes the recording task asynchronously via a thread as follows:

[0082] S20101. Configure the audio format and start AudioRecord recording, and verify the recording status (make sure it enters RECORDSTATE_RECORDING);

[0083] S20102: Use a 16KB buffer to read 5 seconds of audio data in a loop. After each read, push the data stream through emitter.onNext() and reset the buffer.

[0084] S20103: Thread sleep for 10 milliseconds controls CPU utilization, achieving energy-efficient recording;

[0085] S20104, after 5 seconds, emitter.onComplete() is triggered to end the data stream.

[0086] The key code is as follows:

[0087]

[0088]

[0089]

[0090]

[0091]

[0092]

[0093] The asynchronous voice question-and-answer response in step S3 of this embodiment is as follows:

[0094] S301. Asynchronous task execution: Create an elastic thread pool using Executors.newCachedThreadPool() to execute network request tasks asynchronously, avoiding blocking the main thread, and use Log.d to record request content within the thread task for easy debugging and tracing.

[0095] S302, HTTP Request Construction and Sending, specifically: Form Data Construction: Use FormBody.Builder to construct the POST request body, encapsulating the user's question as the value of the parameter "string"; Request Object Creation: Construct a POST request pointing to the local server endpoint http: / / 127.0.0.1:5269 / receive_string, carrying the constructed form data; Synchronous Request Execution: Send a synchronous HTTP request through client.newCall(request).execute() to obtain the server response;

[0096] S303, Response Processing and Parsing: Read the response body string response.body().string(), obtain the raw JSON data, parse the raw JSON data, trigger the onAnswerError callback to report parsing errors when JSONException is caught, print the stack trace when IOException is caught, and record network request exceptions;

[0097] S304, Resource Cleanup and Log Tracking: Record task completion logs in the finally block: startAnswer:finish!, ensuring execution regardless of success or failure.

[0098] The specific steps for parsing the original JSON data in step S303 of this embodiment are as follows:

[0099] S30301. Parse the response string into a JSONObject object;

[0100] S30302. Extract the nested objects in the data field and obtain the text field value (automatically handle Unicode encoding conversion);

[0101] S30303: The parsed result is passed to the caller through the callback interface mCallback.onAnswerReceived(textValue).

[0102] The key code is as follows:

[0103]

[0104]

[0105]

[0106] The asynchronous speech synthesis and transmission in step S4 of this embodiment are as follows:

[0107] S401, Asynchronous speech synthesis task scheduling: The sendToServer3 method is used to start asynchronous tasks in the thread pool via Executors.newCachedThreadPool() to avoid blocking the main thread. The parameter audioResponse is defined as the text content to be synthesized, and callback is used to pass the synthesis result or error status. When an exception is caught, the caller is notified of failure via mCallback.onTranslationFailure().

[0108] S402. Speech is synthesized using the static method startTransSpeech: The authenticated WebSocket URL is generated using getAuthUrl, and the HTTPS protocol is replaced with WSS to establish a secure connection. A temporary MP3 file (prefixed "audio_") is generated in the public music directory to store the synthesized speech data. CountDownLatch is used to convert asynchronous to synchronous operation, ensuring that the main thread waits until the transmission is complete. Specifically, during initialization, latch.countDown() is set to reset the state, and it is initialized to 1 again to ensure subsequent blocking and waiting.

[0109] S403, WebSocket transmission and file writing, specifically: Asynchronous transmission configuration: A WebSocket connection is established by calling the `websocketWork` method, passing in the authenticated URL, the file output stream `outputStream` (used for writing voice binary data), and a custom callback interface; the custom callback interface includes `onCompleted()` and `onError()`; `onCompleted()` is used to trigger `latch.countDown()` to release the waiting thread when transmission is complete; `onError()` is used to release the lock and throw a runtime exception when an exception occurs; Blocking and waiting and timeout handling: The current thread is blocked using `latch.await(2, TimeUnit.MINUTES)`, waiting for a maximum of 2 minutes; a `TimeoutException` is thrown upon timeout to avoid infinite waiting;

[0110] S404. Result Processing and Resource Management: After successful transmission, the absolute path of the temporary file is assigned to OUTPUT_FILE_PATH and returned. The FileOutputStore is automatically closed using try-with-resources to ensure resource release. The temporary file is stored in the public music directory for easy access or playback later.

[0111] The key code is as follows:

[0112]

[0113]

[0114] The audio playback after successful speech synthesis in step S5 of this embodiment is as follows:

[0115] S501, Thread Switching and Parameter Validation: runOnUiThread() ensures that subsequent operations are executed on the UI main thread, guaranteeing the security of interface operations; at the same time, it checks whether the passed file path filepath is not empty: if it is empty, it records the error log and terminates the process;

[0116] S502, Audio Player Resource Management: First, call mAudioPlayer.releasePlayer() to release the old player instance to avoid resource leaks, and set the player reference to null. Then, create a new AudioPlayer instance, rebind the current class as a playback listener (to implement playback status callbacks), and execute initialize().

[0117] S503, Securely Obtain File URI: Use FileProvider.getUriForFile() to convert the file path into a secure URI, and resolve file access permission issues in Android 7.0+ by specifying the application package name + ".fileprovider" authorization method; if the URI acquisition fails (e.g., the file does not exist), a Toast message "File does not exist" will pop up; if successful, playback will be triggered.

[0118] S504 Playback Control and Status Tracking: After successfully obtaining the URI, call mAudioPlayer.play(audioUri) to start playback. The status changes during playback (such as start, pause, finish) will be triggered by the listener bound by setPlaybackListener(this) and key nodes (such as successful path acquisition) will be recorded in Log.d for easy debugging and tracking.

[0119] The key code is as follows:

[0120]

[0121]

[0122] As attached Figure 2 As shown, the working process of the voice assistant in this embodiment is as follows:

[0123] ① Wake-up: Wake up the voice assistant by inputting a wake-up word and then input the question you want to ask into the voice assistant.

[0124] ② Voice monitoring: After the voice assistant listens to the user's voice, it saves the listened voice locally;

[0125] ③ Speech to text: The monitored speech is transmitted to the open source platform (Tongyi Qianwen, Xunfei Voice) via API. After receiving the speech, the open source platform (Tongyi Qianwen, Xunfei Voice) converts the speech into text and returns it to the cloud computer client voice assistant.

[0126] ④ Obtaining text answers from the dictionary: The voice assistant sends the received text to the open-source platform again via API. After receiving the text information, the open-source platform finds the most suitable answer in the answer library and returns it to the voice assistant. After receiving the text answer, the voice assistant sends the text answer back to the platform via API.

[0127] ⑤ Text-to-speech: The voice assistant sends the received text to the open-source platform via API. The open-source platform converts the received text into speech and returns the speech to the voice assistant.

[0128] ⑥ Play voice: The voice assistant plays the received voice answer.

[0129] Example 2:

[0130] This embodiment also provides an electronic device, including: a memory and a processor;

[0131] The memory stores the instructions executed by the computer.

[0132] The processor executes computer execution instructions stored in the memory, causing the processor to execute the AI ​​voice assistant implementation method based on a cloud computer terminal in any embodiment of the present invention.

[0133] The processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor can be a microprocessor or any conventional processor.

[0134] Memory can be used to store computer programs and / or modules. The processor implements various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory. Memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, at least one application program required for a function, etc.; the data storage area can store data created based on the use of the terminal, etc. In addition, memory can also include high-speed random access memory, and can also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart memory cards (SMC), secure digital cards (SD cards), flash memory cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0135] Example 3:

[0136] This embodiment also provides a computer-readable storage medium storing multiple instructions, which are loaded by a processor to cause the processor to execute the AI ​​voice assistant implementation method based on a cloud computer terminal according to any embodiment of the present invention. Specifically, a system or device equipped with a storage medium may be provided, on which software program code implementing the functions of any of the above embodiments is stored, and the computer (or CPU or MPU) of the system or device may read and execute the program code stored in the storage medium.

[0137] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of the present invention.

[0138] Storage media embodiments for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RYM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0139] Furthermore, it should be clear that not only can the program code read by the computer be executed, but also the operating system or other components operating on the computer can be instructed based on the program code to perform some or all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0140] Furthermore, it is understood that the program code read from the storage medium is written to the memory set in the expansion board inserted into the computer or to the memory set in the expansion unit connected to the computer. Then, based on the instructions of the program code, the CPU or other components installed on the expansion board or expansion unit execute some and all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.< / bytebuffer> < / bytebuffer> < / bytebuffer> < / bytebuffer>

Claims

1. A method for implementing an AI voice assistant based on a cloud computer terminal, characterized in that, The method is as follows: Voice wake-up and interaction based on the Android platform: The background thread decompresses the speech model, and automatically starts the microphone monitoring service after completion. The speech recognition service continuously captures microphone input and returns the recognition results in real time. When a preset wake word is detected, a floating window pops up to enter the interaction mode. During the interaction, the text display and recording status are managed, and timeout is automatically handled through delayed tasks. Speech recognition and interactive control based on streaming audio: The background collects 5 seconds of audio data and transmits it to the speech recognition engine frame by frame. The speech recognition engine parses the audio frame by frame. When a complete sentence is obtained, a callback is triggered. The recognition results are then subjected to semantic filtering, including removing wake word prefixes. Then, an exit command is executed or a dialogue interaction is started. The floating window content is updated in real time, and the interaction timeout is controlled by a delayed task. Asynchronous voice Q&A response: A separate thread is started through a thread pool to handle network requests, avoiding blocking of the main thread. The user's voice recognition results are encapsulated as form data and sent to the local server via HTTP POST. After the server returns JSON data, the client parses and extracts the core answer text, and passes the successfully parsed answer through a callback interface, or the error information is notified through a dedicated callback. Asynchronous speech synthesis and transmission: The main thread submits speech synthesis tasks to the thread pool, executes the synthesis process asynchronously, generates a secure WebSocket connection address, creates a temporary storage file, streams speech data through WebSocket, synchronously writes it to an MP3 file, uses CountDownLatch to implement asynchronous operations to complete signal synchronization, supports timeout control, returns the file path on successful synthesis, and notifies the error through callback on failure. Audio playback after successful speech synthesis: Reliable playback of the MP3 file resulting from speech synthesis is achieved through resource management that releases old resources and creates new instances, a secure file access mechanism, and thread-safe operations.

2. The method for implementing an AI voice assistant based on a cloud computer terminal according to claim 1, characterized in that, The voice wake-up and interaction based on the Android platform are as follows: The model initialization and background decompression are implemented through the initModel() method: In the background thread, the speech recognition model named "vosk-model-small-cn-0.22" is decompressed to the local "model" directory through the StorageService.unpack() method: If the decompression is successful, the speech recognition model instance is assigned to this.model, and the recognizeMicrophone() method is triggered to start the speech recognition service. If decompression fails, set the error state using setErrorState() and record the exception information; Manage the speech recognition service through the recognizeMicrophone() method: Manage the lifecycle of the speech recognition service: If the speechService already exists, stop the current service and release resources first; if the service is not running, create a Recognizer object and a SpeechService service, and start listening to the microphone through startListening(); Exception handling: Catch IOException and record error logs, and at the same time feedback error information through setErrorState(); Perform wake word detection and response through the onResult() callback method: Triggered when the speech recognition engine returns a result, detect whether the preset wake word is included in the recognition result hypothesis: If the match is successful, display a floating window in the main thread through showFloatingWindow() to trigger the subsequent interaction process; Manage the user interface through the controlView() method: Update the floating window text to "Please state your needs...", clear the history, start the voice-to-text module, cancel the previously delayed delayedRunnable task, and reset the 20-second delayed task for timeout control or automatic interaction termination.

3. The method for implementing an AI voice assistant based on a cloud computer terminal according to claim 1, characterized in that, The speech recognition and interaction control based on streaming audio are as follows: The recording thread is initialized and data acquisition is implemented using the startRecordingThread() method: a Flowable is created. <bytebuffer>The audio data source asynchronously executes the recording task through a thread;< / bytebuffer> Configure the streaming speech recognition engine: Build a RecognitionParam parameter object and call the recognizer.streamCall() method to establish a streaming recognition connection and subscribe to the recognition result stream: When the sentence end flag (isSentenceEnd()) is detected, extract the final transcribed text transResult; Pass the recognition result through the callback interface mCallback.onRecognitionSuccess(), or trigger onRecognitionFailure() when it fails; Among them, building the RecognitionParam parameter object specifically means: Specify the speech recognition model as "paraformer-realtime-v2", the audio format as PCM, the sampling rate as 16000Hz, and configure the API Key; Process the speech interaction result through the onRecognitionSuccess(String text) callback method: For special instructions: If the recognized text starts with "Are you there", continue to process after filtering the prefix; For pure "Are you there" or "Are you there.", directly return; When the "Exit" instruction is recognized, close the floating window; For the interaction control logic: Cancel the previous delayed task delayedRunnable and reset the 20-second timeout task; Update the floating window text in the UI thread and start the answering recording module RecordingAnswer to generate a response.

4. The method for implementing an AI voice assistant based on a cloud computer terminal according to claim 3, characterized in that, Create Flowable <bytebuffer>The audio data source asynchronously executes the recording task through a thread, specifically as follows:< / bytebuffer> Configure the audio format and start AudioRecord recording, then verify the recording status. Use a 16KB buffer to read 5 seconds of audio data in a loop. After each read, push the data stream using `emitter.onNext()` and reset the buffer. Threads sleep for 10 milliseconds to control CPU usage and achieve energy-efficient recording; The data stream ends after 5 seconds by triggering emitter.onComplete().

5. The method for implementing an AI voice assistant based on a cloud computer terminal according to claim 1, characterized in that, The asynchronous voice question-and-answer response is as follows: Asynchronous task execution: Create an elastic thread pool using Executors.newCachedThreadPool() to execute network request tasks asynchronously, avoiding blocking the main thread, and use Log.d to record request content within the thread task for easy debugging and tracing; The HTTP request construction and sending process is as follows: Form data construction: Use FormBody.Builder to construct the POST request body, encapsulating the user's question as the value of the parameter "string"; Request object creation: Construct a POST request pointing to the local server endpoint http: / / 127.0.0.1:5269 / receive_string, carrying the constructed form data; Synchronous request execution: Send a synchronous HTTP request through client.newCall(request).execute() and obtain the server response; Response processing and parsing: Read the response body string response.body().string(), obtain the raw JSON data, parse the raw JSON data, trigger the onAnswer Error callback to report parsing errors when JSONException is caught, print the stack trace when IOException is caught, and record network request exceptions; Resource cleanup and log tracking: The `startAnswer:finish!` block records task completion logs, ensuring execution regardless of success or failure.

6. The method for implementing an AI voice assistant based on a cloud computer terminal according to claim 5, characterized in that, The specific steps for parsing the raw JSON data are as follows: Parse the response string into a JSONObject object; Extract the nested object from the data field and get the value of the text field; The parsed result is passed to the caller via the callback interface mCallback.onAnswerReceived(textValue).

7. The method for implementing an AI voice assistant based on a cloud computer terminal according to claim 1, characterized in that, The asynchronous speech synthesis and transmission are detailed below: Asynchronous speech synthesis task scheduling: The sendToServer3 method is used to start asynchronous tasks in the thread pool via Executors.newCachedThreadPool() to avoid blocking the main thread. The parameter audioResponse is defined as the text content to be synthesized, and callback is used to pass the synthesis result or error status. When an exception is caught, the caller is notified of failure via mCallback.onTranslationFailure(). Speech is synthesized using the static method `startTransSpeech`: an authenticated WebSocket URL is generated using `getAuthUrl`, and the HTTPS protocol is replaced with WSS to establish a secure connection. A temporary MP3 file is generated in a public music directory to store the synthesized speech data. CountDownLatch is used to convert asynchronous to synchronous operation, ensuring that the main thread waits until the transmission is complete. Specifically, during initialization, `latch.countDown()` is set to reset the state, and re-initializing it to 1 ensures subsequent blocking and waiting. WebSocket transmission and file writing are specifically configured as follows: Asynchronous transmission configuration: The `websocketWork` method is called to establish a WebSocket connection, passing in the authenticated URL, the file output stream `outputStream`, and a custom callback interface. The custom callback interface includes `onCompleted()` and `onError()`. `onCompleted()` is used to trigger `latch.countDown()` to release waiting threads when transmission is complete; `onError()` is used to release the lock and throw a runtime exception when an exception occurs. Blocking and waiting with timeout handling: The current thread is blocked using `latch.await(2, TimeUnit.MINUTES)`, waiting for a maximum of 2 minutes; a `TimeoutException` is thrown upon timeout to avoid infinite waiting. Result processing and resource management: After successful transmission, the absolute path of the temporary file is assigned to OUTPUT_FILE_PATH and returned. The FileOutputStream is automatically closed using try-with-resources to ensure resource release. The temporary file is stored in a public music directory for easy access or playback later.

8. The method for implementing an AI voice assistant based on a cloud computer terminal according to claim 1, characterized in that, The audio playback after successful speech synthesis is as follows: Thread switching and parameter validation: runOnUiThread() ensures that subsequent operations are executed on the UI main thread, guaranteeing the safety of interface operations; at the same time, it checks whether the passed file path filepath is not empty: if it is empty, an error log is recorded and the process is terminated. Audio player resource management: First, call mAudioPlayer.releasePlayer() to release the old player instance to avoid resource leaks, and set the player reference to null. Then, create a new AudioPlayer instance, rebind the current class as a playback listener, and execute the initialize() function. Securely obtain file URI: Use FileProvider.getUriForFile() to convert the file path into a secure URI, and resolve file access permission issues in Android 7.0 and above by specifying the application package name + ".fileprovider" authorization method; if the URI acquisition fails, a Toast message "File does not exist" will pop up; if successful, playback will be triggered. Playback control and status tracking: After successfully obtaining the URI, call mAudioPlayer.play(audioUri) to start playback. The status changes during playback will be triggered by the listener callback bound by setPlaybackListener(this), and key nodes will be recorded in Log.d for easy debugging and tracking.

9. An electronic device, characterized in that, include: Memory and at least one processor; The memory contains computer programs; The at least one processor executes the computer program stored in the memory, causing the at least one processor to perform the AI ​​voice assistant implementation method based on a cloud computer terminal as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that can be executed by a processor to implement the AI ​​voice assistant implementation method based on a cloud computer terminal as described in any one of claims 1 to 8.

Images