Voice interaction method and device

Abstract

The invention provides a voice interaction method and device, and solves the problems of high project cost and difficulty in extension and maintenance caused by incapability of adapting to frequent requirements and logic line changes in the prior art. Through the voice detection node, the voice state machine node and the audio processing node, voice detection, question and answer state control and audio processing are realized, and voice interaction is realized. And meanwhile, the cost of the project is greatly reduced, and subsequent expansion and maintenance of the project are facilitated.

Description

Technical Field

[0001] The present invention relates to the field of human-computer interaction technology, and in particular to a voice interaction method and device. Background Art

[0002] ROS, the full name of Robot Operating System, is not an operating system in the traditional sense, but a flexible framework for robot software development. It provides a series of services designed to support the entire robot application development process from simulation to deployment, including hardware abstraction, underlying driver control, communication functions, visualization tools, message passing mechanisms, package management, etc. In the current ROS system, the common solution is for developers to use the logical flow between nodes to realize the recognition and judgment of human voice, or to create multiple nodes and run each function separately.

[0003] However, in some projects, frequent changes in requirements and cumbersome logic lines lead to increasingly complex task logic in the later stages. During on-site development or when project delivery is urgent, it takes a certain amount of time to sort out the task logic. Repairing or adding one of the details may even introduce new problems. Especially when the project is large, complex task interactions will make it difficult to expand and maintain. Summary of the invention

[0004] The present invention provides a voice interaction method and device to solve the above problems.

[0005] In a first aspect, the present invention provides a voice interaction method, which specifically comprises the following steps: The human voice is detected in real time by a voice detection node. When no human voice is detected, the voice detection node publishes a first state value topic. When a human voice is detected and no wake-up word is detected, the voice detection node publishes a second state value topic and records and saves the audio data containing the human voice. When a human voice and a wake-up word are detected, the voice detection node publishes a third state value topic and records and saves the audio data containing the human voice. Receiving a state value topic through a voice state machine node, when the state of the voice state machine node is an idle state (Idle state, I state) no talking stage (No Talking stage, NT stage) and a third state value topic is received, the state of the voice state machine node is an idle state waking stage (Waking stage, W stage), when the state of the voice state machine node is an idle state no talking stage and a first state value topic is received, the state of the voice state machine node is an idle state no talking stage, when the state of the voice state machine node is an idle state no talking stage and a second state value topic is received, the state of the voice state machine node is an idle state speaking stage (S stage); In the present application, the speech state machine node defaults to an idle state and a silent stage before receiving a state value topic.

[0006] When the state of the voice state machine node enters the idle state speaking stage, the human voice is monitored and recorded. If no human voice is detected, the first counter in the voice state machine node is started until the value of the first counter reaches a first threshold value, and the voice state machine node state enters the end talking stage (End Talking stage, ET stage) of the model sending and audio playing state (P_Play state, P state); When the state of the speech state machine node enters the end-of-speech phase of the model sending and audio playing state, the audio recorded during the monitoring process is input into the large language model, and processed by the large language model to form a processing result of the large language model; When the processing result of the large language model is audio, the processing result of the large language model is played. When the processing result of the large language model is played, the state of the voice state machine node enters the free question and answer state (F_Talking state, F state) wake-up stage and monitors the human voice. When the human voice is not detected again within N seconds after the human voice is detected, the state of the voice state machine node enters the idle state non-speaking stage. When the human voice is detected again within N seconds after the human voice is detected, the state of the voice state machine node enters the free question and answer state speaking stage. After the state of the voice state machine node enters the free question and answer state speaking stage and the human voice is not detected again, the second counter in the voice state machine node is started until the value of the second counter reaches the second threshold value, and the state of the voice state machine node enters the model sending and voice playing state to end the speaking stage; wherein N is a positive integer; When an audio processing task is required, the audio processing task is transmitted to an audio processing node and processed by the audio processing node until the audio processing task is completed; wherein the processing of the audio processing node includes playing fixed audio files, streaming audio output, outputting Agent instructions and publishing Agent task topics.

[0007] Preferably, when the voice detection node performs real-time human voice detection, if a voice wake-up word is detected, it is judged whether there is a voice interaction task in the voice state machine node. When a voice wake-up word exists, the voice interaction task performed in the voice state machine node is interrupted, and the voice state machine node starts to execute a new task according to the voice wake-up word.

[0008] Preferably, when the first counter is started and before reaching the first threshold, if human voice is monitored, the first counter is cleared and restarted, otherwise, the state of the voice state machine node is an idle state and no-speaking stage and fixed audio is played.

[0009] In the present application, the fixed audio refers to the fixed audio played directly, for example, "no human voice received"; the file saved in the fixed configuration folder can be played by directly calling the corresponding file name.

[0010] Preferably, when the voice state machine node state is in the end speaking phase of the model sending and audio playback state and in the process of playing streaming audio, the voice detection node publishes a third state value topic. After the voice state machine node receives the third state value topic, when detecting whether the audio playback is completed, the voice state machine node performs a state detection on the Action of the request for large language model processing. If it is detected that the Action of the request for large language model processing is in the Abort state (termination state, i.e., audio playback is terminated), the voice state machine node enters the speaking phase of the idle state.

[0011] Preferably, after the large language model is processed, the audio recorded during the monitoring process is deleted to avoid wasting resources.

[0012] Preferably, when an audio processing task is required, the audio processing task is transmitted to an audio processing node and processed by the audio processing node until the audio processing task is completed, which specifically includes the following steps: When the audio processing node receives a fixed audio playback task, it calls the file name of the fixed audio and plays the fixed audio; When the audio processing node receives the audio data containing human voice recorded and saved by the speech detection node, it inputs the audio data into the large language model, processes it through the large language model, and outputs audio or Agent instructions.

[0013] Preferably, after the audio processing task is completed, the voice state machine node is waited for to issue a request for the next audio processing task.

[0014] Preferably, the fixed audio is played in a blocking and uninterruptible manner.

[0015] Preferably, when the audio processing node receives the audio data containing human voice recorded and saved by the speech detection node, the audio data is streamed into the large language model. After being processed by the large language model, if audio is output, the output audio is streamed out, and after the audio task is completed, the audio state machine node is returned to wait for the voice state machine node to issue a request for the next audio processing task; wherein, the process of streaming input or output of audio is a non-blocking, interruptible or a blocking, interruptible process; if an Agent instruction is output, the Agent instruction is published to the Agent task topic, and after the audio task is completed, the audio state machine node is returned to wait for the voice state machine node to issue a request for the next audio processing task.

[0016] Preferably, when an audio processing task is required, the audio processing task is transmitted to an audio processing node, processed by the audio processing node and the processing result is output; this process is implemented through the Action of the ROS system.

[0017] Preferably, when the voice state machine node and the audio processing node play fixed audio, the voice detection node stops human voice detection.

[0018] In a second aspect, the present invention further provides a voice interaction device, which specifically includes the following modules: A first processing module is used to detect human voice in real time through a voice detection node. When no human voice is detected, the voice detection node publishes a first state value topic. When a human voice is detected and no wake-up word is detected, the voice detection node publishes a second state value topic and records and saves the audio data containing the human voice. When a human voice and a wake-up word are detected, the voice detection node publishes a third state value topic and records and saves the audio data containing the human voice. A second processing module is used to receive a state value topic through a voice state machine node, when the state of the voice state machine node is an idle state non-speaking stage and a third state value topic is received, the state of the voice state machine node is an idle state wake-up stage, when the state of the voice state machine node is an idle state non-speaking stage and a first state value topic is received, the state of the voice state machine node is an idle state non-speaking stage, when the state of the voice state machine node is an idle state non-speaking stage and a second state value topic is received, the state of the voice state machine node is an idle state speaking stage; The third processing module is used for monitoring and recording the human voice when the state of the voice state machine node enters the idle state speaking stage. If no human voice is detected, the first counter in the voice state machine node is started until the value of the first counter reaches the first threshold value, and the state of the voice state machine node enters the end speaking stage of the model sending and audio playing state; The fourth processing module is used to input the audio recorded during the monitoring process into the large language model when the state of the voice state machine node enters the end-speech stage of the model sending and audio playing state, and process it through the large language model to form a processing result of the large language model; The fifth processing module is used to play the processing result (i.e., play the audio) when the processing result of the large language model is audio. When the processing result is played, the state of the voice state machine node enters the free question and answer state awakening stage and monitors the human voice. When the human voice is not detected again within N seconds after the human voice is detected, the state of the voice state machine node enters the idle state non-speaking stage. When the human voice is detected again within N seconds after the human voice is detected, the state of the voice state machine node enters the free question and answer state speaking stage. After the state of the voice state machine node enters the free question and answer state speaking stage and the human voice is not detected again, the second counter in the voice state machine node is started until the value of the second counter reaches the second threshold value, and the state of the voice state machine node enters the model sending and voice playing state end speaking stage; wherein N is a positive integer; The sixth processing module is used to transmit the audio processing task to the audio processing node when an audio processing task is required, and process it through the audio processing node until the audio processing task is completed; wherein the processing of the audio processing node includes playing fixed audio files, streaming output audio, outputting Agent instructions and publishing Agent task topics.

[0019] Preferably, when the voice detection node performs real-time human voice detection, if a voice wake-up word is detected, it is judged whether there is a voice interaction task in the voice state machine node. When a voice wake-up word exists, the voice interaction task performed in the voice state machine node is interrupted, and the voice state machine node starts to execute a new task according to the voice wake-up word.

[0020] Preferably, when the first counter is started and before reaching the first threshold, if human voice is monitored, the first counter is cleared and restarted, otherwise, the state of the voice state machine node is an idle state and no-speaking stage and fixed audio is played.

[0021] Preferably, when the speech state machine node state is in the end speaking phase of the model sending and audio playing state and in the process of playing streaming audio, the speech detection node publishes a third state value topic. After the speech state machine node receives the third state value topic, when detecting whether the audio playback is completed, the speech state machine node performs a state detection on the Action of the request for large language model processing. If it is detected that the Action of the request for large language model processing is in the Abort state, the speech state machine node enters the speaking phase of the idle state.

[0022] Preferably, after the large language model is processed, the audio recorded during the monitoring process is deleted to avoid wasting resources.

[0023] Preferably, the sixth processing module specifically includes the following submodules: The first submodule is used for calling the file name of the fixed audio and playing the fixed audio when the audio processing node receives a fixed audio playing task; The second submodule is used to input the audio data containing human voice recorded and saved by the speech detection node into the large language model after the audio processing node receives it, process it through the large language model, and output audio or Agent instructions.

[0024] Preferably, after the audio processing task is completed, the voice state machine node is waited for to issue a request for the next audio processing task.

[0025] Preferably, the fixed audio is played in a blocking and uninterruptible manner.

[0026] Preferably, when the audio processing node receives the audio data containing human voice recorded and saved by the speech detection node, the audio data is streamed into the large language model. After being processed by the large language model, if audio is output, the output audio is streamed out, and after the audio task is completed, the audio state machine node is returned to wait for the voice state machine node to issue a request for the next audio processing task; wherein, the process of streaming input or output of audio is a non-blocking, interruptible or a blocking, interruptible process; if an Agent instruction is output, the Agent instruction is published to the Agent task topic, and after the audio task is completed, the audio state machine node is returned to wait for the voice state machine node to issue a request for the next audio processing task.

[0027] Preferably, when an audio processing task is required, the audio processing task is transmitted to an audio processing node, processed by the audio processing node and the processing result is output; this process is implemented through the Action of the ROS system.

[0028] Preferably, when the voice state machine node and the audio processing node play fixed audio, the voice detection node stops human voice detection.

[0029] In a third aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a voice interaction method as described in any one of the first aspects of the present application.

[0030] In a fourth aspect, the present invention further provides an electronic device, comprising: a memory storing a computer program; and a processor communicatively connected to the memory, for executing a voice interaction method described in any one of the first aspects of the present application when the computer program is called.

[0031] Compared with the prior art, the present invention has the following obvious outstanding substantive features and significant advantages: The present invention proposes a voice interaction method and device, which solves the problem that the prior art cannot adapt to frequent demands and logic line changes, which leads to high project costs and difficulty in expansion and maintenance. The voice detection node, the voice state machine node and the audio processing node respectively realize voice detection, question and answer state control and audio processing, thereby realizing voice interaction. At the same time, it also greatly reduces the cost of the project and facilitates the subsequent expansion and maintenance of the project. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings, which constitute part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention. In the accompanying drawings: Figure 1 It is a schematic diagram of node interaction in a voice interaction method according to a preferred embodiment of the present invention.

[0033] Figure 2 It is a schematic diagram of the operation of a voice detection node in a voice interaction method according to a preferred embodiment of the present invention.

[0034] Figure 3 It is a schematic diagram of the operation of speech state machine nodes in a speech interaction method in a preferred embodiment of the present invention.

[0035] Figure 4 It is a schematic diagram of the operation of a speech processing node in a speech interaction method in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0036] The present invention provides a voice interaction method and device. To make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0038] To further illustrate the present invention, the relevant basic knowledge of the present invention is briefly introduced.

[0039] In the ROS system, each node is independent and different from each other. Therefore, data transmission between nodes requires a certain mechanism. The ROS system provides three message communication mechanisms by default, namely topics, services, and actions.

[0040] A topic is a piece of data (which can be numbers, characters, etc.) published from a node. It can be understood as being published to a public space (not a node). Any node connected to this public space can obtain the topic data just published. There are no blocking requirements or permission requirements. Any node can obtain the topic data. Topics are one-time data.

[0041] Service is different, just like two people talking on the phone. Once the number is broadcast, if the other party answers (data received successfully) or does not answer (data not received or reception failed) and other return status, the program will not proceed further. It is blocking, and only user A and user B can communicate in it. The service is also one-time.

[0042] Action is a message communication mechanism of ROS system, similar to making a phone call. For example, user A calls user B. Before user B answers the call, the waiting tone output by user A's phone receiver indicates that the status returned by Action is being executed. At this time, user A is the client and user B is the client. After user B answers the call, it can be understood that the status of Action is successful. If user B does not answer the call for a long time, the status of Action is failed. Action can be blocking or non-blocking; when Action is non-blocking, user A can play with the phone while talking to user B, and it does not affect the process of playing with the phone; when Action is blocking, if no certain status (such as success or failure) is returned, it will be in a waiting state. In addition, Action can also actively give up communication, such as user A actively giving up communication with user B. According to the characteristics of Action, it is used in the wake-up interruption function of this application; wherein, the status of the Action includes Pending, Active, Preempted, Abort, and Succeeded.

[0043] Embodiment 1: like Figure 1-Figure 4 As shown, the voice interaction method described in this embodiment specifically includes the following steps: Step S1: perform real-time detection of human voice through a voice detection node. When no human voice is detected, the voice detection node publishes a first status value topic. When human voice is detected and no wake-up word is detected, the voice detection node publishes a second status value topic and records and saves the audio data containing the human voice. When human voice and the wake-up word are detected, the voice detection node publishes a third status value topic and records and saves the audio data containing the human voice.

[0044] In this embodiment, when the voice detection node performs real-time human voice detection, if a voice wake-up word is detected, it is judged whether there is a voice interaction task in the voice state machine node. When a voice wake-up word exists, the voice interaction task performed in the voice state machine node is interrupted, and the voice state machine node starts executing a new task according to the voice wake-up word.

[0045] In the drawings in the specification, Idle represents the idle state (I state), F_Talking represents the free question and answer state (F state), P_Play represents the model sending and audio playing state (P state); NoTalking represents the non-talking stage (NT stage), Waking represents the waking stage (W stage), Speaking represents the speaking stage (S stage), and EndTalking represents the end of the speaking stage (ET stage).

[0046] Step S2, receiving a state value topic through a voice state machine node, when the state of the voice state machine node is an idle state non-speaking stage and receives a third state value topic, the state of the voice state machine node is an idle state wake-up stage, when the state of the voice state machine node is an idle state non-speaking stage and receives a first state value topic, the state of the voice state machine node is an idle state non-speaking stage, when the state of the voice state machine node is an idle state non-speaking stage and receives a second state value topic, the state of the voice state machine node is an idle state speaking stage.

[0047] Step S3: When the state of the voice state machine node enters the idle state speaking stage, the human voice is monitored and recorded. If no human voice is detected, the first counter in the voice state machine node is started until the value of the first counter reaches a first threshold. The voice state machine node state enters the end speaking stage of the model sending and audio playback state.

[0048] Among them, when the first counter is started and before reaching the first threshold, if human voice is monitored, the first counter is cleared and counted again, otherwise, the state of the voice state machine node is an idle state and no speaking stage and plays fixed audio.

[0049] Step S4: When the state of the speech state machine node enters the end-speaking stage of the model sending and audio playing state, the audio recorded during the monitoring process is input into the large language model, and processed by the large language model to form a processing result of the large language model.

[0050] Step S5, when the processing result of the large language model is audio, play the processing result of the large language model, when the processing result of the large language model is played, the state of the voice state machine node enters the free question and answer state awakening stage and monitors the human voice, when the human voice is not detected again within N seconds after the human voice is detected, the state of the voice state machine node enters the idle state non-speaking stage, when the human voice is detected again within N seconds after the human voice is detected, the state of the voice state machine node enters the free question and answer state speaking stage, when the state of the voice state machine node enters the free question and answer state speaking stage and the human voice is not detected again, the second counter in the voice state machine node is started until the value of the second counter reaches the second threshold, and the state of the voice state machine node enters the model sending and voice playing state to end the speaking stage; wherein N is a positive integer.

[0051] Step S6: When an audio processing task is required, the audio processing task is transmitted to an audio processing node, and processed by the audio processing node until the audio processing task is completed; wherein the processing of the audio processing node includes playing fixed audio files, streaming audio output, outputting Agent instructions, and publishing Agent task topics.

[0052] In this embodiment, when the state of the voice state machine node is the end speaking phase of the model sending and audio playing state and the streaming audio is being played, the voice detection node publishes a third state value topic. After the voice state machine node receives the third state value topic, when detecting whether the audio is played, the voice state machine node performs a state detection on the Action of the request for large language model processing. If it is detected that the Action of the request for large language model processing is in the Abort state, the voice state machine node enters the speaking phase of the idle state.

[0053] Optionally, after the large language model is processed, the audio recorded during the monitoring process is deleted to avoid wasting resources.

[0054] Optionally, in step S6, when an audio processing task is required, the audio processing task is transmitted to an audio processing node, and processed by the audio processing node until the audio processing task is completed, which specifically includes the following steps: Step S601: When the audio processing node receives a fixed audio playback task, it calls the file name of the fixed audio and plays the fixed audio; Step S602: After the audio processing node receives the audio data containing human voice recorded and saved by the speech detection node, it inputs the audio data into the large language model, processes the audio data through the large language model, and outputs audio or Agent instructions.

[0055] When the audio processing task is completed, the voice state machine node is waited for to issue a request for the next audio processing task.

[0056] In this embodiment, the fixed audio is played in a blocking and uninterruptible manner.

[0057] In this embodiment, when the audio processing node receives the audio data containing human voice recorded and saved by the voice detection node, the audio data is streamed into the large language model. After being processed by the large language model, if audio is output, the output audio is streamed out, and after the audio task is completed, the voice state machine node is returned to wait for the voice state machine node to issue a request for the next audio processing task; wherein, the process of streaming input or output of audio is a non-blocking, interruptible or a blocking, interruptible process; if an Agent instruction is output, the Agent instruction is published to the Agent task topic, and after the audio task is completed, the voice state machine node is returned to wait for the voice state machine node to issue a request for the next audio processing task.

[0058] In this embodiment, when an audio processing task is required, the audio processing task is transmitted to an audio processing node, processed by the audio processing node and the processing result is output; this process is implemented through the Action of the ROS system.

[0059] When the voice state machine node and the audio processing node play fixed audio, the voice detection node stops human voice detection.

[0060] Embodiment 2: The voice interaction device described in this embodiment specifically includes a first processing module, a second processing module, a third processing module, a fourth processing module, a fifth processing module and a sixth processing module.

[0061] The first processing module is used to detect human voice in real time through a voice detection node. When no human voice is detected, the voice detection node publishes a first status value topic. When human voice is detected and no wake-up word is detected, the voice detection node publishes a second status value topic and records and saves the audio data containing the human voice. When human voice and the wake-up word are detected, the voice detection node publishes a third status value topic and records and saves the audio data containing the human voice.

[0062] In this embodiment, when the voice detection node performs real-time human voice detection, if a voice wake-up word is detected, it is judged whether there is a voice interaction task in the voice state machine node. When a voice wake-up word exists, the voice interaction task performed in the voice state machine node is interrupted, and the voice state machine node starts executing a new task according to the voice wake-up word.

[0063] The second processing module receives the state value topic through the voice state machine node. When the state of the voice state machine node is the idle state non-speaking stage and receives the third state value topic, the state of the voice state machine node is the idle state wake-up stage. When the state of the voice state machine node is the idle state non-speaking stage and receives the first state value topic, the state of the voice state machine node is the idle state non-speaking stage. When the state of the voice state machine node is the idle state non-speaking stage and receives the second state value topic, the state of the voice state machine node is the idle state speaking stage.

[0064] The third processing module is used to monitor and record the human voice when the state of the voice state machine node enters the idle state speaking stage. If no human voice is detected, the first counter in the voice state machine node is started until the value of the first counter reaches a first threshold, and the voice state machine node state enters the model sending and audio playback state end speaking stage.

[0065] In this embodiment, when the first counter is started and before reaching the first threshold, if human voice is monitored, the first counter is cleared and restarted. Otherwise, the state of the voice state machine node is an idle state and no-speaking stage and fixed audio is played.

[0066] The fourth processing module is used to input the audio recorded during the monitoring process into the large language model when the state of the voice state machine node enters the end-speaking stage of the model sending and audio playing state, and process it through the large language model to form a processing result of the large language model.

[0067] The fifth processing module is used to play the processing result when the processing result of the large language model is audio. After the processing result is played, the state of the voice state machine node enters the free question and answer state awakening stage and monitors the human voice. When the human voice is not detected again within N seconds after the human voice is detected, the state of the voice state machine node enters the idle state non-speaking stage. When the human voice is detected again within N seconds after the human voice is detected, the state of the voice state machine node enters the free question and answer state speaking stage. When the state of the voice state machine node enters the free question and answer state speaking stage and the human voice is not detected again, the second counter in the voice state machine node is started until the value of the second counter reaches the second threshold value, and the state of the voice state machine node enters the model sending and voice playing state to end the speaking stage; wherein N is a positive integer.

[0068] The sixth processing module is used to transmit the audio processing task to the audio processing node when an audio processing task is required, and process it through the audio processing node until the audio processing task is completed; wherein the processing of the audio processing node includes playing fixed audio files, streaming output audio, outputting Agent instructions and publishing Agent task topics.

[0069] Among them, when the state of the voice state machine node is the end speaking stage of the model sending and audio playing state and the streaming audio is played, the voice detection node publishes a third state value topic. After the voice state machine node receives the third state value topic, when detecting whether the audio is played, the voice state machine node performs a state detection on the Action of the request for large language model processing. If it is detected that the Action of the request for large language model processing is in the Abort state, the voice state machine node enters the speaking stage of the idle state.

[0070] Optionally, after the large language model is processed, the audio recorded during the monitoring process is deleted to avoid wasting resources.

[0071] The sixth processing module specifically includes the following first submodule and second submodule.

[0072] The first submodule is used to call the file name of the fixed audio and play the fixed audio when the audio processing node receives a fixed audio playing task.

[0073] The second submodule is used to input the audio data containing human voice recorded and saved by the speech detection node into the large language model after the audio processing node receives it, process it through the large language model, and output audio or Agent instructions.

[0074] When the audio processing task is completed, the voice state machine node is waited for to issue a request for the next audio processing task.

[0075] In this embodiment, the fixed audio is played in a blocking and uninterruptible manner.

[0076] Among them, when the audio processing node receives the audio data containing human voice recorded and saved by the voice detection node, the audio data is streamed into the large language model. After being processed by the large language model, if audio is output, the output audio is streamed out, and after the audio task is completed, the voice state machine node is returned to wait for the voice state machine node to issue a request for the next audio processing task; wherein, the process of streaming input or output of audio is a non-blocking, interruptible or a blocking, interruptible process; if an Agent instruction is output, the Agent instruction is published to the Agent task topic, and after the audio task is completed, the voice state machine node is returned to wait for the voice state machine node to issue a request for the next audio processing task.

[0077] In this embodiment, when an audio processing task is required, the audio processing task is transmitted to an audio processing node, processed by the audio processing node and the processing result is output; this process is implemented through the Action of the ROS system.

[0078] In this embodiment, when the voice state machine node and the audio processing node play fixed audio, the voice detection node stops human voice detection.

[0079] The specific embodiments of the present invention are described in detail above, but they are only examples, and the present invention is not limited to the specific embodiments described above. For those skilled in the art, any equivalent modifications and substitutions made to the present invention are also within the scope of the present invention. Therefore, the equalization changes and modifications made without departing from the spirit and scope of the present invention should be included in the scope of the present invention.

Claims

1. A voice interaction method, characterized in that: The specific steps include: Use the speech detection node to detect human voice in real time and perform: When no human voice is detected, the first status value topic is published; When a human voice is detected and the wake-up word is not detected, the second state value topic is published and the audio is recorded; When human voice and wake-up words are detected, the third state value topic is published and audio is recorded; The speech state machine node performs state transitions based on the received state value topic: The state of the speech state machine node is in the I state NT stage: when receiving the third state value topic, the state is changed to the I state W stage; when receiving the first state value topic, the current state is maintained; when receiving the second state value topic, the state is changed to the I state S stage; After the state of the voice state machine node enters the I state S stage, it monitors and records human voice. If no human voice is detected, it starts the first counter. After the count reaches the first threshold, it changes the state to the P state ET stage. After the state of the speech state machine node enters the P state ET stage, the audio recorded during the monitoring process is input into the large language model for processing to form the processing result of the large language model; When the processing result is audio, the processing result is played, and after the playback is completed, the state is changed to the F state W stage to monitor the human voice: if the human voice is not detected again within N seconds, the state is changed to the I state NT stage; if the human voice is detected again, the state is changed to the F state S stage, and after entering the F state S stage and if the human voice is not detected again, the second counter is started, and the state is changed to the P state ET stage after the count reaches the second threshold; wherein N is a positive integer; Audio processing tasks are performed through audio processing nodes, including playing fixed audio files, streaming audio output, outputting Agent instructions, and publishing Agent task topics.

2. A voice interaction method according to claim 1, characterized in that: When the voice detection node performs real-time human voice detection, if a voice wake-up word is detected, it is determined whether there is a voice interaction task in the voice state machine node. When a voice wake-up word exists, the voice interaction task in the voice state machine node is interrupted, and the voice state machine node starts executing a new task according to the voice wake-up word.

3. A voice interaction method according to claim 1, characterized in that: When the first counter is started and before reaching the first threshold, if human voice is monitored, the first counter is cleared and restarted. Otherwise, the state of the voice state machine node is I state NT stage and fixed audio is played.

4. A voice interaction method according to claim 1, characterized in that: When the voice state machine node state is in the P state ET stage and the streaming audio is played, the voice detection node publishes a third state value topic. After the voice state machine node receives the third state value topic, when detecting whether the audio is played, the voice state machine node performs a state detection on the Action of the request for large language model processing. If it is detected that the Action of the request for large language model processing is in the Abort state, the voice state machine node enters the I state S stage.

5. A voice interaction method according to claim 1, characterized in that: When the large language model is processed, the audio recorded during the monitoring process is deleted to avoid wasting resources.

6. A voice interaction method according to claim 1, characterized in that: When an audio processing task is required, the audio processing task is transmitted to an audio processing node and processed by the audio processing node until the audio processing task is completed, specifically including the following steps: When the audio processing node receives a fixed audio playback task, it calls the file name of the fixed audio and plays the fixed audio; When the audio processing node receives the audio data containing human voice recorded and saved by the speech detection node, it inputs the audio data into the large language model, processes it through the large language model, and outputs audio or Agent instructions.

7. A voice interaction method according to claim 1, characterized in that: When the audio processing task is completed, the process returns to the voice state machine node and waits for the voice state machine node to issue a request for the next audio processing task.

8. A voice interaction method according to claim 1, characterized in that: When the audio processing node receives the audio data containing human voice recorded and saved by the voice detection node, the audio data is streamed into the large language model. After being processed by the large language model, if audio is output, the output audio is streamed out, and after the audio task is completed, the audio processing node returns to the voice state machine node, waiting for the voice state machine node to issue the next audio processing task; wherein, the process of streaming input or output of audio is a non-blocking, interruptible or a blocking, interruptible process; if an Agent instruction is output, the Agent instruction is published to the Agent task topic, and after the audio task is completed, the audio processing node returns to the voice state machine node, waiting for the voice state machine node to issue a request for the next audio processing task.

9. A voice interaction method according to claim 1, characterized in that: When an audio processing task is required, the audio processing task is transmitted to the audio processing node, and the audio processing node processes the task and outputs the processing result; This process is implemented through the Action of the ROS system.

10. A voice interaction method according to claim 1, characterized in that: When the voice state machine node and the audio processing node play fixed audio, the voice detection node stops human voice detection.

11. A voice interaction device, characterized in that: The modules include: The first processing module is used to detect human voice in real time through the voice detection node and perform: When no human voice is detected, the first status value topic is published; When a human voice is detected and the wake-up word is not detected, the second state value topic is published and the audio is recorded; When human voice and wake-up words are detected, the third state value topic is published and audio is recorded; The second processing module is used for the voice state machine node to perform state transition according to the received state value topic: The state of the speech state machine node is in the I state NT stage: when receiving the third state value topic, the state is changed to the I state W stage; when receiving the first state value topic, the current state is maintained; when receiving the second state value topic, the state is changed to the I state S stage; The third processing module is used for monitoring and recording human voice after the state of the voice state machine node enters the I state T stage, and if no human voice is detected, starting the first counter, and changing the state to the P state ET stage after the count reaches the first threshold; A fourth processing module is used for inputting the recorded audio into the large language model for processing after the state of the speech state machine node enters the P state ET stage to form a processing result of the large language model; The fifth processing module is used for playing the processing result when the processing result is audio, and changing the state to the F state W stage to monitor the human voice after the playing is completed: if the human voice is not detected again within N seconds, the state is changed to the I state NT stage; if the human voice is detected again, the state is changed to the F state S stage, and after entering the F state S stage and if the human voice is not detected again, the second counter is started, and the state is changed to the P state ET stage after the count reaches the second threshold; wherein N is a positive integer; The sixth processing module is used to execute audio processing tasks through audio processing nodes, including playing fixed audio files, streaming audio output, outputting Agent instructions and publishing Agent task topics.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, a voice interaction method as described in any one of claims 1 to 10 is implemented.

13. An electronic device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a voice interaction method as described in any one of claims 1 to 10 is implemented.

Images