Voice interaction method and apparatus

Abstract

The disclosure provides a voice interaction method and device, relates to the field of artificial intelligence, and in particular to the field of intelligent sound boxes. A specific implementation scheme is as follows: in response to receiving a voice wake-up instruction, first voice data is collected, and a first semantic is determined based on the first voice data; the first semantic is saved in a cache; in response to detecting a voice pause, a pause type is determined; in response to the pause type being a thinking pause and second voice data being collected, a second semantic is determined based on the second voice data; a voice instruction is determined based on the first semantic and the second semantic, and the voice instruction is executed.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, and more particularly to the field of smart speakers, specifically a voice interaction method and device. Background Technology

[0002] Currently, smart speakers, smart screens, and other smart devices commonly use voice interaction, which has become a frequent way for users to control these devices. Existing voice interaction solutions are mainly designed based on users' expression habits, employing a fixed silence timeout conversation truncation mechanism. They rely solely on the silence duration to determine whether the voice conversation has ended, without taking into account the frequent pauses and discontinuous expressions of some users. Summary of the Invention

[0003] This disclosure provides a voice interaction method, apparatus, device, storage medium, and computer program product.

[0004] According to a first aspect of this disclosure, a voice interaction method is provided, comprising: in response to receiving a voice wake-up command, acquiring first voice data, determining a first semantic based on the first voice data, and storing the first semantic in a cache; in response to detecting a voice pause, determining a pause type; in response to the pause type being a thinking pause and acquiring second voice data, determining a second semantic based on the second voice data; determining a voice command based on the first semantic and the second semantic, and executing the voice command.

[0005] According to a second aspect of this disclosure, a voice interaction device is provided, comprising: a preprocessing unit configured to, in response to receiving a voice wake-up command, acquire first voice data, determine a first semantic based on the first voice data, and store the first semantic in a cache; a judging unit configured to, in response to detecting a voice pause, determine a pause type; a continuation unit configured to, in response to a pause type being a thinking pause and acquiring second voice data, determine a second semantic based on the second voice data; and a fusion unit configured to, based on the first semantic and the second semantic, determine a voice command and execute the voice command.

[0006] According to a third aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described in any one of the first aspects.

[0007] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method described in any one of the first aspects.

[0008] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method described in any one of the first aspects.

[0009] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein: Figure 1 This is an exemplary system architecture diagram to which one embodiment of this disclosure can be applied; Figure 2 This is a flowchart of an embodiment of the voice interaction method according to the present disclosure; Figure 3 This is a schematic diagram of an application scenario of the voice interaction method according to this disclosure; Figure 4 This is a flowchart of yet another embodiment of the voice interaction method according to the present disclosure; Figure 5 This is a schematic diagram of the structure of a voice interaction device according to an embodiment of the present disclosure; Figure 6 This is a schematic diagram of the structure of a computer system suitable for implementing embodiments of the present disclosure. Detailed Implementation

[0011] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0012] Figure 1 An exemplary system architecture 100 is shown that can be applied to embodiments of the voice interaction methods or voice interaction devices disclosed herein.

[0013] like Figure 1 As shown, system architecture 100 may include smart devices 101, home appliances 102 and 103, and server 104. A network is used as the medium to provide communication links between smart devices 101, home appliances 102 and 103, and server 104. The network may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0014] The smart device 101 can interact with home appliances 102 and 103 via a server for voice control and other functions. Various communication client applications can be installed on the smart device 101, such as voice interaction applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, and social media platforms.

[0015] The smart device 101 can be either hardware or software. When the smart device 101 is hardware, it can be various electronic devices with speakers and microphones that support voice interaction, including but not limited to smart speakers, smartphones, smart screens, personal computers, laptops, and desktop computers. When the smart device 101 is software, it can be installed in the electronic devices listed above. It can be implemented as multiple software programs or software modules (e.g., to provide distributed services) or as a single software program or software module. No specific limitations are made here.

[0016] The Smart Device 101 can be implemented using existing hardware (microphone, front-facing camera) such as smart speakers and smart screens, without the need for additional hardware. Its core consists of 4 functional units: Preprocessing unit: Real-time acquisition of user voice content through microphone array, basic noise reduction, simultaneous extraction of speech text and semantic features, temporary storage of semantic features while acquisition, and storage of speech segments; Judgment Unit: Combining silence duration, environmental state, and user interaction posture, accurately distinguish between "thinking pause breakpoint" and "sentence end breakpoint"; Continuation Unit: Locally stores incomplete speech and audio segments and corresponding semantic fragments for subsequent breakpoint continuation and splicing; Fusion Unit: Calls the semantic fusion algorithm to splice the preceding and following speech segments to generate a complete sentence, completes semantic recognition and executes the corresponding function; at the same time, it integrates a timeout fallback mechanism to avoid unnecessary power consumption and misidentification.

[0017] Household appliances 102 and 103 can be various electronic devices that can be controlled by smart devices, including but not limited to air conditioners, refrigerators, washing machines, and robot vacuums.

[0018] Server 104 can be a server that provides various services and can manage smart devices 101, home appliances 102, and 103. Server 104 can also be a cloud server that provides the information required by smart devices 101, such as obtaining weather forecasts and real-time traffic information.

[0019] Alternatively, without using a server, the smart device 101 can communicate directly with the home appliances 102 and 103.

[0020] It's important to note that a server can be either hardware or software. When a server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When a server is software, it can be implemented as multiple software programs or software modules (e.g., multiple software programs or software modules used to provide distributed services), or as a single software program or software module. No specific limitations are made here. A server can also be a server for a distributed system, or a server integrated with blockchain technology. A server can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0021] It should be noted that the voice interaction method provided in the embodiments of this disclosure is generally executed by the smart device 101, and correspondingly, the voice interaction device is generally disposed in the smart device 101.

[0022] It should be understood that Figure 1 The number of smart devices, networks, home appliances, and servers shown is merely illustrative. Any number of smart devices, networks, home appliances, and servers can be included depending on the implementation requirements.

[0023] Continue to refer to Figure 2 The diagram illustrates a flow 200 of an embodiment of a voice interaction method according to the present disclosure. The voice interaction method includes the following steps: Step 201: In response to receiving a voice wake-up command, first voice data is collected, and first semantics is determined based on the first voice data, and the first semantics is stored in a cache.

[0024] In this embodiment, the execution subject of the voice interaction method (e.g. Figure 1 The smart device (shown) is in standby listening mode in real time, continuously monitoring voice. When it detects and receives a voice wake-up command from the user, it automatically activates the voice interruption resume function. It continuously collects the first voice data input by the user. The smart device can have a built-in semantic parsing model to perform noise reduction, feature extraction, and semantic recognition processing on the collected first voice data, accurately parsing the corresponding first semantic meaning. At the same time, it opens a local temporary cache space to store the first semantic meaning in real time, avoiding the loss of early voice semantic data and providing a data foundation for subsequent semantic splicing and integration. Optionally, the smart device can also send the first voice data to a server, where the server performs the semantic parsing process and then returns the result to the smart device.

[0025] Step 202: In response to the detection of a speech pause, determine the pause type.

[0026] In this embodiment, during the acquisition of the first voice data and the user's continuous voice expression, the voice input status is monitored in real time. When a voice interruption or a silent pause is detected, the pause type determination logic is immediately triggered. Combining multiple dimensions of information, such as pause duration, user's historical interaction habits, and the current interaction scenario, the voice pause is classified and determined, distinguishing different pause types such as thinking pauses, ending pauses, and disruptive pauses. This accurately identifies the user's true intention behind the pause and avoids misjudgments caused by relying solely on duration determination. Disruptive pauses can be filtered out without processing.

[0027] Step 203: In response to the pause type being a thinking pause and the acquisition of second speech data, determine the second semantic based on the second speech data.

[0028] In this embodiment, if the current pause type is determined to be a user thinking pause, it means that the user is only briefly thinking during the expression process and has not completed the complete instruction expression. At this time, the audio recording function remains on and the session is not closed, continuously listening to and collecting the second voice data emitted by the user after the pause ends. After the second voice data is collected, the built-in semantic parsing model is reused to intelligently parse and extract the semantics of the second voice data, accurately obtaining the corresponding second semantic information, and realizing the semantic parsing of segmented voice data separately.

[0029] Step 204: Determine the voice command based on the first semantic and the second semantic, and execute the voice command.

[0030] In this embodiment, the first semantic segment stored in the cache is retrieved and combined with the second semantic segment obtained through real-time parsing. The two semantic segments are logically concatenated, semantically fused, and error-corrected, eliminating invalid semantic information and establishing a coherent and complete semantic logic to ultimately determine the user's complete voice command. Subsequently, the corresponding device function is invoked, the operation task corresponding to the voice command is matched, and the corresponding device operation is automatically executed to complete the complete voice interaction process.

[0031] For example, when a user needs to query health data and intermittently says "Help me check... today's blood pressure record", the system first collects the first voice data of "Help me check" and parses and caches the first semantic. After detecting and determining that the user is pausing to think, the system continuously collects the second voice data of "today's blood pressure record" and parses the second semantic. Finally, the two semantics are merged to obtain a complete voice command, which executes the blood pressure record query operation. This effectively adapts to the user's intermittent expression habits and ensures smooth interaction.

[0032] The method provided in the above embodiments of this disclosure employs segmented acquisition and semantic caching, which can retain parsed semantics during pauses in user expression, avoiding semantic loss due to premature truncation of the voice conversation, and adapting to the user's habit of thinking and speaking simultaneously. By distinguishing the types of voice pauses, it can accurately identify pauses in thought, maintain continuous audio recording, effectively distinguish between pauses in expression and the end of the conversation, and reduce the probability of misjudgment caused by environmental noise, non-vocal intervals, etc. The semantics corresponding to multiple voice segments are integrated and spliced ​​to form complete instructions, eliminating the need for users to repeatedly wake up the device and repeat the content, reducing operation steps and interaction burden, and improving the accuracy of voice recognition and the smoothness of interaction. The overall solution overcomes the limitations of judging only the duration of silence, realizes the recognition of breakpoint continuation in intermittent voice, and significantly optimizes the overall voice interaction user experience.

[0033] In some optional implementations of this embodiment, determining the pause type includes: obtaining the environmental state, wherein the environmental state includes at least one of the following: current time, current location, and current ambient noise; determining a pause duration threshold based on the environmental state; determining the pause type as a thinking pause in response to the pause duration being less than the pause duration threshold; and determining the pause type as the end of a statement in response to the pause duration being greater than or equal to the pause duration threshold.

[0034] Upon detecting a pause in voice input, the system can collect the current environmental status, which can be selected from one or more of the following: current time, current location, and real-time ambient noise. Combining the acquired environmental status data, the system dynamically calculates and matches a pause duration threshold suitable for the current scenario.

[0035] The actual detected pause duration is compared with this threshold: if the pause duration is less than the set threshold, the pause is determined to be a thinking pause; if the pause duration is greater than or equal to the threshold, the pause is determined to be the end of a sentence. After determination, the device executes the corresponding subsequent voice interaction logic according to the different pause types.

[0036] For example, when a voice interruption is detected, the device detects that the ambient noise is too high and adjusts the pause duration threshold accordingly. If the user's short pause duration does not exceed the threshold, it is identified as a pause for thought, and the device continues to receive audio and wait for subsequent voice. If the user does not speak for a long time and the pause duration reaches or exceeds the threshold, it is determined that the sentence has ended and the current voice conversation is terminated.

[0037] When a pause is detected during voice interaction, the local current time is obtained, and the pause duration threshold can be configured differently based on day and night: the pause duration threshold set during the daytime is greater than that at night. For example, the pause duration threshold is set to 3.5 seconds during the daytime (6:00-22:00) and 2 seconds at night (22:00-6:00 the next day). If a user's voice pause lasts 2.8 seconds during the daytime, which is less than 3.5 seconds, it is identified as a thinking pause; if the pause lasts 3.5 seconds or more, the statement is considered to have ended. At night, if a user's pause lasts 1.5 seconds, which is less than 2 seconds, it is identified as a thinking pause; if the pause lasts 2 seconds or more, the voice expression is considered to have completed.

[0038] Upon detecting a pause in speech, the system uses a location module to determine the current location, categorizes the scene based on location attributes, and matches a corresponding pause duration threshold. In noisy, busy public places, the pause duration threshold is set longer; in quiet, private indoor spaces, the threshold is set shorter.

[0039] Compare the actual pause duration with the corresponding location threshold: if the pause duration is less than the threshold, it is determined to be a thinking pause, and the device continues to pick up audio; if the pause duration is greater than or equal to the threshold, it is determined to be the end of the sentence, and the current voice interaction ends.

[0040] For example, the preset pause duration threshold is 4 seconds in public places such as shopping malls and streets, and 2.5 seconds in private and quiet places such as bedrooms and studies at home. When a user uses the device in a shopping mall, a 3-second pause is considered a thinking pause; a pause of 4 seconds or more is considered the end of the sentence. In a home environment, a 2-second pause is considered a thinking pause, and a pause of 2.5 seconds or more is considered the end of the sentence.

[0041] Optionally, invalid pauses can be filtered, which is equivalent to extending the pause duration threshold. Non-thinking silent gaps such as coughing, sighing, and drinking water can be identified and filtered through a classification model. These gaps do not trigger the breakpoint continuation judgment, thus avoiding invalid waiting.

[0042] This embodiment dynamically adjusts the pause duration threshold based on environmental conditions such as time, location, and ambient noise, abandoning the fixed duration judgment mode and adapting to different usage scenarios. It distinguishes between thought pauses and sentence endings based on dynamic thresholds, effectively reducing misjudgments caused by external factors such as environmental noise and improving the accuracy of pause type recognition. It accurately distinguishes between brief thought pauses during user expression and genuine sentence endings, preventing premature interruptions and ensuring the continuity of voice interaction. The judgment logic is simple and efficient, quickly completing pause type recognition while balancing recognition accuracy and interaction response speed, optimizing the overall user experience.

[0043] Optionally, a basic pause duration threshold is obtained, and the basic pause duration threshold is adjusted based on the waker's historical voice interaction habits to obtain an initial pause duration threshold.

[0044] Smart devices pre-set a basic pause duration threshold for all users. The system identifies the user initiating each voice interaction through methods such as facial recognition and voice recognition. The system then records the user's speaking pause habits over time (e.g., average pause duration, pause frequency) and automatically adjusts the basic pause duration threshold to obtain the initial pause duration threshold. The more it is used, the more it matches the user's individual speaking rhythm. The initial pause duration threshold can also be adjusted based on environmental conditions.

[0045] In some optional implementations of this embodiment, determining the pause duration threshold based on the environmental state includes: determining the initial pause duration threshold based on the waker's historical voice interaction habits; adjusting the initial pause duration threshold based on the environmental state to obtain the final pause duration threshold.

[0046] After a pause in voice is detected during voice interaction, the system first retrieves the historical voice interaction habit data of the waker stored locally (such as average pause duration and pause frequency), and generates an initial pause duration threshold based on this data.

[0047] Then, the current environmental state is collected, and the initial pause duration threshold is dynamically corrected and fine-tuned based on the environmental state to obtain the final pause duration threshold adapted to the current scenario.

[0048] The actual detected pause duration is compared with the final pause duration threshold: if the pause duration is less than the final pause duration threshold, it is determined to be a thinking pause, and the device continues to listen to the voice while maintaining the recording state; if the pause duration is greater than or equal to the final pause duration threshold, it is determined to be the end of the sentence, and the current voice conversation is terminated.

[0049] For example, based on a user's historical interaction data, it can be determined that their daily pauses are relatively long, and an initial pause duration threshold of 3.2 seconds can be set. If significant ambient noise is detected, the initial threshold is adjusted upwards based on this environmental condition, resulting in a final pause duration threshold of 3.8 seconds. When a pause duration of 3 seconds is detected, which is less than the final threshold, it is determined to be a pause of thought; when the pause duration reaches 3.8 seconds or more, it is determined to be the end of a sentence.

[0050] In this embodiment, an initial threshold is first set based on the user's historical interaction habits, aligning with the individual expression characteristics of each user to reduce misjudgments of pause types from an individual perspective. Then, the initial threshold is dynamically adjusted based on the environmental conditions, taking into account the influence of external scenarios to ensure the threshold adapts to the current actual usage environment. This dual-layer threshold determination method of "user habits + environmental conditions" provides a more comprehensive judgment logic compared to a single fixed threshold, further improving the recognition accuracy of pauses of thought and sentence endings, and ensuring the smoothness of voice interaction.

[0051] In some optional implementations of this embodiment, determining the pause type includes: acquiring the waker's posture image; determining the waker's intention based on the posture image; determining the pause type as a thinking pause in response to the intention being thinking; and determining the pause type as the end of a statement in response to the intention being leaving.

[0052] When an interruption in voice input or a pause in voice is detected, the system immediately uses the front-facing camera of the smart device to capture a real-time image of the user's posture. The captured posture image is then used for face detection, posture recognition, and behavior analysis to determine the user's current interaction intent.

[0053] If image recognition confirms that the user's face is facing the smart screen, their body is in a normal interactive posture, and they have not left the device's field of vision, the user's intention is determined to be thinking. The current pause in speech is then identified as a pause in thinking, and the device continues to record audio, waiting for the user to continue voice input.

[0054] If the image recognition detects that the user is turning their head, looking down, or moving away from the device's field of vision, it determines that the user intends to leave, identifies the current pause in the voice as the end of the sentence, stops the smart device from recording, and terminates the current voice interaction session.

[0055] For example, if a user pauses midway through their speech, and the front-facing camera captures the user's face facing the screen without any change in posture, the system determines that the user is in a thinking state, marks it as a pause in thinking, and continues to record the audio; a moment later, the camera captures the user turning around and walking out of the recording area, the system determines that the user's intention is to leave, and then concludes the sentence, ending the voice interaction.

[0056] In this embodiment, posture images are captured by a front-facing camera, and visual posture information is integrated into the pause determination process, overcoming the limitations of relying solely on voice duration for judgment and providing richer judgment dimensions. By recognizing user interaction intentions through facial orientation, body posture, and spatial location, it can accurately distinguish between scenarios of pauses in thought and the end of a conversation, significantly reducing the probability of misjudgments caused by relying solely on voice judgment. Combining visual state to predict user behavior, it can promptly respond to the user's departure action and terminate the conversation, or it can maintain interaction while the user is pausing to think, effectively improving the intelligence level and user experience of voice interaction.

[0057] In some optional implementations of this embodiment, determining the voice command based on the first semantic and the second semantic includes: determining the fused semantic based on the first semantic and the second semantic through a large language model; and determining the voice command based on the fused semantic.

[0058] After acquiring the first and second semantic segments, both segments are input into a pre-defined large language model. The large language model, considering contextual logic and language habits, organizes, completes, and integrates the two semantic segments to generate a logically coherent and semantically complete fused semantic. Based on this fused semantic, the system parses out complete speech commands and calls the corresponding functional modules to execute them.

[0059] For example, a user might speak in two parts: the first part is "Help me check," and the second part is "This week's medication reminder." The large language model performs semantic fusion on the two parts to obtain the complete fused semantic "Help me check this week's medication reminder." Based on this, the system generates the corresponding voice command to perform the medication reminder query.

[0060] Optionally, various semantic slot information contained in the first and second semantics, such as key fields like operation actions, objects, time, and location, are extracted and combined according to preset slot rules to form a complete fused semantic. The system then parses the fused semantic to obtain the final voice command and execute it.

[0061] For example, the first semantic extraction extracts the action slot "play", and the second semantic extraction extracts the content slot "opera program". By concatenating the two slots according to the slot logic, the fused semantic "play opera program" is obtained. The system recognizes this content, generates a voice command, and controls the device to play the program.

[0062] This embodiment provides two semantic fusion implementation paths, which can be flexibly selected according to actual application scenarios, resulting in stronger adaptability. By leveraging a large language model to fuse semantics, it can clarify contextual logic, correct sentence flaws, and improve the accuracy and coherence of semantic understanding, adapting to discontinuous and fragmented speech content. The slot-based splicing method, relying on standardized field combinations to achieve semantic integration, features simple computational logic, fast response speed, reduced device computing power consumption, and ensured interaction efficiency. Both methods can integrate segmented semantics into complete content, effectively solving the semantic fragmentation problem caused by segmented speech acquisition and ensuring that voice commands can be correctly recognized and executed.

[0063] In some optional implementations of this embodiment, determining the pause type includes: in response to the first semantic being a complete semantic, determining the pause type as the end of a statement.

[0064] Upon detecting a pause in voice interaction, the parsed and cached first semantic meaning is retrieved, and its completeness is verified. If the first semantic meaning is determined to be complete and can independently form a valid instruction, the current pause type is directly identified as the end of a sentence, and the recording stops, ending the current voice conversation. If the first semantic meaning is incomplete, it is determined to be a non-end of a sentence, and the recording continues to wait for subsequent voice input.

[0065] For example, if a user says "Turn off the alarm" and then pauses, and the system parses the first semantic meaning completely, allowing it to independently execute the corresponding operation, then the pause is determined to be the end of the sentence, the device stops recording and executes the command to turn off the alarm. If the user only says "Set for tomorrow" and then pauses, the first semantic meaning is incomplete, so it is not determined to be the end of the sentence, and the system continues to listen for subsequent speech.

[0066] This embodiment, by combining the completeness of parsed semantics to assist in determining the type of pause, supplements the determination with dimensions such as duration and image, making the determination basis more comprehensive. For semantically complete voice content, the end of the session can be quickly identified, and the recording can be terminated in a timely manner to avoid unnecessary device standby and reduce resource consumption. By distinguishing between expression interruption and session completion at the semantic level, the misjudgment of pauses is further reduced, improving the accuracy and operational efficiency of voice interaction.

[0067] In some optional implementations of this embodiment, the method further includes: in response to detecting that the number of voice pauses is greater than a predetermined threshold, ending the current session and clearing the first semantic in the cache.

[0068] During intermittent voice interaction with smart devices, the system continuously and in real time counts the number of voice pauses detected in the current voice conversation and pre-configures a predetermined threshold for the number of pauses. This threshold can be flexibly set according to user habits and interaction scenarios. In this embodiment, the predetermined threshold can be set to 3 times.

[0069] When the cumulative number of pauses exceeds a preset threshold, the system determines that the user has no intention to continue interacting and the current voice interaction cannot be completed, and will proactively terminate the current voice interaction session. At the same time, the system automatically clears all temporary voice semantic data, including the first semantic data, stored in the local cache, releases device cache resources, ends the current voice interaction process, and waits for the next wake-up command.

[0070] This embodiment adds a threshold mechanism for determining the number of voice pauses. Based on the existing criteria of pause duration, user posture, and semantic integrity, it introduces a new dimension of frequency to enrich the logic for terminating a conversation, making the voice interaction judgment system more comprehensive and multi-dimensional. For scenarios where users cannot continue speaking after multiple pauses, the system can proactively terminate invalid conversations, preventing the device from continuously receiving audio and engaging in standby monitoring. This effectively reduces unnecessary power consumption, lowers system background resource usage, and improves device stability. Automatically clearing cached semantic data after conversation termination avoids the retention of invalid or incomplete semantic data from the previous round, preventing residual data from interfering with semantic recognition in subsequent voice interactions, eliminating semantic errors and recognition deviations, and ensuring the accuracy of subsequent voice interactions.

[0071] In some optional implementations of this embodiment, the method further includes: in response to a pause type of statement end, ending the current session and clearing the first semantic in the cache.

[0072] During voice interaction, after determining the type of pause, if the system identifies the pause as the end of a sentence, it immediately terminates the current voice interaction session and stops listening to voice input. Simultaneously, the system automatically clears the first semantic and other relevant voice semantic data stored in the temporary buffer, completing this round of interaction.

[0073] In this embodiment, the session is terminated promptly upon the end of a statement to prevent continuous invalid audio reception, thereby reducing device power consumption and system resource usage. After the session ends, cached semantic data is simultaneously cleared to prevent historical semantic residue and avoid interference with subsequent voice interactions, ensuring accurate semantic parsing. This forms a complete interactive loop with smooth logical connections, improving the standardization of the voice interaction process and overall operational efficiency.

[0074] In some optional implementations of this embodiment, the method further includes: in response to the failure to collect second voice data within a predetermined duration, ending the current session and clearing the first semantic data in the cache.

[0075] During the voice interaction process, when the system detects a pause in speech and determines that the pause is a thinking pause, the device maintains its recording function and initiates a preset duration timer mechanism to continuously listen for and wait for the user to continue with the second voice data, in order to complete the fusion and splicing of segmented semantics. The system can be configured with a preset duration according to the usage scenario; in this embodiment, the preset duration can be set to 5 seconds.

[0076] During the timing process, if the system detects that no second voice data from the user has been collected within the predetermined time range, it determines that the user cannot continue to complete the current voice command input, actively ends the current voice interaction session, and terminates the voice monitoring process. At the same time, the system automatically clears the first semantic data stored in the temporary cache, removes the temporary semantic information of this round of interaction, completes the closed loop of this abnormally interrupted voice interaction, and the device returns to the normal standby monitoring state.

[0077] In this embodiment, a pre-set waiting mechanism is implemented. This mechanism intelligently terminates invalid interaction processes in scenarios such as user pauses leading to forgotten instructions, temporary interruptions in interaction, or unintentional continuation of operations. This avoids prolonged continuous audio monitoring, preventing increased power consumption and misidentification of ambient noise caused by prolonged idle audio reception, thus improving device energy efficiency. Automatic reset for abnormal interaction scenarios is also implemented, eliminating the need for manual session closure by the user, simplifying the user operation process, and significantly enhancing the intelligence of voice interaction and the user experience.

[0078] See also Figure 3 , Figure 3 This is a schematic diagram illustrating an application scenario of the voice interaction method according to this embodiment. Figure 3 In the application scenario, the overall workflow is as follows: 1. Wake-up Enabled: When a user wakes up the smart device with voice (or switches to voice breakpoint resume mode), the system automatically enables the voice breakpoint resume function; 2. Voice Acquisition and Caching: Smart devices acquire voice content in real time through microphones, simultaneously extracting voice text and semantic features, and temporarily storing semantic features and / or sentence fragments in the caching unit while acquiring the data; 3. Silence Breakpoint Detection: When the audio stream enters a silent state, the judgment unit is immediately triggered. The pause type is determined by combining the silence duration, environmental conditions, and user posture. The precise rules for determining silence are as follows: (1) Thinking pause breakpoint (continuation is allowed, and the sound is maintained) If all of the following conditions are met, it is determined that the user is thinking midway through speaking and has not yet finished speaking, and the voice conversation will be maintained without being closed: ① The duration of silent pause falls within a reasonable thinking range: 0.5 seconds to 5 seconds (this threshold can be adaptively adjusted based on the user's historical speaking habits and home environment (day and night); ② During the silence period, there are no sudden loud noises or irrelevant conversations from others, ensuring that the silence is caused by the user's thinking rather than environmental factors; ③ Front-facing camera-assisted judgment: If the system detects that the user's face is facing the smart screen, maintaining a normal interactive posture, and not leaving the device's field of vision, it determines that the user intends to continue the conversation.

[0079] (2) End of statement breakpoint (terminate continuation, turn off radio)

[0080] If any of the following conditions are met, the user is deemed to have finished speaking, the audio recording is immediately turned off, and the current conversation ends: ① The pre-collected voice sentences are complete in structure and semantically closed, conforming to the logic of users' daily complete command expression (such as "help me measure my blood pressure" or "play opera"); ② The duration of silent pause exceeds the user's maximum pause limit (i.e., more than 5 seconds); ③ The front-facing camera detects that the user deviates from the interaction posture (such as turning their head or looking down) or gets up and leaves the device's field of view; ④ After the user has paused silently multiple times (≥2 times) without making any further sounds.

[0081] 4. Handling pauses in thought: If a pause in thought is detected, real-time semantic parsing is paused, voice recording is kept in standby mode, and the collected voice segments are continuously cached, waiting for the user to continue speaking; 5. Voice continuation and splicing: After the user finishes thinking and continues to speak, the system seamlessly continues to collect the second half of the voice, calls the semantic fusion algorithm, splices the semantics and / or sentences before and after, and generates complete semantics and / or complete sentences; 6. Semantic Recognition and Execution: Perform accurate semantic recognition on the concatenated complete sentence to identify the user's actual usage needs and execute the corresponding functions (such as health query, playback content, setting reminders, etc.). 7. Timeout fallback: If the preset maximum waiting time for resuming the call is reached and the user still has not continued speaking, the system will clear the cache and end the current voice interaction; 8. Interaction Closure: After the function is completed, the system automatically exits the voice interaction mode and returns to standby mode, waiting for the next voice wake-up.

[0082] Further reference Figure 4 This illustrates a flow 400 of another embodiment of the voice interaction method. Flow 400 of the voice interaction method includes the following steps: Step 401: In response to receiving a voice wake-up command, first voice data is collected, and first semantics is determined based on the first voice data. The first semantics and the first voice data are then stored in a cache.

[0083] In this embodiment, the smart device listens to the user's voice signal while in standby mode. Upon detecting and receiving the user's voice wake-up command, it collects the user's first voice data. The first voice data is then subjected to noise reduction, feature extraction, and semantic parsing to accurately identify the first semantic meaning. Unlike the method of only caching semantics in step 201, this solution synchronously stores the parsed first semantic meaning and the original collected first voice data in the device's temporary cache, while retaining the original voice material and parsing results, providing data support for subsequent error correction.

[0084] Step 402: In response to the detection of a speech pause, determine the pause type.

[0085] In this embodiment, it is basically the same as step 202, so it will not be described again.

[0086] Step 403: In response to the pause type being a thinking pause and the acquisition of second speech data, determine the second semantic based on the second speech data.

[0087] In this embodiment, it is basically the same as step 203, so it will not be described again.

[0088] Step 404: In response to the failure to determine the voice instruction based on the first semantics and the second semantics, the first voice data and the second voice data are merged into a complete sentence.

[0089] In this embodiment, the system retrieves the first semantic data from the cache and the second semantic data parsed in real time, attempting to integrate and generate a complete voice command through semantic fusion using a large language model or slot splicing. If there are situations such as semantic logic conflicts, missing slots, severe semantic fragmentation, or disordered word order, resulting in the inability to generate a valid and accurate voice command based on the dual semantics, the semantic fusion command determination is deemed to have failed. At this point, the system no longer relies on the erroneous segmented semantic results, but directly retrieves the first voice data stored in the cache and splices it with the real-time acquired second voice data to generate a complete and coherent original voice sentence, thus avoiding the recognition failure problem caused by semantic parsing errors.

[0090] Step 405: Determine the voice command based on the complete sentence.

[0091] In this embodiment, the merged complete original voice statement is re-performed with global speech recognition, noise reduction optimization, and overall semantic analysis. Based on the complete voice statement, the user's true intent is accurately identified, the final valid voice command is determined, and the corresponding device operation is executed to complete the breakpoint resume voice interaction process.

[0092] For example, a user intermittently inputs voice. The first voice data is "I want to in the morning," which is interpreted as "The user has a need in the morning," and the voice data is cached simultaneously. After a brief pause, the second voice data is "Test my blood sugar," which is interpreted as "Measure my blood sugar." When the two segments of semantics are merged separately, a logical gap occurs, and the instruction matching fails. At this point, the system retrieves the cached original voice data, merges the two audio segments "I want to in the morning" and "Test my blood sugar" into a complete voice sentence, re-recognizes the entire system, and finally determines the complete voice instruction as "Measure my blood sugar in the morning," and executes the corresponding health check reminder operation.

[0093] This embodiment synchronously caches segmented raw speech data and parsed semantics, forming a two-layer recognition logic of "semantic fusion first, original audio error correction as a fallback." This solves the problem of instruction recognition failure caused by parsing deviations and semantic fragmentation due to single semantic caching. It avoids the defects of missing details and logical inconsistencies that exist when semantics is used as secondary parsing information, fully preserving the user's original speech expression information, effectively compensating for the technical shortcomings of segmented semantic splicing failure and logical discontinuity, and improving speech recognition accuracy. It accurately adapts to interactive scenarios where users express themselves intermittently, with disjointed word order and incomplete sentences, especially adapting to the user's habit of thinking and expressing themselves simultaneously, significantly reducing the failure rate of voice interaction in complex scenarios. In normal scenarios, instructions are generated quickly through semantic fusion to ensure interaction response speed; when semantic fusion is abnormal, the original speech is automatically spliced ​​and re-recognized globally, eliminating the need for users to repeatedly wake up and repeat content, reducing the burden of interactive operations, and optimizing the overall fluency and intelligence of voice interaction.

[0094] Further reference Figure 5 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a voice interaction device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0095] like Figure 5 As shown, the voice interaction device 500 of this embodiment includes: a preprocessing unit 501, a judgment unit 502, a continuation unit 503, and a fusion unit 504. The preprocessing unit 501 is configured to, in response to receiving a voice wake-up command, collect first voice data, determine a first semantic based on the first voice data, and store the first semantic in a cache; the judgment unit 502 is configured to, in response to detecting a voice pause, determine a pause type; the continuation unit 503 is configured to, in response to the pause type being a thinking pause and collecting second voice data, determine a second semantic based on the second voice data; and the fusion unit 504 is configured to, based on the first semantic and the second semantic, determine a voice command and execute the voice command.

[0096] In this embodiment, the specific processing of the preprocessing unit 501, the judgment unit 502, the continuation unit 503, and the fusion unit 504 of the voice interaction device 500 can be referred to Figure 2 The corresponding steps are 201, 202, 203 and 204 in the embodiment.

[0097] In some optional implementations of this embodiment, the determination unit 502 is further configured to: obtain the environmental state, wherein the environmental state includes at least one of the following: current time, current location, and current ambient noise; determine a pause duration threshold based on the environmental state; determine the pause type as a thinking pause in response to the pause duration being less than the pause duration threshold; and determine the pause type as the end of a statement in response to the pause duration being greater than or equal to the pause duration threshold.

[0098] In some optional implementations of this embodiment, the judgment unit 502 is further configured to: determine the initial pause duration threshold based on the waker's historical voice interaction habits; adjust the initial pause duration threshold based on the environmental state to obtain the final pause duration threshold.

[0099] In some optional implementations of this embodiment, the determination unit 502 is further configured to: acquire the posture image of the waker; determine the waker's intention based on the posture image; determine the pause type as a thinking pause in response to the intention being thinking; and determine the pause type as the end of a statement in response to the intention being leaving.

[0100] In some optional implementations of this embodiment, the fusion unit 504 is further configured to: determine the fusion semantics based on the first semantics and the second semantics through a large language model; and determine the voice command based on the fusion semantics.

[0101] In some optional implementations of this embodiment, the preprocessing unit 501 is further configured to: save the first voice data to a cache; the fusion unit 504 is further configured to: merge the first voice data and the second voice data into a complete sentence in response to the failure to determine the voice instruction based on the first semantics and the second semantics; and determine the voice instruction based on the complete sentence.

[0102] In some optional implementations of this embodiment, the determination unit 502 is further configured to: determine the pause type as the end of a statement in response to the first semantic being a complete semantic.

[0103] In some optional implementations of this embodiment, the preprocessing unit 501 is further configured to: in response to detecting that the number of voice pauses is greater than a predetermined threshold, end the current session and clear the first semantic in the cache.

[0104] In some optional implementations of this embodiment, the preprocessing unit 501 is further configured to: terminate the current session and clear the first semantic in the cache in response to a pause type of statement end.

[0105] In some optional implementations of this embodiment, the preprocessing unit 501 is further configured to: in response to the failure to collect second voice data within a predetermined duration, end the current session and clear the first semantic in the cache.

[0106] The collection, storage, use, processing, transmission, provision, and disclosure of any type of information, such as user personal information, in this technical solution comply with relevant laws and regulations and do not violate public order and good morals.

[0107] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0108] An electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method described in process 200 or 400.

[0109] A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method described in process 200 or 400.

[0110] A computer program product includes a computer program that, when executed by a processor, implements the method described in process 200 or 400.

[0111] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0112] like Figure 6As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 may also store various programs and data required for the operation of the electronic device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0113] Multiple components in electronic device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of displays, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows electronic device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0114] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as voice interaction methods. For example, in some embodiments, the voice interaction method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the voice interaction method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the voice interaction method by any other suitable means (e.g., by means of firmware).

[0115] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0116] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0117] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or electronic device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or electronic devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0118] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0119] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0120] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0121] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0122] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A voice interaction method, comprising: In response to receiving a voice wake-up command, first voice data is collected, and a first semantic is determined based on the first voice data, and the first semantic is stored in a cache; In response to the detection of a speech pause, the type of pause is determined; In response to the pause type being a thinking pause and the acquisition of second speech data, a second semantic is determined based on the second speech data; The voice command is determined based on the first semantic and the second semantic, and the voice command is executed.

2. The method according to claim 1, wherein determining the pause type includes: Obtain the environmental state, wherein the environmental state includes at least one of the following: current time, current location, and current ambient noise; The pause duration threshold is determined based on the environmental conditions. If the duration of a speech pause is less than the pause duration threshold, the pause type is determined to be a thinking pause. In response to a pause duration greater than or equal to the pause duration threshold, the pause type is determined to be the end of a statement.

3. The method according to claim 2, wherein determining the pause duration threshold based on the environmental state includes: The initial pause duration threshold is determined based on the waker's historical voice interaction habits; The initial pause duration threshold is adjusted based on the environmental conditions to obtain the final pause duration threshold.

4. The method according to claim 2, wherein determining the pause type includes: Obtain the pose image of the waker; The intention of the awakener is determined based on the posture image; In response to the stated intention of thinking, the pause type is determined to be a thinking pause; In response to the stated intent to leave, the pause type is determined to be the end of statement.

5. The method according to any one of claims 2-4, wherein, The process of determining voice commands based on the first semantic and the second semantic includes: Based on the first semantics and the second semantics, the fusion semantics are determined through a large language model; Voice commands are determined based on the fused semantics.

6. The method according to claim 5, further comprising: Save the first voice data to the cache; In response to the failure of determining a voice instruction based on the first semantics and the second semantics, the first voice data and the second voice data are merged into a complete sentence; The voice command is determined based on the complete statement.

7. The method according to claim 1, wherein determining the pause type comprises: In response to the first semantic being complete, the pause type is determined to be the end of a statement.

8. The method according to claim 1, further comprising: In response to the detection that the number of voice pauses exceeds a predetermined threshold, the current session is terminated and the first semantic in the cache is cleared.

9. The method according to claim 1, further comprising: In response to the pause type being statement end, the current session is terminated, and the first semantic in the cache is cleared.

10. The method according to claim 1, further comprising: If no second voice data is collected within a predetermined time period, the current session ends and the first semantic data in the cache is cleared.

11. A voice interaction device, comprising: The preprocessing unit is configured to, in response to receiving a voice wake-up command, acquire first voice data, determine first semantics based on the first voice data, and store the first semantics in a cache; The judgment unit is configured to determine the type of pause in response to the detection of a speech pause; The continuation unit is configured to determine a second semantic based on the second speech data in response to the pause type being a thinking pause and the acquisition of second speech data; The fusion unit is configured to determine a voice instruction based on the first semantic and the second semantic, and to execute the voice instruction.

12. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method of any one of claims 1-10.

13. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-10.

14. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-10.

Images