Device control method and apparatus based on voice interaction, device, and storage medium

Abstract

The application discloses a device control method and device based on voice interaction, and a storage medium. A voice instruction of a user is acquired, a target desktop card identifier and a target interface identifier matched with the voice instruction are determined from preset desktop card identifiers and interface identifiers, a corresponding target desktop card is activated according to the target desktop card identifier, the target desktop card is controlled to call a target service interface corresponding to the target interface identifier, and a controlled device is controlled to execute a function corresponding to the voice instruction. A dynamic mapping mechanism between the voice instruction and an application desktop card and a service interface is established, the analysis and execution tasks of device control are sunk into the application program with original driving capability, an adaptation defect caused by the fact that a large number of private protocols cannot be preset in an operating system is effectively overcome, and therefore, the user can directly control various smart home devices through a system voice assistant without unlocking a screen or starting an application.

Description

Technical Field

[0001] This invention belongs to the field of voice interaction, and specifically relates to a device control method, apparatus, device and storage medium based on voice interaction. Background Technology

[0002] With the rapid development of IoT technology and the popularization of smart homes, users can remotely control various devices in their homes, such as lights, air conditioners, curtains, and robot vacuums, through smartphone applications (Apps). However, they also hope to further control home appliances directly with verbal commands through the voice assistant built into the mobile phone system, without having to unlock the screen or open a specific App, thus achieving a natural "speak and control" interaction.

[0003] However, due to the limitations of mobile operating system manufacturers in fully adapting to the heterogeneous underlying instructions of a massive number of brands and models of devices, the system lacks the ability to directly parse atomic operations of specific devices. As a result, voice assistants cannot directly call device functions like native applications, and cannot meet users' real needs for convenient control of smart homes. Summary of the Invention

[0004] In view of the above problems, embodiments of the present invention are proposed to provide a voice-interactive-based device control method, apparatus, device, and storage medium that overcomes or at least partially solves the above problems.

[0005] In a first aspect, embodiments of the present invention provide a device control method based on voice interaction, the method comprising: Obtain the user's voice commands; From the preset desktop card identifiers and interface identifiers, determine the target desktop card identifier and target interface identifier that match the voice command; the desktop card identifier corresponds to the desktop card of the application, and the interface identifier corresponds to the business interface of the application that is called by the desktop card; the business interface is configured to: when called, drive the application to generate control commands for the controlled device and send the control commands to the controlled device; Based on the target desktop card identifier, the corresponding target desktop card is activated, and the target desktop card is controlled to call the target service interface corresponding to the target interface identifier, so as to control the controlled device to perform the function corresponding to the voice command.

[0006] Optionally, determining the target desktop card identifier and target interface identifier that match the voice command from preset desktop card identifiers and interface identifiers includes: Extract keywords from the voice commands; Based on the keywords, a search is conducted in a preset speech analysis lexicon to determine the target desktop card identifier and target interface identifier corresponding to the keywords; the preset speech analysis lexicon stores the mapping relationship between keywords and desktop card identifiers and interface identifiers.

[0007] Optionally, the method further includes: Obtain the controlled device information and business function description information corresponding to the desktop card in the application; the business function description information includes the interface identifier corresponding to the desktop card and the function description text corresponding to the interface identifier. Based on the controlled device information and the function description text, generate keywords; Establish a mapping relationship between the keywords, the desktop card identifiers, and the interface identifiers, and store the mapping relationship in the preset speech analysis lexicon.

[0008] Optionally, the controlled device information includes at least one of the controlled device name, the room name where the controlled device is located, and the household identifier to which the controlled device belongs; the step of generating keywords based on the controlled device information and the functional description text includes: Extract at least one of the following as attribute keywords from the controlled device information: device type words, location limiting words, and home affiliation words; Extract control behavior words and control target words from the functional description text as functional keywords; The keywords are obtained based on the attribute keywords and the functional keywords.

[0009] Optionally, activating the corresponding target desktop card based on the target desktop card identifier includes: Detect the current lifecycle status of the target desktop card; If the target desktop card is in an unloaded or dormant state, the operating system will be triggered to start or wake up the target desktop card to an active state.

[0010] Optionally, controlling the target desktop card to call the target service interface corresponding to the target interface identifier includes: Parse the function parameter values ​​corresponding to the target business interface from the voice command; The function parameter values ​​are sent to the target desktop card as input parameters for the target business interface.

[0011] Optionally, the method further includes: If the target desktop card does not receive a user interaction command within a preset time after the control command is sent, a resource release command is generated to trigger the operating system to put the target desktop card into a hibernation state.

[0012] Secondly, embodiments of the present invention provide a device control apparatus based on voice interaction, the apparatus comprising: The voice command acquisition module is used to acquire the user's voice commands; The determining module is used to determine the target desktop card identifier and target interface identifier that match the voice command from preset desktop card identifiers and interface identifiers; the desktop card identifier corresponds to the desktop card of the application, and the interface identifier corresponds to the service interface of the application that is called by the desktop card; the service interface is configured to: when called, drive the application to generate control commands for the controlled device and send the control commands to the controlled device; The control module is used to activate the corresponding target desktop card according to the target desktop card identifier, and control the target desktop card to call the target service interface corresponding to the target interface identifier, so as to control the controlled device to perform the function corresponding to the voice command.

[0013] Optionally, the determining module includes: The keyword extraction submodule is used to extract keywords from the voice commands; The retrieval submodule is used to search in a preset speech analysis lexicon based on the keywords to determine the target desktop card identifier and target interface identifier corresponding to the keywords; the preset speech analysis lexicon stores the mapping relationship between keywords and desktop card identifiers and interface identifiers.

[0014] Optionally, the device further includes: The acquisition module is used to acquire controlled device information and business function description information corresponding to the desktop card in the application; the business function description information includes an interface identifier corresponding to the desktop card and a function description text corresponding to the interface identifier. The keyword generation module is used to generate keywords based on the controlled device information and the functional description text; The mapping relationship establishment module is used to establish a mapping relationship between the keyword, the desktop card identifier and the interface identifier, and store the mapping relationship in the preset speech analysis lexicon.

[0015] Optionally, the controlled device information includes at least one of the following: the controlled device name, the room name where the controlled device is located, and the household identifier to which the controlled device belongs; the keyword generation module includes: The attribute keyword extraction submodule is used to extract at least one of the following from the controlled device information: device type words, location limiting words, and family affiliation words as attribute keywords. The functional keyword extraction submodule is used to extract control behavior words and control target words from the functional description text as functional keywords. The keyword generation submodule is used to obtain the keywords based on the attribute keywords and the functional keywords.

[0016] Optionally, the control module includes: The status detection submodule is used to detect the current lifecycle status of the target desktop card; The activation submodule is used to trigger the operating system to start or wake up the target desktop card to an active state if the target desktop card is in an unloaded or dormant state.

[0017] Optionally, the control module includes: The function parameter value determination submodule is used to parse the function parameter value corresponding to the target service interface from the voice command; The function parameter value sending submodule is used to send the function parameter value to the target desktop card as an input parameter of the target business interface.

[0018] Optionally, the device further includes: The resource release module is used to generate a resource release command if the target desktop card does not receive a user interaction command within a preset time after the control command is sent, so as to trigger the operating system to put the target desktop card into a hibernation state.

[0019] Thirdly, embodiments of the present invention provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0020] Fourthly, embodiments of the present invention provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0021] This invention discloses a device control method based on voice interaction. It requires acquiring the user's voice commands, then determining the target desktop card identifier and target interface identifier matching the voice command from a preset set of desktop card identifiers and interface identifiers. The desktop card identifier corresponds to the application's desktop card, and the interface identifier corresponds to the application's business interface for the desktop card to call. The business interface is configured to: when called, drive the application to generate control commands for the controlled device and send the control commands to the controlled device; finally, based on the target desktop card identifier, activate the corresponding target desktop card and control the target desktop card to call the target business interface corresponding to the target interface identifier, thereby controlling the controlled device to perform the function corresponding to the voice command. By establishing a dynamic mapping mechanism between voice commands and application desktop cards and business interfaces, the parsing and execution tasks of device control are pushed down to the application with native driving capabilities. This effectively overcomes the adaptation defects caused by the operating system's inability to pre-configure a large number of private protocols, allowing users to directly control various smart home devices through the system's voice assistant without unlocking the screen or launching the application, thus improving the convenience of smart home control. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating the steps of a device control method based on voice interaction according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of a voice-interactive device control apparatus according to an embodiment of the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] With the rapid development of IoT technology and the popularization of smart homes, users can remotely control various devices in their homes, such as lights, air conditioners, curtains, and robot vacuums, through smartphone applications. However, they also hope to further control home appliances directly with verbal commands through the voice assistant built into their mobile phone system, without having to unlock the screen or open a specific application, thus achieving a natural "speak and control" interaction.

[0025] However, due to the limitations of mobile operating system manufacturers in fully adapting to the heterogeneous underlying instructions of a vast number of brands and models of devices, the system lacks the ability to directly parse atomic operations of specific devices. As a result, voice assistants cannot directly call device functions like native applications, making the function of controlling home appliances with mobile phone voice not very practical in actual applications and failing to meet users' real needs for convenient control of smart homes.

[0026] Current mobile phone system voice assistants lack the ability to perceive the private protocols of third-party devices, making it impossible to directly parse and issue device-level control commands. They can only rely on user-preset "scenario-based" solutions for indirect control. In this mode, users need to disassemble device functions one by one and manually configure the binding relationship between trigger conditions and execution actions. This not only completely transfers the complexity of adapting to heterogeneous devices to users, resulting in a time-consuming and lengthy configuration process with extremely high operational barriers, but also produces rigid and fixed scenario commands that cannot flexibly adapt to sudden needs or subtle adjustments. Ultimately, the practicality of voice control, a core function that should be convenient, is greatly reduced due to the cumbersome initial setup, severely limiting its popularization and application value in daily life.

[0027] Reference Figure 1 The diagram illustrates a flowchart of a device control method based on voice interaction according to an embodiment of the present invention, which may specifically include the following steps: Step 101: Obtain the user's voice commands.

[0028] Users' voice commands can be obtained through the intelligent voice assistant service built into the operating system of intelligent terminal devices. Intelligent terminal devices include, but are not limited to, electronic devices with voice acquisition and processing capabilities such as smartphones, tablets, smart wearable devices or smart home control screens. The intelligent voice assistant service refers to the intelligent voice interaction program integrated into the operating system that does not require users to install it separately.

[0029] Step 102: Determine the target desktop card identifier and target interface identifier that match the voice command from the preset desktop card identifier and interface identifier; the desktop card identifier corresponds to the desktop card of the application, and the interface identifier corresponds to the service interface of the application that is called by the desktop card; the service interface is configured to: when called, drive the application to generate control commands for the controlled device and send the control commands to the controlled device.

[0030] After acquiring the user's voice command, the operating system's intelligent voice assistant can determine the target desktop card identifier and target interface identifier that match the voice command from preset desktop card identifiers and interface identifiers based on the semantics of the voice command.

[0031] For example, a smart terminal device has a smart home application. This smart home application has desktop cards for the living room air conditioner, bedroom lights, etc. The living room air conditioner desktop card has corresponding business interfaces such as switch switching, temperature adjustment, and status query. The business interfaces can be functions in the code. When the user issues a voice command "set the living room air conditioner to 26 degrees", the smart voice assistant performs semantic search and matching from the preset desktop card identifiers and interface identifiers. The living room air conditioner desktop card is the card for managing the living room air conditioner. Therefore, the desktop card identifier of the living room air conditioner desktop card is determined to be the target desktop card identifier corresponding to the voice command. The temperature adjustment business interface corresponding to the living room air conditioner desktop card is the business interface for adjusting the temperature of the living room air conditioner. Therefore, the interface identifier of the temperature adjustment business interface corresponding to the living room air conditioner desktop card is determined to be the target interface identifier corresponding to the voice command.

[0032] Desktop cards, or desktop widgets, serve as the front-end interactive interface, primarily responsible for the intuitive display of device status and the immediate response to user operation commands. They handle the basic functions of interface rendering and user intent capture, but do not contain the core business logic for device control, thus ensuring the smoothness and stability of card operation. The interface acts as a communication bridge connecting the card and the application. Essentially, it encapsulates and integrates the application's original device control logic, exposing only standardized call entry points and completely hiding complex device interaction details, achieving effective decoupling between business logic and the interactive interface. In actual operation, the desktop card transmits the user's voice operation request to the application by calling the encapsulated business interface. After parsing the request, the application directly calls the encapsulated control logic within the interface, automatically generating control commands adapted to the controlled device and distributing them.

[0033] Step 103: Activate the corresponding target desktop card according to the target desktop card identifier, and control the target desktop card to call the target service interface corresponding to the target interface identifier, so as to control the controlled device to perform the function corresponding to the voice command.

[0034] After identifying the target desktop card identifier and target interface identifier matching the voice command through semantic analysis, the intelligent voice assistant locates and activates the corresponding desktop card component in the desktop environment based on the target desktop card identifier, putting it into an interactive highlighted state or directly waking up its running process in the background. Subsequently, it controls the target desktop card to call the business interface corresponding to the target interface identifier. The business interface drives the application to generate a control protocol packet for the specific controlled device and sends the command to the device through the network channel. After receiving and parsing the command, the controlled device executes the function operation corresponding to the voice command. Ultimately, this achieves the technical effect of users directly and instantly controlling intelligent devices through voice, without requiring users to manually enter the application or configure complex scenarios, significantly improving the convenience and response efficiency of intelligent interaction.

[0035] This invention discloses a device control method based on voice interaction. It requires acquiring the user's voice commands, then determining the target desktop card identifier and target interface identifier matching the voice command from a preset set of desktop card identifiers and interface identifiers. The desktop card identifier corresponds to the application's desktop card, and the interface identifier corresponds to the application's business interface for the desktop card to call. The business interface is configured to: when called, drive the application to generate control commands for the controlled device and send the control commands to the controlled device; finally, based on the target desktop card identifier, activate the corresponding target desktop card and control the target desktop card to call the target business interface corresponding to the target interface identifier, thereby controlling the controlled device to perform the function corresponding to the voice command. By establishing a dynamic mapping mechanism between voice commands and application desktop cards and business interfaces, the parsing and execution tasks of device control are pushed down to the application with native driving capabilities. This effectively overcomes the adaptation defects caused by the operating system's inability to pre-configure a large number of private protocols, allowing users to directly control various smart home devices through the system's voice assistant without unlocking the screen or launching the application, thus improving the convenience of smart home control.

[0036] In this embodiment of the invention, step 102, determining the target desktop card identifier and target interface identifier that match the voice command from preset desktop card identifiers and interface identifiers, may specifically include the following sub-steps: Sub-step S11: Extract keywords from the voice command.

[0037] Sub-step S12: Based on the keyword, search in a preset speech analysis lexicon to determine the target desktop card identifier and target interface identifier corresponding to the keyword; the preset speech analysis lexicon stores the mapping relationship between the keyword and the desktop card identifier and the interface identifier.

[0038] When a user issues a voice command, the intelligent voice assistant can first convert the voice signal into text information, and then use natural language processing technology to perform preprocessing on the text, such as word segmentation and stop word removal, to extract core keywords with clear semantic meaning. For example, if the user issues a voice command such as "Turn on the air conditioner in the living room", the extracted keyword set will be {"living room", "air conditioner", "turn on"}.

[0039] The pre-built speech analysis lexicon can store the mapping relationship between keywords and desktop card identifiers and interface identifiers in the form of key-value pairs. For example, the keywords corresponding to the living room air conditioner desktop card identifier in the pre-built speech analysis lexicon may include "living room" and "air conditioner", and the keywords corresponding to the on / off switching business interface identifier of the living room air conditioner desktop card may include "on" and "off".

[0040] After extracting keywords, a multi-keyword joint search can be performed in a preset voice analysis lexicon to determine the target desktop card identifier and target interface identifier most relevant to the semantics of the current command. Continuing the previous example, based on the extracted keywords "living room" and "air conditioner," the matching target desktop card identifier can be determined to be the living room air conditioner desktop card identifier. Based on the extracted keyword "open," the matching target interface identifier can be determined to be the on / off switching service interface of the living room air conditioner desktop card. Through the mapping relationship between keywords, desktop card identifiers, and interface identifiers, the target desktop card identifier and target interface identifier that best match the semantics of the current voice command can be quickly determined, providing an accurate basis for subsequent desktop card activation and interface calls, thereby realizing the automated conversion of voice commands into specific device control functions.

[0041] In this embodiment of the invention, the method further includes: Obtain the controlled device information and business function description information corresponding to the desktop card in the application; the business function description information includes the interface identifier corresponding to the desktop card and the function description text corresponding to the interface identifier. Based on the controlled device information and the function description text, generate keywords; Establish a mapping relationship between the keywords, the desktop card identifiers, and the interface identifiers, and store the mapping relationship in the preset speech analysis lexicon.

[0042] During the desktop card creation phase, developers can develop the core business logic of the card based on the actual functional requirements of the controlled device. Taking an air conditioner as an example, the business logic needs to cover specific control functions such as power on / off, temperature adjustment, and operating mode switching. To achieve reuse and decoupling of the core capabilities of the card and the application, developers need to extract and encapsulate the complex business code originally used in the application for controlling the device (such as network protocol encapsulation, command generation, and status verification) into independent business interface functions. These interfaces expose standardized call entry points, so that when the card needs to perform control operations, it does not need to repeatedly write the underlying logic; it only needs to call the corresponding interface functions to drive the controlled device to perform the corresponding functions.

[0043] In the process of creating a card, in order for the intelligent voice assistant to accurately identify and locate the desktop card, the metadata information of the desktop card needs to be defined and exposed in the desktop card's configuration file. The metadata information includes the controlled device information corresponding to the desktop card. The intelligent voice assistant of the operating system can obtain the exposed metadata information of the desktop card and build a semantic mapping relationship between the desktop card and the controlled device, thereby being able to understand the specific desktop card object pointed to by the voice command (such as "turn on the living room air conditioner").

[0044] Developers can encapsulate the core business logic of desktop cards into independent function interfaces and expose them to the operating system for invocation. At the same time, they can associate each function interface with a preset functional description text (semantic term). This semantic association mechanism provides a clear instruction mapping relationship. The operating system's intelligent voice assistant can accurately match the corresponding function interface based on the user's natural language instructions, realizing the automated conversion from voice commands to device control operations.

[0045] Once the card is created and configured on the application, users can add it to their desktop using the native interaction methods provided by their operating system. While the addition process varies across different operating systems, the essence is the same: calling system interfaces to instantiate cards already created by the developer. For example, a user can long-press an empty area of ​​the desktop to trigger desktop editing mode, then click the "Edit" button in the upper left corner of the screen. In the pop-up menu, select "Add Widget," find the desktop card for the user-controlled device (e.g., "Living Room Air Conditioner Desktop Card") in the widget list, and click the "Add" button to place the card in the specified location on the desktop. Alternatively, a user can long-press the icon of an installed device control application (e.g., a smart home application), click the "Card" option in the pop-up shortcut menu, and the system will display all card styles supported by the application. After selecting a desktop card (e.g., "Living Room Air Conditioner Desktop Card"), the user can simply drag and drop it to an empty area of ​​the desktop to complete the addition. Regardless of the interaction method, once a card is added to the desktop, the system can scan and read the registered desktop card configurations in the application to obtain the desktop card's metadata and business function description information. The metadata information includes the controlled device information corresponding to the desktop card, and the business function description information includes the interface identifier corresponding to the desktop card and the function description text corresponding to the interface identifier.

[0046] Natural language processing is performed on the acquired information about the controlled devices and their functional descriptions to generate multi-dimensional search keywords. This process includes word segmentation and synonym expansion of the original text, thereby constructing a keyword set that encompasses users' colloquial expression habits. For example, for the functional description of "air conditioning cooling," the system may generate multiple related keywords such as "cool," "cooling down," and "turn on the air conditioning."

[0047] The generated keywords, desktop card identifiers, and interface identifiers are associated and bound to form a mapping relationship, which is then stored in a pre-defined voice analysis lexicon. This allows the system to quickly locate the correct desktop card and corresponding interface based on the user's voice commands. By constructing a semantically mapped voice analysis lexicon, the system can understand diverse spoken expressions. Even if the command contains non-standard vocabulary or ambiguous semantics, it can accurately match the corresponding business interface through the association relationship in the lexicon. This effectively solves the problem of rigid semantic matching in traditional command control. Furthermore, the unified voice analysis lexicon manages desktop cards from different applications, breaking down barriers between applications. Users do not need to remember which specific application controls the device; they only need to describe their needs in natural language, and the system can retrieve and schedule the corresponding desktop card service in the background through the lexicon, greatly improving the user experience in smart home and other scenarios.

[0048] In this embodiment of the invention, the controlled device information includes at least one of the following: the controlled device name, the room name where the controlled device is located, and the household identifier to which the controlled device belongs; based on the controlled device information and functional description text, keywords are generated, including: Extract at least one of the following as attribute keywords from the controlled device information: device type words, location limiting words, and household affiliation words; Extract control behavior words and control target words from the functional description text as functional keywords; Keywords are derived based on attribute keywords and functional keywords.

[0049] From the information of the controlled devices, the device name is parsed into "device type words" that represent the device category (such as "air conditioner" and "robot vacuum cleaner"), the room name is parsed into "location qualifiers" that represent the physical space (such as "living room" and "master bedroom"), and the family identifier is parsed into "family affiliation words" that represent the user's affiliation. These keywords together constitute the set of attribute keywords.

[0050] From the functional description text, natural language processing technology is used to identify and extract "control behavior words" (such as "turn on", "adjust", "turn off") that represent the operation actions, and "control target words" (such as "temperature", "mode", "wind speed") that represent the specific object of the operation, thus forming a set of functional keywords.

[0051] Attribute keywords and functional keywords can be used together as keywords in a preset speech analysis lexicon. Alternatively, attribute keywords and functional keywords can be logically linked and merged. For example, the location qualifier "living room," the device type "air conditioner," the control action word "turn on," and the control target word "temperature" can be combined to generate compound keywords such as "turn on the living room air conditioner" and "turn on the living room air conditioner." These compound keywords not only include "what to do" (function) but also specify "to whom to do it" (object) and "where to do it" (location), thereby constructing a high-coverage semantic index library to ensure that the system can accurately respond to users' natural language commands. Through fine-grained semantic decomposition, the system's semantic understanding ability and command matching efficiency are significantly improved. By extracting control action words and control target words from the functional description text, the system is no longer limited to fixed command sentences. Users can use various expressions such as "lower the living room air conditioner temperature" or "cool the living room air conditioner," and the system can identify the same control intent through the matching logic of attribute keywords and functional keywords, greatly optimizing the user's interactive experience.

[0052] In this embodiment of the invention, step 103, activating the corresponding target desktop card according to the target desktop card identifier, may specifically include the following sub-steps: Sub-step S21: Detect the current lifecycle status of the target desktop card.

[0053] Sub-step S22: If the target desktop card is in an unloaded or hibernating state, the operating system is triggered to start or wake up the target desktop card to an active state.

[0054] After parsing the target desktop card identifier based on the voice command, the system does not blindly call the card function directly. Instead, it first checks the current lifecycle state of the target desktop card. In mobile operating systems, to save memory and power, desktop cards that are not currently displayed or have not been used for a long time are usually placed in an "unloaded" or "sleep / suspended" state. Therefore, it is necessary to first determine whether the target card is in these inactive states. If it is detected that the target desktop card has not been loaded into memory or is in a sleep state suspended by the system, the operating system's card management service needs to be triggered to perform a start or wake-up operation. This process includes allocating the necessary system resources to the desktop card, loading its runtime environment, and migrating its state to an active state. Only when the desktop card successfully enters an active state can the subsequent interface call steps be executed to control the desktop card to call the target business interface, thereby driving the controlled device to perform the corresponding function.

[0055] By introducing a lifecycle detection and automatic wake-up mechanism, the problem of delayed or unresponsive desktop cards when running in the background is effectively solved. By forcibly detecting and waking up the desktop card before executing control logic, it is ensured that the desktop card is in a fully ready state when calling business interfaces, eliminating the risk of command loss or execution failure caused by the desktop card being in a dormant state, thereby ensuring smoothness from voice command issuance to device action execution.

[0056] In this embodiment of the invention, step 103, controlling the target desktop card to call the target service interface corresponding to the target interface identifier, may specifically include the following sub-steps: Sub-step S31: Parse the function parameter values ​​corresponding to the target business interface from the voice command.

[0057] Sub-step S32: Send the function parameter value to the target desktop card as the input parameter for the target business interface.

[0058] Natural language processing technology can be used to identify and extract specific numerical values ​​or state descriptions carried in voice commands, and convert them into standardized functional parameter values. For example, if the voice command is "set the living room air conditioner temperature to 26 degrees", while determining that the target interface is "temperature adjustment interface", the value "26" will be extracted from the command as a functional parameter value.

[0059] Subsequently, the extracted functional parameter values ​​are encapsulated in a call request and sent directly to the activated target desktop card. Upon receiving the parameter value, the target desktop card passes it as an input parameter to the target business interface. The internal logic of the business interface then generates specific device control instructions based on the parameter value (e.g., "26") and issues them to the controlled device, thus completing the conversion from fuzzy natural language instructions to precise device parameter configuration. Decoupling parameter extraction from interface calls allows the same business interface (e.g., "adjust temperature") to respond to an infinite number of parameter combinations (e.g., 20 degrees, 25 degrees, 30 degrees, etc.) without needing to pre-define independent interface identifiers for each possible value; simply extracting and transmitting the parameters reduces the number of predefined interfaces, lowers the complexity of dictionary maintenance, and enhances the scalability of the card's business logic.

[0060] In this embodiment of the invention, the method further includes: If the target desktop card does not receive a user interaction command within a preset time after the control command is sent, a resource release command is generated to trigger the operating system to put the target desktop card into a hibernation state.

[0061] Once the target desktop card calls the business interface and sends control commands, the card has completed its current control mission. To prevent the card from continuing to occupy system resources after completing its task, a preset timer (e.g., set to 5 or 10 minutes) can be started. During this period, the target desktop card is monitored to see if it receives any new user interaction commands (such as clicks, swipes, or voice commands). If the target desktop card remains silent within the preset time, i.e., no user interaction is detected, it is determined that the card is no longer frequently used by the user. At this time, a resource release command can be generated and sent to the operating system's card management service. In response to the resource release command, the operating system performs a suspend or hibernation operation on the target desktop card. Although the card remains in the desktop layout, its background process is terminated, thus entering a low-power standby state, waiting to be woken up again.

[0062] Desktop cards consume a certain amount of memory and processor resources when they are active. By automatically triggering resource release after the task is completed, the system can promptly reclaim the resources occupied by idle cards, preventing memory leaks or system lag caused by running multiple desktop cards for a long time. This ensures efficient management of system resources while maintaining a good user experience.

[0063] When an application creates a desktop card, it exposes its business requirements (such as "control switch" or "adjust temperature") and the corresponding controlled device attributes (such as "living room" or "air conditioner") to the mobile operating system through standardized configuration files or metadata declarations. The intelligent voice assistant on the operating system side does not need to parse or understand the complex business logic code, network communication protocols, or underlying data architecture of the application. It reads the semantic description information (i.e., "business requirements") exposed by the desktop card. Based on this description information, it builds an intent mapping index locally and directly associates the user's natural language commands with specific desktop cards and their bound business interfaces. Once the user completes the creation and addition of desktop cards, the application's business capabilities are automatically "registered" into the system's voice control ecosystem. Through the desktop card as an intermediary layer, the system achieves unified scheduling and integration of business capabilities in different application scenarios, without the need for separate, in-depth customization development for each application.

[0064] Reference Figure 2 This illustration shows a device control apparatus based on voice interaction according to an embodiment of the present invention, which may specifically include the following structure: The voice command acquisition module 201 is used to acquire the user's voice commands.

[0065] The determining module 202 is used to determine, from preset desktop card identifiers and interface identifiers, a target desktop card identifier and a target interface identifier that match the voice command; the desktop card identifier corresponds to the desktop card of the application, and the interface identifier corresponds to the service interface of the application that is called by the desktop card; the service interface is configured to: when called, drive the application to generate control commands for the controlled device and send the control commands to the controlled device.

[0066] The control module 203 is used to activate the corresponding target desktop card according to the target desktop card identifier, and control the target desktop card to call the target service interface corresponding to the target interface identifier, so as to control the controlled device to perform the function corresponding to the voice command.

[0067] This invention discloses a device control device based on voice interaction. It acquires user voice commands and then determines a target desktop card identifier and a target interface identifier matching the voice command from a preset set of desktop card identifiers and interface identifiers. The desktop card identifier corresponds to the desktop card of an application, and the interface identifier corresponds to a business interface of the application that the desktop card can call. The business interface is configured to: when called, drive the application to generate control commands for the controlled device and send the control commands to the controlled device; finally, based on the target desktop card identifier, activate the corresponding target desktop card and control the target desktop card to call the target business interface corresponding to the target interface identifier, thereby controlling the controlled device to perform the function corresponding to the voice command. By establishing a dynamic mapping mechanism between voice commands and application desktop cards and business interfaces, the parsing and execution tasks of device control are pushed down to the application with native driving capabilities. This effectively overcomes the adaptation defects caused by the operating system's inability to pre-configure a large number of private protocols, allowing users to directly control various smart home devices through the system's voice assistant without unlocking the screen or launching the application, thus improving the convenience of smart home control.

[0068] In this embodiment of the invention, the determining module includes: The keyword extraction submodule is used to extract keywords from the voice commands; The retrieval submodule is used to search in a preset speech analysis lexicon based on the keywords to determine the target desktop card identifier and target interface identifier corresponding to the keywords; the preset speech analysis lexicon stores the mapping relationship between keywords and desktop card identifiers and interface identifiers.

[0069] In this embodiment of the invention, the device further includes: The acquisition module is used to acquire controlled device information and business function description information corresponding to the desktop card in the application; the business function description information includes an interface identifier corresponding to the desktop card and a function description text corresponding to the interface identifier. The keyword generation module is used to generate keywords based on the controlled device information and the functional description text; The mapping relationship establishment module is used to establish a mapping relationship between the keyword, the desktop card identifier and the interface identifier, and store the mapping relationship in the preset speech analysis lexicon.

[0070] In this embodiment of the invention, the controlled device information includes at least one of the controlled device name, the room name where the controlled device is located, and the household identifier to which the controlled device belongs; the keyword generation module includes: The attribute keyword extraction submodule is used to extract at least one of the following from the controlled device information: device type words, location limiting words, and family affiliation words as attribute keywords. The functional keyword extraction submodule is used to extract control behavior words and control target words from the functional description text as functional keywords. The keyword generation submodule is used to obtain the keywords based on the attribute keywords and the functional keywords.

[0071] In this embodiment of the invention, the control module includes: The status detection submodule is used to detect the current lifecycle status of the target desktop card; The activation submodule is used to trigger the operating system to start or wake up the target desktop card to an active state if the target desktop card is in an unloaded or dormant state.

[0072] In this embodiment of the invention, the control module includes: The function parameter value determination submodule is used to parse the function parameter value corresponding to the target service interface from the voice command; The function parameter value sending submodule is used to send the function parameter value to the target desktop card as an input parameter of the target business interface.

[0073] In this embodiment of the invention, the device further includes: The resource release module is used to generate a resource release command if the target desktop card does not receive a user interaction command within a preset time after the control command is sent, so as to trigger the operating system to put the target desktop card into a hibernation state.

[0074] The terms "first," "second," etc., used in the specification and claims of this invention are used to distinguish similar objects and are not used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0075] This invention also provides an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the various processes of the above-described embodiments of the device control method based on voice interaction and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0076] It should be noted that the electronic devices in the embodiments of the present invention include the mobile electronic devices and non-mobile electronic devices described above.

[0077] This invention also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described embodiments of the voice-interaction-based device control method and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0078] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0079] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of the present invention is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0080] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0081] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

Claims

1. A device control method based on voice interaction, characterized in that, include: Obtain the user's voice commands; From the preset desktop card identifiers and interface identifiers, determine the target desktop card identifier and target interface identifier that match the voice command; the desktop card identifier corresponds to the desktop card of the application, and the interface identifier corresponds to the business interface of the application that is called by the desktop card; the business interface is configured to: when called, drive the application to generate control commands for the controlled device and send the control commands to the controlled device; Based on the target desktop card identifier, the corresponding target desktop card is activated, and the target desktop card is controlled to call the target service interface corresponding to the target interface identifier, so as to control the controlled device to perform the function corresponding to the voice command.

2. The equipment control method according to claim 1, characterized in that, The step of determining the target desktop card identifier and target interface identifier that match the voice command from the preset desktop card identifiers and interface identifiers includes: Extract keywords from the voice commands; Based on the keywords, a search is conducted in a preset speech analysis lexicon to determine the target desktop card identifier and target interface identifier corresponding to the keywords; the preset speech analysis lexicon stores the mapping relationship between keywords and desktop card identifiers and interface identifiers.

3. The equipment control method according to claim 1, characterized in that, The method further includes: Obtain the controlled device information and business function description information corresponding to the desktop card in the application; the business function description information includes the interface identifier corresponding to the desktop card and the function description text corresponding to the interface identifier. Based on the controlled device information and the function description text, generate keywords; Establish a mapping relationship between the keywords, the desktop card identifiers, and the interface identifiers, and store the mapping relationship in the preset speech analysis lexicon.

4. The equipment control method according to claim 3, characterized in that, The controlled device information includes at least one of the following: controlled device name, room name where the controlled device is located, and household identifier to which the controlled device belongs; the generation of keywords based on the controlled device information and the functional description text includes: Extract at least one of the following as attribute keywords from the controlled device information: device type words, location limiting words, and home affiliation words; Extract control behavior words and control target words from the functional description text as functional keywords; The keywords are obtained based on the attribute keywords and the functional keywords.

5. The equipment control method according to claim 1, characterized in that, The step of activating the corresponding target desktop card based on the target desktop card identifier includes: Detect the current lifecycle status of the target desktop card; If the target desktop card is in an unloaded or dormant state, the operating system will be triggered to start or wake up the target desktop card to an active state.

6. The equipment control method according to claim 1, characterized in that, The step of controlling the target desktop card to call the target service interface corresponding to the target interface identifier includes: Parse the function parameter values ​​corresponding to the target business interface from the voice command; The function parameter values ​​are sent to the target desktop card as input parameters for the target business interface.

7. The equipment control method according to claim 1, characterized in that, The method further includes: If the target desktop card does not receive a user interaction command within a preset time after the control command is sent, a resource release command is generated to trigger the operating system to put the target desktop card into a hibernation state.

8. A device control apparatus based on voice interaction, characterized in that, The device includes: The voice command acquisition module is used to acquire the user's voice commands; The determining module is used to determine the target desktop card identifier and target interface identifier that match the voice command from preset desktop card identifiers and interface identifiers; the desktop card identifier corresponds to the desktop card of the application, and the interface identifier corresponds to the service interface of the application that is called by the desktop card; the service interface is configured to: when called, drive the application to generate control commands for the controlled device and send the control commands to the controlled device; The control module is used to activate the corresponding target desktop card according to the target desktop card identifier, and control the target desktop card to call the target service interface corresponding to the target interface identifier, so as to control the controlled device to perform the function corresponding to the voice command.

9. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the voice-interactive device control method as described in any one of claims 1-7.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the voice-interactive device control method as described in any one of claims 1-7.

Images