Device control method, device control apparatus, device, storage medium, and program product

Abstract

The application relates to a device control method and device, equipment, a storage medium and a program product. The method comprises the following steps: obtaining voice control text information, target running parameters of each candidate Internet of Things device, and target environment state information; determining a dynamic physical capability boundary based on the target running parameters and the target environment state information; based on the voice control text information and the dynamic physical capability boundary, at least one of a candidate action set of a large language model, a tool definition, a generation path, a candidate word element probability distribution or a generation action space is constrained, so that the large language model generates control information conforming to the dynamic physical capability boundary and containing a target control instruction; if the target control instruction and a target Internet of Things device contained in the target control instruction are parsed from the control information, the target control instruction is subjected to a security check; and if the check is passed, the target Internet of Things device executes the target control instruction. The method can improve the accuracy and safety of device control.

Description

Technical Field

[0001] This application relates to the fields of Internet of Things and artificial intelligence technologies, and in particular to a device control method, apparatus, device, storage medium, and program product. Background Technology

[0002] With the rapid popularization of Internet of Things (IoT) technology, intelligent voice assistants have become a core entry point for human-computer interaction. However, in practical application and implementation, traditional control solutions mainly face the following technical bottlenecks:

[0003] (1) Limitations of traditional rule-based and slot-filling semantic understanding technologies: Currently, mainstream commercial voice assistants generally adopt a pipeline architecture of automatic speech recognition + natural language understanding, which relies on slot filling or template matching technology. The technical shortcomings are: such systems are essentially finite state machines based on predefined rules. User commands must strictly hit the preset intent word slots, for example, they must be expressed as [action] + [device name]. The system lacks the ability to reason about unstructured, ambiguous semantics (such as "feeling a little bored") or implicit intents, resulting in a rigid interactive experience, poor generalization ability, and poor accuracy in device control.

[0004] (2) The "probability-determinism" contradiction (illusion risk) of general generative models in industrial control: In order to solve the problem of insufficient understanding, it is currently possible to introduce large language models (LLM). However, general LLM is essentially an autoregressive generative model based on probabilistic prediction, while IoT hardware control requires extremely high determinism and security. There is a natural inherent technical contradiction between the two, which is prone to producing factual illusions. That is, when the large model is not subject to physical constraints, it is very easy to generate instructions that are out of touch with reality, which poses unacceptable security risks in the fields of security and home control. Summary of the Invention

[0005] Therefore, it is necessary to provide a device control method, apparatus, equipment, storage medium, and program product that can improve the accuracy and safety of device control in response to the above-mentioned technical problems.

[0006] In a first aspect, this application provides a device control method, the method comprising:

[0007] Acquire voice control text information for the target network, and acquire target operating parameters and environmental status information of each candidate IoT device under the target network;

[0008] Based on the target operating parameters and the target environment state information, the dynamic physical capability boundary at the current moment is determined; the dynamic physical capability boundary is used to characterize at least one of the following: actions that the controlled physical system is allowed to perform, actions that are prohibited from performing, parameter value ranges, state mutual exclusion relationships, and safety constraints in the current state;

[0009] Based on the voice control text information and the dynamic physical capability boundary, at least one of the following is constrained in the large language model: candidate action set, tool definition, generation path, candidate lexical probability distribution, or generation action space, so that the large language model generates control information that conforms to the dynamic physical capability boundary and contains target control instructions; the target control instructions are one of at least one reference control instructions set for the candidate IoT device contained in the generation action space.

[0010] If the target control command and the target IoT device contained in the target control command are parsed from the control information, then the target control command is subjected to security verification.

[0011] If the verification passes, the target control command is sent to the target IoT device so that the target IoT device executes the target control command.

[0012] In one embodiment, obtaining the voice control text information for the target network includes:

[0013] In low-power mode, a keyword detection algorithm is run through a low-power processing unit to perform specific wake-up word matching;

[0014] If the specific wake-up word is matched, the high-performance processing unit in sleep mode is woken up by a hardware interrupt signal;

[0015] The high-performance processing unit, which has been activated, acquires the control voice signal for the target network, performs speech recognition on the control voice signal, and obtains the voice control text information for the target network.

[0016] In one embodiment, obtaining the target operating parameters and environmental status information of each candidate IoT device under the target network, as collected by the IoT devices, includes:

[0017] During the process of waking up the high-performance processing unit, the hardware communication module polls each candidate IoT device and environmental status IoT device under the target network to obtain the target operating parameters of each candidate IoT device and the target environmental status information collected by the environmental status IoT device.

[0018] The high-performance processing unit, once activated, receives the target operating parameters and target environment status information sent by the hardware communication module.

[0019] In one embodiment, the process of waking up the high-performance processing unit and using the hardware communication module to poll each candidate IoT device and environmental status IoT device under the target network to obtain the target operating parameters of each candidate IoT device and the target environmental status information collected by the environmental status IoT device includes:

[0020] During the process of waking up the high-performance processing unit, the hardware communication module polls each candidate IoT device and environmental status IoT device under the target network to obtain the original operating parameters of each candidate IoT device under the target network and the original environmental status information collected by the environmental status IoT device; the candidate IoT devices and the environmental status IoT device include devices with heterogeneous protocols.

[0021] By using a pre-built protocol abstraction layer, the original operating parameters and the original environmental state information are cleaned and standardized and mapped to obtain the target operating parameters and the target environmental state information respectively.

[0022] In one embodiment, the security verification includes logical conflict verification and parameter compliance verification; passing the verification means that both the logical conflict verification and the parameter compliance verification pass simultaneously.

[0023] The security verification of the target control command includes:

[0024] Obtain the current device status of the target IoT device;

[0025] If the current device state is inconsistent with the device state indicated by the target control command, then the logical conflict check is deemed to have passed.

[0026] Determine the range of physical capabilities supported by the target IoT device;

[0027] If the target control command contains numerical parameters and the numerical parameters fall within the range of the physical capabilities, then the parameter compliance verification is deemed to have passed.

[0028] In one embodiment, after the step of sending the target control command to the target IoT device if the verification passes, so that the target IoT device executes the target control command; the method further includes:

[0029] Obtain the current device status of the target IoT device;

[0030] If the current device status is consistent with the device status indicated by the target control command, then a success message is returned.

[0031] If there is a discrepancy, an execution failure message will be returned.

[0032] In one embodiment, the method further includes:

[0033] If parsing the control information fails, or if the target control command does not include the target IoT device, the control information is input to the large language model to generate an error message based on the control information.

[0034] In one embodiment, the dynamic physical capability boundary is calculated using at least one of a rule engine, a security policy model, a device capability model, a user preference model, or a scenario constraint model.

[0035] The dynamic physical capability boundary includes at least one of the following: an action allowable list, an action prohibition list, parameter upper and lower limits, device mutual exclusion relationship, action priority, action triggering condition, or safety circuit breaker condition.

[0036] Secondly, this application provides a device control apparatus, the apparatus comprising:

[0037] The acquisition module is used to acquire voice control text information for the target network, and to acquire the target operating parameters and environmental status information of each candidate IoT device under the target network, as well as the target environmental status information collected by the IoT devices.

[0038] The determination module is used to determine the dynamic physical capability boundary at the current moment based on the target operating parameters and the target environment state information; the dynamic physical capability boundary is used to characterize at least one of the following: actions that the controlled physical system is allowed to perform, actions that are prohibited from performing, parameter value ranges, state mutual exclusion relationships, and safety constraints in the current state;

[0039] The generation module is used to constrain at least one of the candidate action set, tool definition, generation path, candidate lexical probability distribution, or generation action space of the large language model based on the voice control text information and the dynamic physical capability boundary, so that the large language model generates control information that conforms to the dynamic physical capability boundary and contains target control instructions; the target control instructions are one of at least one reference control instructions set for the candidate IoT device contained in the generation action space.

[0040] The verification module is used to perform security verification on the target control command if the target control command and the target IoT device contained in the target control command are parsed from the control information.

[0041] The sending module is configured to send the target control command to the target IoT device if the verification passes, so that the target IoT device executes the target control command.

[0042] Thirdly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the various method embodiments of this application.

[0043] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the various method embodiments of this application.

[0044] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps in the various method embodiments of this application.

[0045] The aforementioned device control method, apparatus, equipment, storage medium, and program product acquire voice control text information for a target network, and acquire target operating parameters and environmental state information collected by the IoT devices for each candidate IoT device under the target network; based on the target operating parameters and target environmental state information, determine the dynamic physical capability boundary at the current moment; the dynamic physical capability boundary is used to characterize at least one of the following in the current state of the controlled physical system: allowed actions, prohibited actions, parameter value range, state mutual exclusion relationship, and security constraints; based on the voice control text information and the dynamic physical capability boundary, constrain at least one of the following in the large language model: candidate action set, tool definition, generation path, candidate lexical probability distribution, or generation action space, so that the large language model generates control information that conforms to the dynamic physical capability boundary and contains target control instructions; the target control instruction is one of at least one reference control instruction set for candidate IoT devices contained in the generation action space; if the target control instruction and the target IoT device contained in the target control instruction are parsed from the control information, the target control instruction is subjected to security verification; if the verification passes, the target control instruction is sent to the target IoT device so that the target IoT device executes the target control instruction. Compared to traditional device control methods, this application effectively addresses the industry pain points of rigid semantic understanding in traditional rule-based systems and the "probability-determinism" contradiction in general large models by introducing dynamic physical capability boundaries. On one hand, it breaks through the dependence on fixed instruction templates, giving the system the ability to generalize and understand ambiguous or implicit intentions; on the other hand, it utilizes real-time operating parameters and environmental states to construct physical constraints, fundamentally suppressing the risk of generating illusory instructions in large models. Simultaneously, combined with a security verification mechanism, it ensures that generated instructions strictly conform to the controllable range of physical devices, improving both the naturalness of interaction and the accuracy and security of device control. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart illustrating a device control method in one embodiment;

[0048] Figure 2 This is a schematic diagram of the prompt word structure and data flow in one embodiment;

[0049] Figure 3 This is a schematic diagram of the architecture of a device hierarchical wake-up system in one embodiment;

[0050] Figure 4 This is a flowchart illustrating the device control method in another embodiment;

[0051] Figure 5 This is a structural block diagram of the device control apparatus in one embodiment;

[0052] Figure 6 This is an internal structural diagram of a computer device in one embodiment;

[0053] Figure 7 This is a diagram of the internal structure of a computer device in another embodiment. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0055] In one embodiment, such as Figure 1 As shown, a device control method is provided. This method can be applied to a computer device, which can be a terminal or a server. That is, the method can be executed independently by the terminal or the server, or it can be implemented through interaction between the terminal and the server. This embodiment uses the application of this method to a computer device as an example for illustration, and includes the following steps:

[0056] Step 102: Obtain voice control text information for the target network, and obtain the target operating parameters and environmental status information of each candidate IoT device under the target network.

[0057] Among them, the target operating parameters are parameters reflecting the working status, health status, and energy consumption of the candidate IoT device itself. The target environmental status information is information used to characterize the surrounding environment of the candidate IoT device.

[0058] In one embodiment, a computer device can acquire control voice signals for a target network via a microphone, and perform speech recognition on the control voice signals to obtain voice control text information for the target network.

[0059] Step 104: Based on the target operating parameters and target environment state information, determine the dynamic physical capability boundary at the current moment; the dynamic physical capability boundary is used to characterize at least one of the following: actions that the controlled physical system is allowed to perform, actions that are prohibited from performing, parameter value range, state mutual exclusion relationship, and safety constraint conditions in the current state.

[0060] Step 106: Based on the voice control text information and dynamic physical capability boundary, constrain at least one of the candidate action set, tool definition, generation path, candidate lexical probability distribution, or generation action space of the large language model, so that the large language model generates control information that conforms to the dynamic physical capability boundary and contains target control instructions; the target control instructions are one of at least one reference control instructions set for candidate IoT devices contained in the generation action space.

[0061] Step 108: If the target control command and the target IoT device contained in the target control command are parsed from the control information, then the target control command is subjected to security verification.

[0062] In one embodiment, security verification includes at least one of logical conflict verification and parameter compliance verification.

[0063] In one embodiment, such as Figure 2 As shown, the computer device can construct prompt words for a large language model based on voice control text information (such as "It's too hot"), target operating parameters, target environmental status information, and instruction set. The large language model generates control information containing target control instructions based on the prompt words and dynamic physical capability boundaries. The control information is parsed, and if the target control instructions and the target IoT device contained in the target control instructions are parsed from the control information, the security of the target control instructions is verified.

[0064] Step 110: If the verification passes, the target control command is sent to the target IoT device so that the target IoT device executes the target control command.

[0065] In one embodiment, security verification includes at least one of logical conflict verification and parameter compliance verification. Verification passing means that at least one of the logical conflict verification and parameter compliance verification passes the verification.

[0066] In the aforementioned device control method, the following steps are taken: First, voice control text information for the target network is acquired, along with target operating parameters and environmental state information collected by the IoT devices for each candidate IoT device within the target network. Second, based on the target operating parameters and environmental state information, the dynamic physical capability boundary at the current moment is determined. The dynamic physical capability boundary characterizes at least one of the following in the current state of the controlled physical system: allowed actions, prohibited actions, parameter value ranges, state mutual exclusion relationships, and security constraints. Third, based on the voice control text information and the dynamic physical capability boundary, at least one of the following constraints is applied to the large language model: candidate action set, tool definition, generation path, candidate lexical probability distribution, or generated action space, so that the large language model generates control information that conforms to the dynamic physical capability boundary and contains target control instructions. The target control instruction is one of at least one reference control instruction set for the candidate IoT device within the generated action space. Fourth, if the target control instruction and the target IoT device contained in the target control instruction are parsed from the control information, the target control instruction undergoes security verification. Fifth, if the verification passes, the target control instruction is sent to the target IoT device so that the target IoT device executes the target control instruction. Compared to traditional device control methods, this application effectively addresses the industry pain points of rigid semantic understanding in traditional rule-based systems and the "probability-determinism" contradiction in general large models by introducing dynamic physical capability boundaries. On one hand, it breaks through the dependence on fixed instruction templates, giving the system the ability to generalize and understand ambiguous or implicit intentions; on the other hand, it utilizes real-time operating parameters and environmental states to construct physical constraints, fundamentally suppressing the risk of generating illusory instructions in large models. Simultaneously, combined with a security verification mechanism, it ensures that generated instructions strictly conform to the controllable range of physical devices, improving both the naturalness of interaction and the accuracy and security of device control.

[0067] In one embodiment, obtaining voice control text information for a target network includes: running a keyword detection algorithm through a low-power processing unit in a low-power mode to match a specific wake-up word; if a specific wake-up word is matched, waking up a high-performance processing unit in a dormant state through a hardware interrupt signal; obtaining a control voice signal for the target network through the awakened high-performance processing unit, performing speech recognition on the control voice signal, and obtaining voice control text information for the target network.

[0068] In one embodiment, the low-power processing unit can be one of a digital signal processor (DSP), a microcontroller unit (MCU), an application-specific integrated circuit (ASIC), and a field-programmable gate array (FPGA). The high-performance processing unit can be one of a system-on-a-chip (SoC), an application processor (AP), and a cloud computing connectivity unit.

[0069] In one embodiment, a specific wake word could be something like "Hello housekeeper".

[0070] In the above embodiments, through the layered wake-up mechanism, only lightweight keyword detection is run in low-power mode, which greatly reduces standby power consumption; only after a valid wake-up word is detected is the high-performance unit accurately woke up through hardware interrupt to perform complex voice processing, avoiding continuous standby power consumption of the high-performance unit, achieving a balance between low power consumption and high response speed, and significantly improving the battery life of the IoT voice control terminal.

[0071] In one embodiment, obtaining the target operating parameters and environmental status information of each candidate IoT device in the target network, and the target environmental status information collected by the IoT devices, includes: during the process of waking up the high-performance processing unit, and exercising the hardware communication module to poll each candidate IoT device and the environmental status IoT device in the target network to obtain the target operating parameters and environmental status information collected by each candidate IoT device; and receiving the target operating parameters and target environmental status information sent by the hardware communication module through the woken-up high-performance processing unit.

[0072] In the above embodiments, by using a parallel-enabled hardware communication module, device parameters and environmental status information are collected in parallel during the intervals between waking up the high-performance processing unit, transforming the serial waiting time into parallel operation. This avoids the idle waiting after the high-performance unit is woken up and significantly shortens the end-to-end latency from wake-up to command execution, thereby improving the response speed and smoothness of voice control.

[0073] In one embodiment, during the process of waking up the high-performance processing unit, the functional hardware communication module polls each candidate IoT device and environmental state IoT device under the target network to obtain the target operating parameters of each candidate IoT device and the target environmental state information collected by the environmental state IoT device. This includes: during the process of waking up the high-performance processing unit, the functional hardware communication module polls each candidate IoT device and environmental state IoT device under the target network to obtain the original operating parameters of each candidate IoT device and the original environmental state information collected by the environmental state IoT device under the target network; each candidate IoT device and environmental state IoT device includes devices with heterogeneous protocols; through a pre-built protocol abstraction layer, the original operating parameters and original environmental state information are cleaned and standardized and mapped to obtain the target operating parameters and target environmental state information respectively.

[0074] In one embodiment, devices with heterogeneous protocols may include devices using protocols such as ZigBee, Wi-Fi, and Matter.

[0075] In one embodiment, such as Figure 3 As shown, in low-power mode, a keyword detection algorithm is run through the low-power processing unit to match specific wake-up words. If a specific wake-up word is matched, the high-performance processing unit in sleep mode is woken up by a hardware interrupt signal. The high-performance processing unit that has been woken up obtains the control voice signal for the target network, performs speech recognition on the control voice signal to obtain the voice control text information for the target network, and constructs prompt words for the large language model based on the voice control text information, target operating parameters, target environment state information and instruction set. The original operating parameters and original environment state information are cleaned and standardized and mapped through a pre-built protocol abstraction layer to obtain the target operating parameters and target environment state information respectively.

[0076] In the above embodiments, data is collected by parallel enabling hardware communication modules, and a protocol abstraction layer is introduced to standardize, clean, and map the raw parameters of heterogeneous devices. This shortens the wake-up latency while shielding the differences in underlying protocols. This improves the system's compatibility with multiple brands and protocols, provides unified and standardized data input for upper-layer voice control, and reduces development complexity and maintenance costs.

[0077] In one embodiment, security verification includes logical conflict verification and parameter compliance verification; verification passing means that both logical conflict verification and parameter compliance verification pass simultaneously; security verification of the target control command includes: obtaining the current device status of the target IoT device; if the current device status is inconsistent with the device status indicated by the target control command, the logical conflict verification is determined to pass; determining the range of physical capabilities supported by the target IoT device; if the target control command contains numerical parameters and the numerical parameters fall within the range of physical capabilities, the parameter compliance verification is determined to pass.

[0078] Specifically, the computer device can obtain the current device status of the target IoT device; if the current device status is inconsistent with the device status indicated by the target control command, the logical conflict check is deemed to have passed; otherwise, the logical conflict check is deemed to have failed (i.e., state redundancy, such as sending an enable command to an already enabled IoT device); determine the physical capability range supported by the target IoT device (e.g., the temperature adjustment range of an air conditioner is 16-30 degrees Celsius); if the target control command contains numerical parameters and the numerical parameters fall within the physical capability range, the parameter compliance check is deemed to have passed.

[0079] In the above embodiments, a dual verification mechanism is used to verify the logical consistency between the instruction and the current state of the device, as well as whether the numerical parameters fall within the physical capabilities of the device. While leveraging a large model to enhance interactive flexibility, the security and executability of instructions are ensured from both logical and physical dimensions, effectively preventing device conflicts and control overreach risks caused by model illusions or user misoperations.

[0080] In one embodiment, after the step of sending the target control command to the target IoT device to cause the target IoT device to execute the target control command if the verification passes, the method further includes: obtaining the current device state of the target IoT device; if the current device state is consistent with the device state indicated by the target control command, then feedback execution success information; if they are inconsistent, then feedback execution failure information.

[0081] In the above embodiments, an execution feedback mechanism is introduced after the command is issued. By comparing the actual state of the device with the expected state of the command, the execution result is confirmed in a timely manner and success or failure information is fed back to the user. This enhances the system's closed-loop monitoring capability of the control results, facilitates user awareness of the operating status, and allows for rapid problem location in case of anomalies, thereby improving the reliability and transparency of the interaction.

[0082] In one embodiment, the method further includes: if parsing the control information fails or the target control instruction does not contain the target IoT device, then inputting the control information into a large language model to generate an error message based on the control information through the large language model.

[0083] In the above embodiments, when parsing fails or instructions are missing, control information is sent back to the large language model to generate user-friendly error messages. Compared to traditional systems that only return generic error codes, this mechanism can utilize the model's semantic understanding capabilities to explain the reasons for failure or guide users to supplement device information, reducing interaction frustration and improving user experience and error correction efficiency.

[0084] In one embodiment, the dynamic physical capability boundary is calculated using at least one of a rule engine, a security policy model, a device capability model, a user preference model, or a scenario constraint model; wherein, the dynamic physical capability boundary includes at least one of an action allowed list, an action prohibited list, parameter upper and lower limits, device mutual exclusion relationships, action priority, action triggering conditions, or security circuit breaker conditions.

[0085] In the above embodiments, a physical capability boundary containing elements such as an allowed list, parameter upper and lower limits, and mutual exclusion relationships is dynamically generated through a multi-source model including a rule engine, user preferences, and scenario constraints. By deeply integrating security rules with user habits, fine-grained and adaptive construction of constraint boundaries is achieved, effectively preventing device malfunctions or conflicts, and improving user experience and scenario adaptability while ensuring control security.

[0086] In one embodiment, such as Figure 4As shown, in low-power mode, a keyword detection algorithm is run by the low-power processing unit to match specific wake-up words. If a specific wake-up word is matched, a hardware interrupt signal wakes up the high-performance processing unit in sleep mode. The awakened high-performance processing unit acquires the control voice signal for the target network, performs speech recognition on the control voice signal, and obtains the voice control text information for the target network. During the wake-up process of the high-performance processing unit, the hardware communication module polls each candidate IoT device and environmental status IoT device under the target network to obtain the original operating parameters of each candidate IoT device and the original environmental status information collected by the environmental status IoT device. Each candidate IoT device and environmental status IoT device contains devices with heterogeneous protocols. Through a pre-built protocol abstraction layer, the original operating parameters and original environmental status information are cleaned and standardized to obtain the target operating parameters and target environmental status information respectively. The awakened high-performance processing unit receives the target operating parameters and target environmental status information sent by the hardware communication module. Based on the target operating parameters and target environment state information, the dynamic physical capability boundary at the current moment is determined. The dynamic physical capability boundary is used to characterize at least one of the following in the current state of the controlled physical system: allowed actions, prohibited actions, parameter value range, state mutual exclusion relationship, and safety constraints. Based on the voice control text information and the dynamic physical capability boundary, at least one of the following is constrained in the large language model: candidate action set, tool definition, generation path, candidate lexical probability distribution, or generated action space, so that the large language model generates control information that conforms to the dynamic physical capability boundary and contains the target control instruction. The target control instruction is one of at least one reference control instruction set for the candidate IoT device contained in the generated action space. If the target control instruction and the target IoT device contained in the target control instruction are parsed from the control information, the current device state of the target IoT device is obtained. If the current device state is inconsistent with the device state indicated by the target control instruction, the logical conflict verification is deemed to have passed. The range of physical capabilities supported by the target IoT device is determined. If the target control instruction contains numerical parameters and the numerical parameters fall within the range of physical capabilities, the parameter compliance verification is deemed to have passed. Obtain the current device status of the target IoT device; if the current device status matches the device status indicated by the target control command, a success message is returned; otherwise, a failure message is returned. If parsing the control information fails, or the target control command does not contain the target IoT device, the control information is input into the large language model to generate an error message based on the control information.

[0087] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially, these steps are not necessarily executed in that order. Unless otherwise expressly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the above embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0088] Based on the same inventive concept, this application also provides a device control apparatus for implementing the device control method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more device control apparatus embodiments provided below can be found in the limitations of the device control method described above, and will not be repeated here.

[0089] In one embodiment, such as Figure 5 As shown, a device control device 500 is provided, which specifically includes:

[0090] The acquisition module 502 is used to acquire voice control text information for the target network, and to acquire the target operating parameters and environmental status information of each candidate IoT device under the target network, as well as the target environmental status information collected by the IoT devices.

[0091] The determination module 504 is used to determine the dynamic physical capability boundary at the current moment based on the target operating parameters and the target environment state information; the dynamic physical capability boundary is used to characterize at least one of the following: actions that the controlled physical system is allowed to perform, actions that are prohibited from performing, parameter value ranges, state mutual exclusion relationships, and safety constraints in the current state;

[0092] The generation module 506 is used to constrain at least one of the candidate action set, tool definition, generation path, candidate word probability distribution or generation action space of the large language model based on voice control text information and dynamic physical capability boundary, so that the large language model generates control information that conforms to the dynamic physical capability boundary and contains target control instructions; the target control instructions are one of at least one reference control instructions set for candidate IoT devices contained in the generation action space.

[0093] The verification module 508 is used to perform security verification on the target control command if the target control command and the target IoT device contained in the target control command are parsed from the control information.

[0094] The sending module 510 is used to send the target control command to the target IoT device if the verification passes, so that the target IoT device can execute the target control command.

[0095] In one embodiment, the acquisition module 502 is further configured to run a keyword detection algorithm through a low-power processing unit to perform specific wake-up word matching in a low-power mode; if a specific wake-up word is matched, the high-performance processing unit in a dormant state is woken up through a hardware interrupt signal; the control voice signal for the target network is acquired through the woken-up high-performance processing unit, and speech recognition is performed on the control voice signal to obtain voice control text information for the target network.

[0096] In one embodiment, the acquisition module 502 is further configured to, during the process of waking up the high-performance processing unit, poll each candidate IoT device and environmental status IoT device under the target network using the hardware communication module to obtain the target operating parameters of each candidate IoT device and the target environmental status information collected by the environmental status IoT device; and receive the target operating parameters and target environmental status information sent by the hardware communication module through the woken-up high-performance processing unit.

[0097] In one embodiment, the acquisition module 502 is further configured to, during the process of waking up the high-performance processing unit, use the power hardware communication module to poll each candidate IoT device and environmental status IoT device under the target network to obtain the original operating parameters of each candidate IoT device and the original environmental status information collected by the environmental status IoT device under the target network; each candidate IoT device and environmental status IoT device includes devices with heterogeneous protocols; through a pre-built protocol abstraction layer, the original operating parameters and original environmental status information are cleaned and standardized and mapped to obtain the target operating parameters and target environmental status information respectively.

[0098] In one embodiment, security verification includes logical conflict verification and parameter compliance verification; verification passing means that both logical conflict verification and parameter compliance verification pass simultaneously; the verification module 508 is also used to obtain the current device status of the target IoT device; if the current device status is inconsistent with the device status indicated by the target control command, the logical conflict verification is determined to pass; the physical capability range supported by the target IoT device is determined; if the target control command contains numerical parameters and the numerical parameters fall within the physical capability range, the parameter compliance verification is determined to pass.

[0099] In one embodiment, the apparatus further includes:

[0100] The feedback module is used to obtain the current device status of the target IoT device; if the current device status is consistent with the device status indicated by the target control command, it will provide feedback on successful execution; otherwise, it will provide feedback on failed execution.

[0101] In one embodiment, the apparatus further includes:

[0102] The feedback module is used to input the control information into the large language model if the parsing of control information fails or the target control command does not contain the target IoT device, so that the large language model can generate error message based on the control information.

[0103] In one embodiment, the dynamic physical capability boundary is calculated using at least one of a rule engine, a security policy model, a device capability model, a user preference model, or a scenario constraint model; wherein, the dynamic physical capability boundary includes at least one of an action allowed list, an action prohibited list, parameter upper and lower limits, device mutual exclusion relationships, action priority, action triggering conditions, or security circuit breaker conditions.

[0104] The aforementioned device control unit acquires voice control text information for the target network, and acquires target operating parameters and environmental state information collected by the IoT devices for each candidate IoT device under the target network. Based on the target operating parameters and target environmental state information, it determines the dynamic physical capability boundary at the current moment. The dynamic physical capability boundary is used to characterize at least one of the following: actions allowed to be executed, actions prohibited to be executed, parameter value range, state mutual exclusion relationship, and security constraints of the controlled physical system in the current state. Based on the voice control text information and the dynamic physical capability boundary, it constrains at least one of the following in the large language model: candidate action set, tool definition, generation path, candidate lexical probability distribution, or generation action space, so that the large language model generates control information that conforms to the dynamic physical capability boundary and contains target control instructions. The target control instruction is one of at least one reference control instruction set for the candidate IoT device contained in the generation action space. If the target control instruction and the target IoT device contained in the target control instruction are parsed from the control information, the target control instruction is subjected to security verification. If the verification passes, the target control instruction is sent to the target IoT device so that the target IoT device executes the target control instruction. Compared to traditional device control methods, this application effectively addresses the industry pain points of rigid semantic understanding in traditional rule-based systems and the "probability-determinism" contradiction in general large models by introducing dynamic physical capability boundaries. On one hand, it breaks through the dependence on fixed instruction templates, giving the system the ability to generalize and understand ambiguous or implicit intentions; on the other hand, it utilizes real-time operating parameters and environmental states to construct physical constraints, fundamentally suppressing the risk of generating illusory instructions in large models. Simultaneously, combined with a security verification mechanism, it ensures that generated instructions strictly conform to the controllable range of physical devices, improving both the naturalness of interaction and the accuracy and security of device control.

[0105] Each module in the aforementioned equipment control device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0106] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and databases. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a device control method.

[0107] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a device control method. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0108] Those skilled in the art will understand that Figure 6 and Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0109] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0110] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0111] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0112] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0113] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0114] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0115] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A device control method, characterized in that, The method includes: Acquire voice control text information for the target network, and acquire target operating parameters and environmental status information of each candidate IoT device under the target network; Based on the target operating parameters and the target environment state information, the dynamic physical capability boundary at the current moment is determined; the dynamic physical capability boundary is used to characterize at least one of the following: actions that the controlled physical system is allowed to perform, actions that are prohibited from performing, parameter value ranges, state mutual exclusion relationships, and safety constraints in the current state; Based on the voice control text information and the dynamic physical capability boundary, at least one of the candidate action set, tool definition, generation path, candidate lexical probability distribution, or generation action space of the large language model is constrained so that the large language model generates control information that conforms to the dynamic physical capability boundary and contains target control instructions; the target control instructions are one of at least one reference control instructions set for the candidate IoT device contained in the generation action space. If the target control command and the target IoT device contained in the target control command are parsed from the control information, then the target control command is subjected to security verification. If the verification passes, the target control command is sent to the target IoT device so that the target IoT device executes the target control command.

2. The method according to claim 1, characterized in that, The acquisition of voice control text information for the target network includes: In low-power mode, a keyword detection algorithm is run through a low-power processing unit to perform specific wake-up word matching; If the specific wake-up word is matched, the high-performance processing unit in sleep mode is woken up by a hardware interrupt signal; The high-performance processing unit, which has been activated, acquires the control voice signal for the target network, performs speech recognition on the control voice signal, and obtains the voice control text information for the target network.

3. The method according to claim 2, characterized in that, The acquisition of target operating parameters and environmental status information of each candidate IoT device under the target network includes: During the process of waking up the high-performance processing unit, the hardware communication module polls each candidate IoT device and environmental status IoT device under the target network to obtain the target operating parameters of each candidate IoT device and the target environmental status information collected by the environmental status IoT device. The high-performance processing unit, once activated, receives the target operating parameters and target environment status information sent by the hardware communication module.

4. The method according to claim 3, characterized in that, During the process of waking up the high-performance processing unit, the hardware communication module polls each candidate IoT device and environmental status IoT device under the target network to obtain the target operating parameters of each candidate IoT device and the target environmental status information collected by the environmental status IoT device, including: During the process of waking up the high-performance processing unit, the hardware communication module polls each candidate IoT device and environmental status IoT device under the target network to obtain the original operating parameters of each candidate IoT device under the target network and the original environmental status information collected by the environmental status IoT device; the candidate IoT devices and the environmental status IoT device include devices with heterogeneous protocols. By using a pre-built protocol abstraction layer, the original operating parameters and the original environmental state information are cleaned and standardized and mapped to obtain the target operating parameters and the target environmental state information respectively.

5. The method according to claim 1, characterized in that, The security verification includes logical conflict verification and parameter compliance verification; The verification passing means that both the logical conflict verification and the parameter compliance verification pass simultaneously; The security verification of the target control command includes: Obtain the current device status of the target IoT device; If the current device state is inconsistent with the device state indicated by the target control command, then the logical conflict check is deemed to have passed. Determine the range of physical capabilities supported by the target IoT device; If the target control command contains numerical parameters and the numerical parameters fall within the range of the physical capabilities, then the parameter compliance verification is deemed to have passed.

6. The method according to claim 1, characterized in that, After the step of sending the target control command to the target IoT device if the verification passes, so that the target IoT device executes the target control command; the method further includes: Obtain the current device status of the target IoT device; If the current device status is consistent with the device status indicated by the target control command, then a success message is returned. If there is a discrepancy, an execution failure message will be returned.

7. The method according to claim 1, characterized in that, The method further includes: If parsing the control information fails, or if the target control command does not include the target IoT device, the control information is input to the large language model to generate an error message based on the control information.

8. The method according to claim 1, characterized in that, The dynamic physical capability boundary is calculated using at least one of the following: rule engine, security policy model, device capability model, user preference model, or scenario constraint model. The dynamic physical capability boundary includes at least one of the following: an action allowable list, an action prohibition list, parameter upper and lower limits, device mutual exclusion relationship, action priority, action triggering condition, or safety circuit breaker condition.

9. A device control system, characterized in that, The device includes: The acquisition module is used to acquire voice control text information for the target network, and to acquire the target operating parameters and environmental status information of each candidate IoT device under the target network, as well as the target environmental status information collected by the IoT devices. The determination module is used to determine the dynamic physical capability boundary at the current moment based on the target operating parameters and the target environment state information; the dynamic physical capability boundary is used to characterize at least one of the following: actions that the controlled physical system is allowed to perform, actions that are prohibited from performing, parameter value ranges, state mutual exclusion relationships, and safety constraints in the current state; The generation module is used to constrain at least one of the candidate action set, tool definition, generation path, candidate lexical probability distribution, or generation action space of the large language model based on the voice control text information and the dynamic physical capability boundary, so that the large language model generates control information that conforms to the dynamic physical capability boundary and contains target control instructions; the target control instructions are one of at least one reference control instructions set for the candidate IoT device contained in the generation action space. The verification module is used to perform security verification on the target control command if the target control command and the target IoT device contained in the target control command are parsed from the control information. The sending module is configured to send the target control command to the target IoT device if the verification passes, so that the target IoT device executes the target control command.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images