A physical device control method and device based on a dynamic large language model tool

Abstract

This invention provides a physical device control method and apparatus based on a dynamic large language model tool. The method includes: acquiring operation intent information for a target physical device and retrieving the corresponding object model of the target physical device; extracting the device action definition and the attribute parameter constraints associated with the device action definition; converting the device action definition into a tool invocation format supported by the large language model, and mapping and encoding the physical constraint boundaries in the attribute parameter constraints into the parameter rules of the tool invocation format to generate a tool definition; injecting the tool definition into the interaction context of the large language model; and, based on the interaction context containing the tool definition, using the large language model to infer the operation intent information and output device operation instructions that conform to the physical constraint boundaries. By using a deterministic object model-driven and runtime dynamic adaptation mechanism, combined with pre-coded physical boundary constraints, the reliability of intelligent device control and the system's zero-code extensibility are improved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and the Internet of Things, and in particular to a physical device control method based on a dynamic large language model tool. Background Technology

[0002] With the rapid development of artificial intelligence and Internet of Things (IoT) technologies, using Large Language Models (LLMs) to control physical devices (such as smart home devices and industrial IoT devices) through natural language interaction has become an important trend in intelligent upgrading. LLMs, through tool invocation capabilities, can translate users' natural language intentions into specific device operation commands.

[0003] However, existing solutions combining large language models with physical devices still have many pain points. Specifically, existing solutions typically hard-code the device's operation interface and parameter range into the system code or model prompts. This static coupling results in extremely poor system scalability. Once a physical device is added or changed, the backend code needs to be modified or the prompts rewritten, making it difficult to adapt to massive and dynamically changing IoT scenarios.

[0004] More importantly, large language models, due to their probabilistic generation mechanism, inevitably suffer from the "illusion" problem. When the model is allowed to run freely, it is very easy to generate illegal parameters that exceed the limits of physical devices (for example, setting the air conditioner temperature to 100 degrees). Existing technologies often lack a mechanism to deeply bind the attribute constraints of physical devices (i.e., the boundaries defined in the object model) with the tool definitions of the large language model, causing the model to be unable to perceive the physical constraint boundaries during the inference phase, which can easily damage real physical devices or cause safety accidents.

[0005] At present, the integration of large language models and physical devices lacks a dynamic tool generation mechanism and fails to effectively combine physical models to establish a safe boundary to prevent model illusion. Summary of the Invention

[0006] This invention provides a physical device control method and apparatus based on a dynamic large language model tool, which solves the problems of high device access coupling and security risks caused by model illusion when controlling physical devices.

[0007] In a first aspect, the present invention provides a physical device control method based on a dynamic large language model tool, comprising: Obtain the operation intent information for the target physical device, and retrieve the object model corresponding to the target physical device; The object model is parsed to extract the device action definition of the target physical device and the attribute parameter constraints associated with the device action definition. The device action definition is converted into a tool call format supported by a large language model, and the physical constraint boundaries in the attribute parameter constraints are mapped and encoded into the parameter rules of the tool call format, and a tool definition carrying the physical constraint boundaries is jointly generated. The tool definition is injected into the interaction context of the large language model; Based on the interaction context including the tool definition, the large language model is used to infer the operation intent information and output device operation instructions that conform to the physical constraint boundaries.

[0008] Optionally, before obtaining the operation intent information for the target physical device and retrieving the object model corresponding to the target physical device, the method further includes: Receives a structured model file containing device action definitions and attribute parameter constraints from external input; performs structural verification on the structured model file, and persistently stores it in a preset database after successful verification; configures an enable flag for the stored structured model file to indicate whether large language model calls are allowed.

[0009] Optionally, obtaining the operation intent information for the target physical device and retrieving the object model corresponding to the target physical device includes: Receive natural language instructions input by the end user as the operation intent information, and extract the session context identifier associated with the natural language instructions; Based on the session context identifier and the enabled flag that is in an active state, at least one of the target physical devices available in the current scenario is jointly matched in the database; Read the object models corresponding to each target physical device that is matched from the database.

[0010] Optionally, the object model is a JSON description file; the step of converting the device action definition into a tool invocation format supported by a large language model, mapping and encoding the physical constraint boundaries in the attribute parameter constraints to the parameter rules of the tool invocation format, and jointly generating a tool definition carrying the physical constraint boundaries includes: Parse the action attributes in the JSON description file and convert the action name and input parameter structure into a function call architecture file supported by the large language model; Extract the attribute parameter constraints from the JSON description file, and obtain the numerical maximum and minimum values ​​of the corresponding parameters, or the enumeration type range constraints, as the physical constraint boundaries. The obtained physical constraint boundaries are hard-coded into the parameter validation rules of the function call architecture file to generate the tool definition carrying deterministic validation conditions.

[0011] Optionally, injecting the tool definition into the interaction context of the large language model includes: When multiple target physical devices are matched, the tool definition corresponding to each target physical device is generated respectively; Using a pre-defined tool injection module, the generated tool definitions are aggregated into a dynamically available toolbar table for the current scene; The dynamically available toolbar table is injected into and updated in real time into the system prompts of the large language model to construct the interaction context that includes the boundaries of real-time device capabilities.

[0012] Optionally, after inferring the operation intent information using the large language model based on the interaction context including the tool definition and outputting device operation instructions that conform to the physical constraint boundaries, the method further includes: The device operation command is intercepted and parsed, and its compliance with the physical constraint boundary is verified a second time using preset post-processing rules. If the verification passes, the device operation command will be sent to the corresponding target physical device to execute the action; The system receives action execution status data from the target physical device, generates corresponding natural language response content based on the action execution status data using the large language model, and sends the natural language response content to the corresponding user terminal.

[0013] Secondly, the present invention provides a physical device control device based on a dynamic large language model tool, comprising: The information acquisition module is used to acquire the operation intention information for the target physical device and retrieve the object model corresponding to the target physical device; The parsing and extraction module is used to parse the object model and extract the device action definition of the target physical device, as well as the attribute parameter constraints associated with the device action definition. The mapping generation module is used to convert the device action definition into a tool call format supported by a large language model, and to map and encode the physical constraint boundaries in the attribute parameter constraints to the parameter rules of the tool call format, and jointly generate a tool definition carrying the physical constraint boundaries; An information injection module is used to inject the tool definition into the interaction context of the large language model; The reasoning output module is used to reason about the operation intent information based on the interaction context containing the tool definition, using the large language model, and output device operation instructions that conform to the physical constraint boundaries.

[0014] Thirdly, the present invention provides an electronic device including a processor and a memory, the memory storing computer-readable instructions that, when executed by the processor, perform the steps of the method provided in the first aspect above.

[0015] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the method provided in the first aspect above.

[0016] Fifthly, the present invention provides a computer program product comprising a computer program that, when executed by a processor, performs the steps of the method provided in the first aspect above.

[0017] As can be seen from the above technical solutions, the present invention has the following advantages: This invention provides a physical device control method and apparatus based on a dynamic large language model tool. The method includes: retrieving the object model corresponding to the physical device; parsing and extracting action definitions and attribute parameter constraints; converting them into a tool invocation format and mapping and encoding the physical constraint boundaries into parameter rules to generate tool definitions; injecting them into the context of the large language model and inferring and outputting instructions that conform to the physical constraints. This invention decouples the configuration-state object model from the runtime large language model tool. Device changes only require updating the object model to dynamically generate the latest tool, eliminating the tedious hard-coding modifications and significantly improving the scalability of the IoT system. Simultaneously, by hard-coding the deterministic physical boundaries of the device hardware (such as extreme values ​​and enumerated values) into tool parameter rules during the tool generation stage, physical constraints are placed before the model inference stage, effectively suppressing illegal out-of-bounds instructions caused by the illusion of the large language model, significantly improving the security and reliability of intelligent device control. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the steps of a physical device control method based on a dynamic large language model tool according to the present invention. Figure 2 This is a flowchart illustrating the steps of a second embodiment of a physical device control method based on a dynamic large language model tool according to the present invention. Figure 3 This is a general workflow diagram provided for Embodiment 2 of the present invention; Figure 4 This is a structural block diagram of an embodiment of a physical device control device based on a dynamic large language model tool according to the present invention; Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0020] Preferred embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0021] This invention provides a physical device control method and apparatus based on a dynamic large language model tool, which solves the problems of high device access coupling and security risks caused by model illusion when controlling physical devices.

[0022] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0023] Example 1 Please see Figure 1 , Figure 1 This is a flowchart illustrating the physical device control method based on a dynamic large language model tool according to the present invention. The method includes: Step S101: Obtain the operation intention information for the target physical device, and retrieve the object model corresponding to the target physical device; In this embodiment, the system first receives the user's operational intent information, such as an interactive command like "set the air conditioner temperature in the master bedroom to 22 degrees Celsius" input via natural language voice or text. Then, based on this operational intent information and the current conversational environment (such as a specific hotel room number, family scene ID, or other contextual information), the system accurately retrieves the object model corresponding to the "target physical device" (i.e., the master bedroom air conditioner) from a pre-set database. This object model is a standardized document used to describe the device's digital functions and physical attributes.

[0024] Step S102: Parse the object model and extract the device action definition of the target physical device and the attribute parameter constraints associated with the device action definition; In this embodiment, after retrieving the object model of the target physical device, it undergoes structured analysis. Specifically, this involves accurately extracting the action definitions that the device can perform (e.g., control methods such as "setting temperature" and "switching mode" for an air conditioner) from the object model, and simultaneously extracting the "attribute parameter constraints" closely bound to these actions (e.g., specifying that the set temperature range must be between a minimum of 16 degrees and a maximum of 30 degrees, or that the fan speed can only be an enumeration type of "high, medium, low"). These extracted contents constitute an accurate numerical expression of the physical device's capabilities and its security boundaries.

[0025] Step S103: Convert the device action definition into a tool call format supported by a large language model, and map and encode the physical constraint boundary in the attribute parameter constraint to the parameter rules of the tool call format, and jointly generate a tool definition carrying the physical constraint boundary; In the specific implementation, the extracted device action definitions are translated into a tool call format that the large language model can natively recognize and support, such as automatically generating the corresponding large model JSON Schema data structure. Not only are the function names and parameter data structures converted, but the previously extracted physical constraint boundaries (such as extreme value constraints of minimum 16 and maximum 30) are also forcibly hard-coded into the parameter validation rules of this schema. In this way, a tool definition with built-in physical error prevention capabilities and a natural understanding of the precise boundaries of the hardware is dynamically generated.

[0026] Step S104: Inject the tool definition into the interaction context of the large language model; In this embodiment, the security tool definitions dynamically generated in the previous step and specific to the target physical device in the current scenario are loaded and injected into the interaction context of the large language model (such as the list of available tools in the system prompts). Through this on-demand dynamic injection method, the large language model can see and understand in real time which operable device tools it currently possesses when performing the next inference step, and clearly know the physical parameter limits that cannot be infringed when using each tool, thereby replacing the traditional redundant and error-prone global hard-coding method.

[0027] Step S105: Based on the interaction context including the tool definition, the large language model is used to infer the operation intent information and output device operation instructions that conform to the physical constraint boundaries.

[0028] In this embodiment, the large language model combines the user's initial operational intent with the dynamically injected tool definition carrying deterministic physical constraint boundaries to perform deep semantic understanding and logical reasoning. Since the JSON schema of the tool definition tightly defines the range of legal parameters, the output of the large model when generating specific function call requests is strictly limited to these physical constraint boundaries (for example, even if the user maliciously commands "set the air conditioner to 100 degrees," the model will reject illegal outputs or use the nearest legal threshold based on the constraint mechanism). Ultimately, structured, secure, and compliant device operation instructions are output for subsequent direct delivery to the target physical device to execute actions.

[0029] This invention provides a physical device control method based on a dynamic large language model tool. The method involves acquiring operational intent information for a target physical device and retrieving the corresponding object model. The object model is parsed to extract the device action definition and associated attribute parameter constraints of the target physical device. The device action definition is converted into a tool invocation format supported by the large language model, and the physical constraint boundaries in the attribute parameter constraints are mapped and encoded into the parameter rules of the tool invocation format, jointly generating a tool definition carrying the physical constraint boundaries. The tool definition is injected into the interaction context of the large language model. Based on the interaction context containing the tool definition, the large language model is used to infer the operational intent information, outputting device operation instructions that conform to the physical constraint boundaries. By employing a "deterministic object model-driven" and "runtime dynamic adaptation" mechanism, complete decoupling between the device access interface and the large language model prompting logic is achieved, eliminating the high hard-coding maintenance costs associated with adding or changing devices. Simultaneously, it creatively maps the static physical boundaries in the object model to built-in parameter constraints in the large model tool calling layer, fundamentally curbing security risks such as out-of-bounds operations and device damage caused by model "illusions." This significantly improves the reusability, iteration efficiency, and absolute security of intelligent control in cross-scenario deployment of the Internet of Things. This invention provides a physical device control method and apparatus based on a dynamic large language model tool, addressing issues such as high device access coupling and the security risks of out-of-bounds operation commands arising from model illusions when controlling physical devices using large language models.

[0030] Example 2 Please see Figure 2 and Figure 3 , Figure 2 This is a flowchart illustrating another physical device control method based on a dynamic large language model tool according to the present invention. Figure 3This is an overall workflow diagram provided for an embodiment of the present invention. The method in this embodiment is divided into an upper configuration environment (i.e., offline configuration state) and a lower runtime environment (i.e., real-time runtime state) in terms of workflow architecture, so as to achieve complete decoupling between device capability configuration and real-time calling logic of large models.

[0031] The steps of the physical device control method based on the dynamic large language model tool include: Step S201: Receive an externally input structured object model file containing device action definitions and attribute parameter constraints; perform structural verification on the structured object model file, and persistently store it in a preset database after successful verification; configure an enable flag for the stored structured object model file to indicate whether large language model calls are allowed. In this embodiment, this step belongs to the offline configuration stage (configuration state). Product managers or system administrators can upload a standard-compliant (such as W3C WoT standard) JSON file through a visual management backend. This file details the capabilities of the physical device. The file's format validity and field completeness are verified, and it is stored in a database such as PostgreSQL or MongoDB. Specific devices requiring access to large model control are marked with an enable flag (e.g., configuring ai_enabled=true), thus achieving zero-code extension in the configuration state without modifying any backend business code.

[0032] like Figure 3 As shown in the configuration environment section, non-technical personnel (such as product managers, PMs) can perform the "1. Upload / Edit Object Model" operation through the configuration module (PMDashboard), enter the JSON file conforming to the W3C WoT standard shown in the dashed box, and then perform "2. Storing Data" to persist the structured data to the backend database (such as PostgreSQL / Mongo).

[0033] Step S202: Receive natural language instructions input by the terminal user as the operation intent information, and extract the session context identifier associated with the natural language instructions; In practice, the system receives natural language commands from users (e.g., "Could you dim the lights for me?") via smart terminals (such as smart speakers, mobile apps, or hotel room tablets). Simultaneously, it automatically parses the source of the request, extracting information such as the hotel room number (hotel_id), family group ID, or user unique identifier. This information serves as a session context identifier, defining the physical scope of subsequent device control.

[0034] Step S203: Based on the session context identifier and the enabled flag that is in an active state, jointly match at least one of the target physical devices available in the current scenario in the database; In this embodiment of the application, the obtained session context identifier (such as a specific hotel room number) is associated with the database for querying, and the aforementioned enable flag is added as a filtering condition, so as to accurately filter out all online physical devices (such as smart lights, smart curtains, etc. in the room) in the specific room that can be controlled by the large language model.

[0035] Step S204: Read the object models corresponding to each matched target physical device from the database; In this embodiment of the application, for each available target physical device selected in the previous step, the complete and standardized object model data that was pre-entered in the configuration stage is extracted from the database and used as the underlying driving base for the subsequent dynamic generation of large model tools.

[0036] like Figure 3 As shown in the process node connecting the upper and lower environments, the data flow of "4. Query ai_enabled devices" will be triggered in the running state, and the matching device object model will be pulled from the underlying PostgreSQL / Mongo database on demand, thereby connecting the link between static configuration data and the dynamic large language model at runtime.

[0037] Step S205: Parse the object model and extract the device action definition of the target physical device and the attribute parameter constraints associated with the device action definition; In the specific implementation, for each object model file read, the core conversion and adaptation module (SchemaAdapter) starts working, accurately analyzing the action definition module (such as "setBrightness") and attribute definition module (such as brightness value range) to prepare data for the next step of protocol format conversion.

[0038] Step S206: Parse the action attributes in the JSON description file and convert the action name and input parameter structure into a function call architecture file supported by the large language model; like Figure 3 As shown in the Runtime Phase section, the queried device model data first enters the "Schema Adapter" module, which performs the "5. Parse W3C WoT TD" operation and then flows to the "Extract Methods & Properties" processing node.

[0039] In this embodiment, the object model is specifically a JSON description file. Relevant action information is extracted from the JSON and, according to the function calling data interaction specifications required by a specific large language model (such as OpenAI), it is transformed into a standardized function call architecture file (JSON Schema format). This "translates" the originally complex IoT device protocol into skill declarations that the large language model can directly invoke.

[0040] Step S207: Extract the attribute parameter constraints from the JSON description file, and obtain the numerical maximum and minimum values ​​of the corresponding parameters, or the enumeration type range constraints, as the physical constraint boundaries. In this embodiment, to prevent large models from generating out-of-bounds parameters, the original constraints in the model are extracted in depth. For example, the minimum and maximum values ​​set by the device are read, or the supported enumeration ranges such as wind speed are read, and these real physical hardware limits are extracted as the physical constraint boundaries of the large model.

[0041] Step S208: The obtained physical constraint boundary is hard-coded into the parameter verification rules of the function call architecture file to generate the tool definition carrying deterministic verification conditions; In the actual implementation, the extreme values ​​or specific enumerated values ​​extracted in the previous step are directly written into the function call architecture file generated earlier as a defined verification rule. This step gives the generated tool definition a built-in "anti-misoperation" security feature, and when the large language model calls this tool, the data it generates will be restricted to this compliant framework.

[0042] Combination Figure 3 As can be seen, through the above conversion and hard-coded mapping mechanism, executing "6. Generate OpenAI ToolSchema" will finally output the Function Calling Schema shown in the dashed box on the right side of the figure, providing a standard tool specification for subsequent large language model calls.

[0043] Step S209: When multiple target physical devices are matched, the tool definition corresponding to each target physical device is generated respectively; In this application embodiment, since multiple smart devices often exist concurrently in an application scenario (such as a room), the above parsing and conversion logic will be executed cyclically to generate a tool definition for each device that is unique to it and has strict physical constraint boundaries.

[0044] Step S210: Using a preset tool injection module, aggregate the generated multiple tool definitions into a dynamic available toolbar table for the current scene; In this embodiment of the application, the tool injection module (such as MCP Server) will aggregate and package the multiple independent tool definitions just generated to form a dedicated "dynamic toolbox" or "toolbar table" for the current session user, so as to reflect the list of all real and available devices in the current scenario.

[0045] Step S211: The dynamic available toolbar table is injected and updated in real time into the system prompt words of the large language model to construct the interaction context containing the real-time device capability boundary; like Figure 3 As shown, the “3. Natural Language Command” initiated by the end user and the aforementioned dynamically generated Function Calling Schema tool definition are converged into the “Tool Injection Module (MCP Server)”. This module aggregates the intent command with the available tool pool and splices them into a complete prompt word data stream of “7. System Prompt + Tools”.

[0046] In this embodiment, before initiating an inference request to the large language model, the aforementioned aggregated dynamic available toolbar table is injected in real time as an available toolset into the interaction context of the large language model (such as the system prompt). This dynamic, on-demand injection mechanism ensures that the large model always has the most accurate information about the currently available devices and avoids the waste of redundant token resources caused by traditional global hard coding.

[0047] Step S212: Based on the interaction context including the tool definition, the large language model is used to reason about the operation intention information and output device operation instructions that conform to the physical constraint boundaries. In this embodiment, the large language model integrates the user's natural language commands and the newly injected dynamic toolset for inference. Since the tools it calls inherently possess physical constraint boundary conditions, the structured tool call requests generated by the model naturally converge within the valid range, thereby outputting standard and safe device operation commands.

[0048] like Figure 3 As shown at the bottom of the process, after receiving the complete "System Prompt + Tools" context, the Large Language Model (LLM) performs inference, executes "8. Output Structured JSON", and finally parses and sends out the underlying "Device Execution Instructions".

[0049] Step S213: Intercept and parse the device operation command, and use preset post-processing rules to perform a second verification to check whether it conforms to the physical constraint boundary; if the verification passes, send the device operation command to the corresponding target physical device to perform the action; receive the action execution status data fed back by the target physical device, and generate the corresponding natural language response content based on the action execution status data through the large language model, and send the natural language response content to the corresponding user terminal.

[0050] In this embodiment, as a double security measure, a post-processing interception layer is set up before actually controlling the hardware. This layer performs a secondary verification of the large model's output based on the previously defined physical constraints, completely blocking illegal instructions caused by the extremely low probability of the model's "illusion." After successful verification, the instruction is finally sent to the physical device for execution. Finally, the device feeds back the execution result, driving the large language model to reorganize into natural and coherent semantics and reply to the end user, completing a full closed-loop interaction of "natural language-physical device-natural language."

[0051] The following section provides a detailed description of three representative application scenarios.

[0052] Scenario 1: Smart Control Scenario for Hotel Rooms In the context of smart hotels, different hotels, and even different rooms (such as standard rooms and suites) in the same hotel, are usually equipped with completely different models of physical equipment.

[0053] Suppose a traveler checks into room 808 of hotel A and issues a vague, emotional natural language command through the smart speaker in the room: "It's so hot, turn the air conditioning down to the lowest setting right now and freeze me into an ice cube!"

[0054] In traditional technologies, large language models are highly susceptible to being misled by extreme terms such as "lowest" and "frozen into ice," leading to "illusions" and the output of illegal control commands such as {"temperature": 0} or even negative numbers. Without exhaustive interception of business code, this can directly cause errors on the air conditioner's mainboard or damage to the device.

[0055] In this embodiment: First, dynamic matching is performed (corresponding to S202-S204) to extract the context identifier of the current session (hotel_id=A, room_id=808), and the W3C WoT object model of the "model X air conditioner" installed in room 808 is accurately retrieved from the underlying database.

[0056] Then, constraint extraction and tool conversion are performed (corresponding to S205-S208). When parsing the object model, the conversion module accurately extracts the temperature attribute constraint of the air conditioner as properties: { "targetTemperature": { "type": "integer", "minimum": 16, "maximum": 30}}. The two physical extreme values, minimum: 16 and maximum: 30, are then hard-coded into the JSON Schema validation rules provided to OpenAI Function Calling.

[0057] Finally, safe reasoning and execution (corresponding to S211-S213) are performed. When the large language model receives the interaction context containing the strict boundary tool definition, it clearly "perceives" the physical limits of the hardware during the reasoning stage. Even in extreme contexts, the large model will find the optimal solution within the legal constraints, thus safely outputting the instruction {"targetTemperature": 16}. Subsequently, this instruction is sent to the physical device, and the speaker naturally responds: "The air conditioner has been adjusted to the lowest temperature of 16 degrees for you. Please enjoy the coolness." This fundamentally prevents out-of-bounds operations that could damage the hardware.

[0058] Scenario 2: High-frequency iteration and expansion of smart home devices In the smart home ecosystem, device categories and models are updated extremely rapidly. For example, a family that originally had an old-fashioned smart curtain that only supported "open / close" recently purchased and replaced it with a new smart curtain that supports "stepless dimming and specific preset positions".

[0059] In traditional technologies, when faced with such changes in underlying hardware capabilities, development teams must modify the backend API processing logic, rewrite the System Prompt of the large model, and even re-release the service version, resulting in extremely high maintenance costs and an inability to meet the access needs of a massive number of long-tail devices.

[0060] In this embodiment: Before running, offline zero-code configuration is required. Product managers or system administrators with no coding experience only need to log in to the PM Dashboard (configuration module) and upload the standard W3C WoT object model JSON file provided by the new curtain manufacturer. This file defines new actions: `setCurtainState` and enumeration constraints: `enum: ["fully open", "half open", "ventilation mode", "fully closed"]`. After verification, the configuration is stored in the database and `ai_enabled=true` is checked. The configuration takes effect immediately, without requiring a service restart or code modification.

[0061] Then, the runtime will be mounted in real time. When the user arrives home and says "turn the curtains to ventilation mode", the latest uploaded object model will be pulled in real time. The tool injection module (MCP Server) will dynamically assemble the tool definition containing the new enum constraints into the toolbar table.

[0062] Based on the newly injected tool declarations, the large language model immediately "learned" the control method of the new device and accurately output {"action": "setCurtainState", "state": "ventilation mode"}. This scenario fully demonstrates the high scalability of "one core logic, direct reuse in multiple scenarios", completely breaking the strong coupling between the device and the AI ​​control logic.

[0063] Scenario 3: High-risk physical control scenarios for service robots / industrial IoT In scenarios involving delivery robots in hotel lobbies or robotic arms in industrial workshops, the safety tolerance for physical movements is zero. Exceeding these limits could lead to serious physical collisions or personal injury.

[0064] Suppose a user gives a natural language command to a lobby service robot: "There's an emergency at the front desk, rush over there as fast as you can!" In this embodiment: By parsing the robot object model, the maximum speed threshold for safe operation is extracted (e.g., properties: { "speed": { "maximum": 1.2}}, in m / s), and this threshold is forcibly encoded into the tool invocation rules (Schema) of the larger model. When the larger model reasones about the semantics of "fastest speed" and "rushing through," it will be forcibly converged to the legal limit value of 1.2 m / s by this preceding physical constraint boundary.

[0065] Considering rare cases (such as sudden fluctuations in the cloud-based large model base), the large model might disregard the input schema constraints and forcibly output the illusionary command {"speed": 5.0}. In this case, the post-processing interception network preset in step S213 will force a secondary comparison and verification between the generated 5.0 and the original maximum=1.2 in the object model database before sending the command to the robot's underlying control board. Since 5.0>1.2, a verification failure interception will be triggered, refusing to send the high-risk command, and throwing an error callback to the large model, requiring it to regenerate or directly broadcasting a safety warning to the user ("Sorry, for safety reasons, I can only go to the foreground at a maximum speed of 1.2m / s").

[0066] This scenario demonstrates a dual security closed-loop mechanism of "pre-emptive physical boundary constraints (constraining LLM thinking) + post-instruction output interception (covering up LLM behavior)," ensuring the absolute operational safety of AI-driven high-risk physical devices.

[0067] This invention provides a physical device control method based on a dynamic large language model tool. The method receives an offline configured object model file and establishes an activation flag. During runtime, it dynamically matches available physical devices using natural language and contextual identifiers, retrieves the corresponding object model, converts the object model into a tool call format of the large language model, and hard-codes physical constraint boundaries. It then injects a dynamically generated toolbar table into the large language model context in real time, thereby inferring and outputting secure device operation commands, supplemented by post-verification and natural language feedback. This solution not only achieves "zero-code dynamic translation" and on-demand assembly from the IoT underlying protocol to the large model application layer protocol, significantly reducing the cost of cross-scenario reuse and device system iteration; but also, through a security defense line of "pre-physical boundary constraint verification + post-command output verification," it completely solves the problem of potential damage to physical devices due to illusions caused by the large language model, improving the security and reliability of the entire AIoT interaction system.

[0068] Example 3 Please see Figure 4 , Figure 4 This is a structural block diagram of an embodiment of a physical device control device based on a dynamic large language model tool according to the present invention. The device includes: The information acquisition module 301 is used to acquire operation intention information for the target physical device and retrieve the object model corresponding to the target physical device; The parsing and extraction module 302 is used to parse the object model and extract the device action definition of the target physical device and the attribute parameter constraints associated with the device action definition. The mapping generation module 303 is used to convert the device action definition into a tool call format supported by a large language model, and to map and encode the physical constraint boundary in the attribute parameter constraint to the parameter rules of the tool call format, and jointly generate a tool definition carrying the physical constraint boundary; Information injection module 304 is used to inject the tool definition into the interaction context of the large language model; The reasoning output module 305 is used to reason about the operation intent information based on the interaction context containing the tool definition using the large language model, and output device operation instructions that conform to the physical constraint boundaries.

[0069] In an optional embodiment, the device further includes a configuration front-end module, configured to receive an externally input structured object model file containing device action definitions and attribute parameter constraints before acquiring operation intent information for the target physical device; perform structural verification on the structured object model file, and persistently store it in a preset database after successful verification; and configure an enable flag for the stored structured object model file to characterize whether large language model calls are allowed.

[0070] In an optional embodiment, the information acquisition module 301 includes: The instruction receiving submodule is used to receive natural language instructions input by the terminal user as the operation intent information, and extract the session context identifier associated with the natural language instructions. The joint matching submodule is used to jointly match at least one of the target physical devices available in the current scenario in the database based on the session context identifier and the enable flag that is in an active state. The model reading submodule is used to read the object models corresponding to each target physical device that has been matched from the database.

[0071] In an optional embodiment, the object model is a JSON description file; the mapping generation module 303 includes: The architecture transformation submodule is used to parse the action attributes in the JSON description file and convert its action name and input parameter structure into a function call architecture file supported by the large language model. The boundary extraction submodule is used to extract the attribute parameter constraints in the JSON description file and obtain the numerical maximum and minimum values ​​of the corresponding parameters, or the enumeration type range constraints, as the physical constraint boundaries. The rule hard-coding submodule is used to hard-code the acquired physical constraint boundaries into the parameter validation rules of the function call architecture file, thereby generating the tool definition carrying deterministic validation conditions.

[0072] In an optional embodiment, the information injection module 304 includes: It also generates a submodule, which is used to generate the tool definition corresponding to each of the target physical devices when multiple target physical devices are matched; The tool aggregation submodule is used to aggregate multiple generated tool definitions into a dynamic available toolbar table for the current scene using a preset tool injection module; The prompt word update submodule is used to inject and update the dynamic available toolbar table into the system prompt words of the large language model in real time, so as to construct the interaction context containing the real-time device capability boundaries.

[0073] In an optional embodiment, the device further includes a post-processing verification module, which is used to intercept and parse the device operation command after it is output, and to perform a second verification using preset post-processing rules to check whether it conforms to the physical constraint boundary; if the verification passes, the device operation command is sent to the corresponding target physical device to perform the action; the device receives the action execution status data fed back by the target physical device, and generates the corresponding natural language response content based on the action execution status data through the large language model, and sends the natural language response content to the corresponding user terminal.

[0074] Example 4 Based on the same inventive concept, embodiments of the present invention also provide an electronic device. Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Figure 5 As shown, an embodiment of the present invention provides an electronic device including: one or more processors 401, a memory 402, and one or more I / O interfaces 403. The memory 402 stores one or more programs, which, when executed by the one or more processors, enable the one or more processors to implement any of the physical device control methods based on dynamic large language model tools in the above embodiments; the one or more I / O interfaces 403 are connected between the processor and the memory, configured to enable information interaction between the processor and the memory.

[0075] Among them, processor 401 is a device with data processing capabilities, including but not limited to central processing unit (CPU); memory 402 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); I / O interface (read-write interface) 403 is connected between processor 401 and memory 402, and can realize information interaction between processor 401 and memory 402, including but not limited to data bus (Bus).

[0076] In some embodiments, the processor 401, memory 402, and I / O interface 403 are interconnected via bus 404, and thus connected to other components of the computing device.

[0077] In some embodiments, the one or more processors 401 include a field-programmable gate array.

[0078] Example 5 This invention also provides a computer storage medium storing a computer program thereon, wherein the computer program, when executed by the processor, implements the steps of a physical device control method based on a dynamic large language model tool according to any embodiment.

[0079] Example 6 This invention also provides a computer program product on which a computer program is stored. When the computer program is executed by the processor, it implements the steps of a physical device control method based on a dynamic large language model tool according to any embodiment.

[0080] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0081] In the several embodiments provided in this application, it should be understood that the methods, apparatuses, electronic devices, and storage media disclosed in this invention can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0082] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0083] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0084] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0085] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A physical device control method based on a dynamic large language model tool, characterized in that, include: Obtain the operation intent information for the target physical device, and retrieve the object model corresponding to the target physical device; The object model is parsed to extract the device action definition of the target physical device and the attribute parameter constraints associated with the device action definition. The device action definition is converted into a tool call format supported by a large language model, and the physical constraint boundaries in the attribute parameter constraints are mapped and encoded into the parameter rules of the tool call format, and a tool definition carrying the physical constraint boundaries is jointly generated. The tool definition is injected into the interaction context of the large language model; Based on the interaction context including the tool definition, the large language model is used to infer the operation intent information and output device operation instructions that conform to the physical constraint boundaries.

2. The physical device control method based on a dynamic large language model tool according to claim 1, characterized in that, Before obtaining the operation intent information for the target physical device and retrieving the object model corresponding to the target physical device, the method further includes: Receives a structured model file containing device action definitions and attribute parameter constraints from external input; performs structural verification on the structured model file, and persistently stores it in a preset database after successful verification; configures an enable flag for the stored structured model file to indicate whether large language model calls are allowed.

3. The physical device control method based on a dynamic large language model tool according to claim 2, characterized in that, The step of obtaining the operation intent information for the target physical device and retrieving the object model corresponding to the target physical device includes: Receive natural language instructions input by the end user as the operation intent information, and extract the session context identifier associated with the natural language instructions; Based on the session context identifier and the enabled flag that is in an active state, at least one of the target physical devices available in the current scenario is jointly matched in the database; Read the object models corresponding to each target physical device that is matched from the database.

4. The physical device control method based on a dynamic large language model tool according to claim 1, characterized in that, The object model is a JSON description file; The process of converting the device action definition into a tool invocation format supported by a large language model, mapping and encoding the physical constraint boundaries in the attribute parameter constraints to the parameter rules of the tool invocation format, and jointly generating a tool definition carrying the physical constraint boundaries includes: Parse the action attributes in the JSON description file and convert the action name and input parameter structure into a function call architecture file supported by the large language model; Extract the attribute parameter constraints from the JSON description file, and obtain the numerical maximum and minimum values ​​of the corresponding parameters, or the enumeration type range constraints, as the physical constraint boundaries. The obtained physical constraint boundaries are hard-coded into the parameter validation rules of the function call architecture file to generate the tool definition carrying deterministic validation conditions.

5. The physical device control method based on a dynamic large language model tool according to claim 1, characterized in that, The step of injecting the tool definition into the interaction context of the large language model includes: When multiple target physical devices are matched, the tool definition corresponding to each target physical device is generated respectively; Using a pre-defined tool injection module, the generated tool definitions are aggregated into a dynamically available toolbar table for the current scene; The dynamically available toolbar table is injected into and updated in real time into the system prompts of the large language model to construct the interaction context that includes the boundaries of real-time device capabilities.

6. The physical device control method based on a dynamic large language model tool according to claim 1, characterized in that, After the step of reasoning about the operation intent information using the large language model based on the interaction context containing the tool definition and outputting device operation instructions that conform to the physical constraint boundaries, the method further includes: The device operation command is intercepted and parsed, and its compliance with the physical constraint boundary is verified a second time using preset post-processing rules. If the verification passes, the device operation command is sent to the corresponding target physical device to perform the action. The action execution status data fed back by the target physical device is received, and the corresponding natural language response content is generated based on the action execution status data through the large language model. The natural language response content is then sent to the corresponding user terminal.

7. A physical device control device based on a dynamic large language model tool, characterized in that: The information acquisition module is used to acquire the operation intention information for the target physical device and retrieve the object model corresponding to the target physical device; The parsing and extraction module is used to parse the object model and extract the device action definition of the target physical device, as well as the attribute parameter constraints associated with the device action definition. The mapping generation module is used to convert the device action definition into a tool call format supported by a large language model, and to map and encode the physical constraint boundaries in the attribute parameter constraints to the parameter rules of the tool call format, and jointly generate a tool definition carrying the physical constraint boundaries; An information injection module is used to inject the tool definition into the interaction context of the large language model; The reasoning output module is used to reason about the operation intent information based on the interaction context containing the tool definition, using the large language model, and output device operation instructions that conform to the physical constraint boundaries.

8. An electronic device, characterized in that, include: processor; The memory is used to store processor-executable instructions; The processor executes the physical device control method based on the dynamic large language model tool as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the physical device control method based on a dynamic large language model tool as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the physical device control method based on the dynamic large language model tool as described in any one of claims 1 to 6.

Images