Heterogeneous device unified control method and apparatus, storage medium, and computer device

Abstract

The heterogeneous device unified control method, apparatus, storage medium, and computer equipment provided in this application register each brand of device that needs to be controlled in the MCP tool list for use by the large language model, breaking down the access barriers between heterogeneous devices of different brands. After receiving the control command input by the user, the large language model generates a tool call request corresponding to the control command based on the list to unify the command format. Then, it parses the target device and semantic parameters in the request and performs security verification. After the verification is passed, a request message is generated to filter out risky commands. At the same time, the protocol route of the target device is obtained by adaptive matching based on the multi-protocol mapping table. After the request message is accurately sent to the target device through the protocol route, the target device performs the corresponding operation and returns a response message. The large language model outputs the natural language description text corresponding to the response message, thereby uniformly realizing the closed loop of automated control of devices of different brands.

Description

Technical Field

[0001] This application relates to the field of equipment control technology, and in particular to a unified control method, device, storage medium and computer equipment for heterogeneous equipment. Background Technology

[0002] With the rapid development of smart homes, industrial IoT and other fields, a large number of heterogeneous devices of different brands and protocol types are often deployed in various scenarios such as homes, factories and offices. The number of connected devices continues to rise, and the original independent control mode of a single device is no longer able to meet the current needs.

[0003] Currently, the mainstream approach to device control is for different brands to build their own dedicated control platforms. Users who want to operate devices of a specific brand can only do so through their respective dedicated control platforms. To achieve cross-brand collaborative control, users need to switch between multiple control applications, resulting in low control efficiency. Even with the rapid development of large language model technology in recent years, and natural language-based interactive control gradually becoming a popular direction for smart device control, the lack of unified access standards for heterogeneous devices from different brands makes it difficult for large language models to directly achieve unified invocation and control of devices from all brands. Summary of the Invention

[0004] The purpose of this application is to at least address one of the aforementioned technical deficiencies, particularly the technical deficiency in the existing technology where the access standards for heterogeneous devices from different brands are not unified, making it difficult for large language models to directly achieve unified invocation and control of all brand devices.

[0005] This application provides a unified control method for heterogeneous devices, the method comprising: Identify multiple brands of devices that need to be controlled, and register each brand of device in the MCP tool list using an inverted index pattern for use by the large language model; When the user inputs a control command through the large language model, the control command is semantically recognized based on the MCP tool list, and a tool invocation request is generated based on the recognition result. The target device and semantic parameters in the tool call request are parsed to perform security verification on the target device and the semantic parameters, and a request message containing the target device and the semantic parameters is generated when the verification passes. The protocol route of the target device is obtained by matching based on a preset multi-protocol mapping table, and the request message is sent to the target device through the protocol route to control the target device to perform operations according to the semantic parameters, and return a response message after the execution is completed; Generate a natural language description text corresponding to the response message, and output the natural language description text through the large language model.

[0006] Optionally, the step of registering each brand of device to the MCP tool list using an inverted index pattern includes: Obtain the actual deployment information of each brand of equipment, and convert each actual deployment information into a structured configuration file through a preset device type template; Load each structured configuration file, automatically generate an MCP tool description for each brand of device, and build an inverted index between device alias and device ID in each MCP tool description; Each brand of device is registered in the MCP tool list based on its MCP tool description and inverted index.

[0007] Optionally, the step of converting each actual deployment information into a structured configuration file using a preset device type template includes: For each actual deployment information, extract the device type, device ID, and protocol type from the actual deployment information, and obtain the protocol routing metadata of the protocol type; Determine the standard tool name corresponding to the device type from the preset device type template, and concatenate the device ID and the standard tool name to obtain the MCP tool name; Based on the device alias, location information, and device capabilities in the actual deployment information, tool description text is generated, and input parameter constraints are generated based on the device capabilities; A structured configuration file is generated based on the protocol routing metadata, the MCP tool name, the tool description text, and the input parameter constraints.

[0008] Optionally, the step of semantically recognizing the control commands based on the MCP tool list and generating a tool invocation request based on the recognition result includes: Extract the device alias and operation behavior from the control command, and index the device ID corresponding to the device alias from the MCP tool list; Extract the MCP tool description of the device ID, and generate a tool invocation request based on the MCP tool description and the operation behavior.

[0009] Optionally, the parsing obtains the target device and semantic parameters in the tool invocation request, including: In response to the tool invocation request, determine the MCP tool description and operation behavior in the tool invocation request; The MCP tool name described in the MCP tool description is parsed to obtain the device ID, and the target device corresponding to the device ID is searched from the device instance library; Based on the MCP tool description, semantic recognition is performed on the operation behavior to obtain semantic parameters.

[0010] Optionally, the security verification of the target device and the semantic parameters includes: Determine the target device's operating permissions and current operating mode, and parse the input parameters and input parameter constraints of the semantic parameters; Determine whether the operation permission includes the user, whether the current operation mode is operable, and whether the input parameters are within the range of the input parameter constraints; If so, then the security verification of the target device and the semantic parameters is confirmed to be successful; If not, then the security verification of the target device and the semantic parameters is confirmed to have failed.

[0011] Optionally, controlling the target device to perform operations according to the semantic parameters and returning a response message after execution includes: The operation steps for the target device are generated based on the semantic parameters; the operation steps include single steps and multiple steps. When the operation step is a single step, the target device is controlled to execute the operation step, and a response message is generated based on the execution result; When the operation steps are multiple, the target device is controlled to execute each step in a preset execution order. If any step fails, the process is rolled back to the step before the failed step and the operation is re-executed until all steps are successfully executed or the number of failures reaches the upper limit. Then, a response message is generated based on the execution result.

[0012] This application also provides a unified control device for heterogeneous devices, including: The device registration module is used to identify multiple brands of devices that need to be controlled, and uses an inverted index pattern to register each brand of device into the MCP tool list for use by the large language model. The request generation module is used to perform semantic recognition on the control command input by the user based on the MCP tool list when the large language model receives the control command input by the user, and generate a tool call request based on the recognition result. The message generation module is used to parse the target device and semantic parameters in the tool call request, to perform security verification on the target device and the semantic parameters, and to generate a request message containing the target device and the semantic parameters when the verification is successful. The operation execution module is used to obtain the protocol route of the target device based on a preset multi-protocol mapping table, and send the request message to the target device through the protocol route, so as to control the target device to perform operation execution according to the semantic parameters, and return a response message after the execution is completed; The result output module is used to generate natural language description text corresponding to the response message, and output the natural language description text through the large language model.

[0013] This application also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the heterogeneous device unified control method as described in any of the above embodiments.

[0014] This application also provides a computer device, including: one or more processors, and memory; The memory stores computer-readable instructions, which, when executed by the one or more processors, perform the steps of the heterogeneous device unified control method as described in any of the above embodiments.

[0015] As can be seen from the above technical solutions, the embodiments of this application have the following advantages: The heterogeneous device unified control method, device, storage medium and computer equipment provided in this application can, after identifying multiple brands of devices that need to be controlled, register each brand of device in the MCP tool list using an inverted index pattern for use by the large language model, thus breaking down the access barriers of heterogeneous devices of different brands. After receiving control commands input by the user, the large language model can perform semantic recognition on the control commands from the tool list, and then generate tool call requests based on the recognition results to unify the format differences of various control commands. Subsequently, it can parse the target device and semantic parameters in the tool call request, and generate a request message after the target device and semantic parameters are verified. This can filter out risky commands such as misoperation and unauthorized control, avoiding abnormal problems such as device misstart and parameter mismatch. At the same time, it can match the protocol route of the target device based on a preset multi-protocol mapping table to achieve automatic adaptation of heterogeneous communication protocols, and accurately send the request message to the target device through the protocol route, so that the target device can perform operations according to the semantic parameters and return a response message after the execution is completed. Finally, it can generate natural language description text corresponding to the response message, and output the natural language description text through the large language model to realize the closed loop of automated control of different brands of devices. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart illustrating a unified control method for heterogeneous devices provided in an embodiment of this application; Figure 2 A flowchart illustrating a branded device registration process provided for an embodiment of this application; Figure 3 A schematic diagram of a unified control device for heterogeneous devices provided in this application embodiment; Figure 4 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] Currently, the mainstream approach to device control is for different brands to build their own dedicated control platforms. Users who want to operate devices of a specific brand can only do so through their respective dedicated control platforms. To achieve cross-brand collaborative control, users need to switch between multiple control applications, resulting in low control efficiency. Even with the rapid development of large language model technology in recent years, and natural language-based interactive control gradually becoming a popular direction for smart device control, the lack of unified access standards for heterogeneous devices from different brands makes it difficult for large language models to directly achieve unified invocation and control of devices from all brands.

[0020] Based on this, this application proposes the following technical solution, as detailed below: In one embodiment, such as Figure 1 As shown, Figure 1 This is a flowchart illustrating a unified control method for heterogeneous devices provided in an embodiment of this application. The unified control method for heterogeneous devices provided in this application specifically includes the following: S110: Identify multiple brands of devices that need to be controlled, and register each brand of device in the MCP tool list using an inverted index pattern for use by the large language model.

[0021] In this step, after identifying multiple brands of devices that need to be controlled, the semantic control system can use an inverted index pattern to register each brand of device into the MCP (Minecraft Coder Pack) tool list for use by the large language model, breaking down the access barriers of heterogeneous devices from different brands.

[0022] Specifically, after identifying multiple brand devices requiring unified control in the target scenario, this application allows the semantic control system to uniformly register and manage these devices, providing foundational data for the invocation of the large language model. During registration and management, the semantic control system can generate a corresponding MCP tool for each brand device, thus forming an MCP tool list. Here, the MCP tool refers to a standardized device invocation interface adapted to the large language model's invocation specification. Therefore, through the MCP tool list, the large language model can directly invoke various brand devices according to a unified specification, without needing to adapt different invocation logic for devices of different brands and protocols.

[0023] Furthermore, the semantic control system can use an inverted index pattern to build device registration relationships. This inverted index refers to a data organization method for fast retrieval, such as looking up the corresponding object in reverse based on keywords such as attributes and names. Therefore, after receiving a user's natural language command, the large language model can directly locate the corresponding target device based on the keywords mentioned in the command, thereby saving the process of traversing the entire list for matching and improving the efficiency of device recognition and retrieval.

[0024] Furthermore, if new branded devices are needed in the future, the semantic control system can directly add and register the devices in the existing MCP tool list without modifying the overall structure of the existing MCP tool list, and it will not affect the normal use of registered devices, thus demonstrating good scalability.

[0025] S120: When receiving control commands input by the user through the large language model, semantic recognition of the control commands is performed based on the MCP tool list, and a tool call request is generated based on the recognition results.

[0026] In this step, based on the MCP tool list in step S110, after the large language model receives the control command input by the user, it can perform semantic recognition on the control command from the MCP tool list, and then generate a tool call request based on the recognition result to unify the format differences of various control commands.

[0027] Specifically, after a user inputs a control command into the large language model, the large language model can first perform semantic parsing to extract semantic elements such as the control object, control intent, and control target. Then, based on the registered MCP tool list, it can perform tool matching on the semantic parsing results, retrieve the device invocation tool corresponding to the control object and control intent from the tool list, and convert the semantic parsing results into a standardized tool invocation request based on the device invocation tool.

[0028] Understandably, through the MCP tool list, the large language model can encapsulate the control requirements of different devices in a unified format, thereby shielding the interface differences between different brands of devices, achieving the unification of control command formats for different protocols and brands of devices, and thus improving the compatibility and control efficiency after heterogeneous devices are connected.

[0029] Furthermore, when the semantic control system detects that the user-input control command belongs to a batch control command for devices, it can parse the multiple target devices that need to be controlled simultaneously and the corresponding control operations for each target device in the control command, and generate tool call requests one by one based on the MCP tool list. Subsequently, the semantic control system can submit multiple tool call requests to a unified cross-protocol request queue, where the scheduler centrally manages and schedules each tool call request. The scheduler can allocate the requests to the corresponding protocol-level scheduling queue according to the protocol type of each tool call request, thereby realizing parallel processing of each request.

[0030] During the scheduling process, the scheduler can further determine the execution order based on the priority of tool call requests. Priorities can include high-priority requests for alarm linkage and safety operations, ordinary control requests triggered by the user, and low-priority requests such as periodic data collection and background synchronization. When the scheduler detects idle concurrent resources in the corresponding protocol queue, it can retrieve the highest-priority request from that queue and determine if the current time has exceeded the deadline for that request. If the deadline has not been exceeded, the scheduler can call the corresponding protocol adapter to convert the tool call request into a control message in the target protocol format and send it asynchronously to the target device for execution. If the deadline has been exceeded, the scheduler can directly return a timeout response to prevent invalid requests from continuing to occupy communication resources.

[0031] S130: Parse the target device and semantic parameters in the tool call request, perform security verification on the target device and semantic parameters, and generate a request message containing the target device and semantic parameters when the verification passes.

[0032] In this step, after the tool call request is generated in step S120, the semantic control system can parse the target device and semantic parameters in the tool call request, and generate a request message after the target device and semantic parameters are verified. This can filter out risky instructions such as misoperation and unauthorized control, and avoid abnormal problems such as device malfunction or parameter mismatch.

[0033] Specifically, after receiving a tool invocation request, the semantic control system can parse it to obtain the target device and semantic parameters. The target device refers to the branded device to which the user's control command specifically targets the operation to be performed, while the semantic parameters include control information such as the control parameters and thresholds required by the user for the target device. After obtaining the target device and semantic parameters, the semantic control system can initiate a security verification process. This process combines pre-set access control rules, device operating status rules, and parameter range rules to perform multi-dimensional verification. Only when the verification result meets all security rule requirements will a standardized request message containing the target device identifier and semantic parameters be generated and proceed to the subsequent forwarding and execution stage.

[0034] More specifically, the semantic control system can generate request messages through a private protocol adapter driver layer. This private protocol adapter driver layer can provide a standard "hardware abstraction layer" interface for multiple industrial protocols, isolating the differences in underlying communication. That is, it specifies a unified standard function interface for connection, execution of instructions, and disconnection for all protocol adapters, ensuring that the upper-layer scheduling module does not need to care about the specific protocol details.

[0035] For example, for the Modbus protocol, the mapping table predefines the mapping relationship between semantic operations and function codes, such as mapping temperature reading to reading holding register function codes, and configuring a register address mapping table for the device model, specifying the register address, data type, scaling factor, and read / write attributes corresponding to each parameter; for the BACnet protocol, the mapping table predefines the mapping between BACnet object types, such as analog input, analog output, binary output, and semantic data types, and determines the standard attribute ID corresponding to the read attributes; for the ONVIF protocol, semantic operations are mapped to specific ONVIF service interfaces, such as mapping PTZ rotation to the absolute motion service of PTZ, and a fixed structure template for generating corresponding SOAP request messages is generated.

[0036] Furthermore, when the verification fails, the semantic control system will directly intercept the tool call request and generate corresponding security prompts, which will be fed back to the user in natural language by the large language model to prevent abnormal commands from triggering abnormal device operation.

[0037] S140: Based on the preset multi-protocol mapping table, the protocol route of the target device is obtained, and the request message is sent to the target device through the protocol route to control the target device to perform operations according to semantic parameters, and return a response message after the execution is completed.

[0038] In this step, after the request message is generated in step S130, the semantic control system can match the protocol route of the target device based on the preset multi-protocol mapping table, realize the automatic adaptation of heterogeneous communication protocols, and accurately send the request message to the target device through the protocol route, so that the target device can perform operations according to the semantic parameters and return a response message after the execution is completed.

[0039] The multi-protocol mapping table refers to the storage of mapping relationships between different brands of devices, including communication protocol types, protocol conversion rules, data encapsulation formats, and device access addresses. For example, for smart home appliances using the Wi-Fi communication protocol, smart sensing devices using the ZigBee protocol, and industrial control devices using the Modbus protocol, an association between device identifiers and corresponding protocol routes can be established in the multi-protocol mapping table.

[0040] Specifically, after generating the tool invocation request, the semantic control system can further query a pre-built multi-protocol mapping table based on the target device identifier, device type, and communication requirements in the request to determine the protocol route corresponding to the target device. This allows for the establishment of a corresponding protocol routing channel, through which the adapted request message is directly forwarded to the target device. Upon receiving the adapted request message, the target device can complete the corresponding operation based on the semantic parameters carried in the message and encapsulate the operation execution result into a response message according to the original protocol path, sending it back to the semantic control system.

[0041] For example, when a user inputs the control command "Turn on the lighting equipment in the smart warehouse," the semantic control system, after semantic parsing and device matching, can generate a request message containing the lighting equipment identifier and activation parameters. Subsequently, the semantic control system can query a multi-protocol mapping table based on the target lighting equipment's device identifier to determine that the device uses the ZigBee communication protocol, and obtain the corresponding protocol route. This route can then be used to convert the request message into a ZigBee protocol format control command and send it to the target lighting equipment, causing the target equipment to perform the activation operation according to the control parameters. After the device completes the operation, it can generate a response message containing the execution status, current device status, and any exception information, and return it to the semantic control system via the corresponding protocol route.

[0042] Understandably, by matching the protocol routes obtained through the multi-protocol mapping table, this application can automatically convert request messages of a unified format into private protocol messages that the target device can recognize and parse, without having to manually configure protocol conversion logic for each type of device. This ensures compatibility with the private communication protocols of various heterogeneous devices from different brands, reducing the transformation cost and access threshold for unified access of heterogeneous devices.

[0043] S150: Generate natural language description text corresponding to the response message, and output the natural language description text through a large language model.

[0044] In this embodiment, after the response message is returned in step S130, the semantic control system can generate natural language description text corresponding to the response message, and output the natural language description text through the large language model to realize the closed loop of automated control for different brands of equipment.

[0045] Specifically, after receiving the response message from the target device, the semantic control system can call the corresponding reverse translation function to parse the response message according to the protocol type of the target device, extract structured results such as execution status, current device parameters, and error messages, and then generate natural language description text that conforms to the user's reading habits based on these structured results, and send it to the large language model so that the large language model can output the text so that the user can intuitively obtain the device execution results and current status.

[0046] In the above embodiments, after identifying multiple brand devices that need to be controlled, an inverted index pattern can be used to register each brand device in the MCP tool list for use by the large language model, breaking down the access barriers between heterogeneous devices of different brands. After receiving the control commands input by the user, the large language model can perform semantic recognition on the control commands from the tool list, and then generate a tool call request based on the recognition results to unify the format differences of various control commands. Subsequently, it can parse the target device and semantic parameters in the tool call request, and generate a request message after the target device and semantic parameters have passed verification. This can filter out risky commands such as misoperation and unauthorized control, avoiding abnormal problems such as device misstart and parameter mismatch. Simultaneously, it can match the protocol route of the target device based on a preset multi-protocol mapping table, achieving automatic adaptation of heterogeneous communication protocols. The request message is then accurately sent to the target device through this protocol route, enabling the target device to perform operations according to the semantic parameters and return a response message after execution. Finally, it can generate a natural language description text corresponding to the response message and output the natural language description text through the large language model, realizing an automated control closed loop for different brand devices.

[0047] In one embodiment, such as Figure 2 As shown, Figure 2A flowchart illustrating a branded device registration process provided for an embodiment of this application; Figure 2 In step S110, the step of registering each brand of device to the MCP tool list using an inverted index pattern may include: S111: Obtain the actual deployment information of each brand of equipment, and convert each actual deployment information into a structured configuration file through the preset device type template.

[0048] S112: Load each structured configuration file, automatically generate an MCP tool description for each brand of device, and build an inverted index between the device alias and device ID in each MCP tool description.

[0049] S113: Register each brand of device to the MCP tool list based on the MCP tool description and inverted index of each brand of device.

[0050] In this embodiment, during device registration, the semantic control system can first obtain the actual deployment information of each brand of device, and then convert each actual deployment information into a structured configuration file through a preset device type template. Then, it loads each structured configuration file, automatically generates an MCP tool description for each brand of device, and constructs an inverted index between the device alias and device ID in each MCP tool description. In this way, each brand of device can be registered to the MCP tool list according to the MCP tool description and inverted index of each brand of device.

[0051] The device type template refers to a standardized semantic description model pre-established for a class of devices with the same device type or similar functional characteristics. It can abstract and encapsulate common control capabilities among devices of different brands and models, forming a standardized tool description framework independent of specific device brands. The device type template in this application stores the correspondence between multiple device types and standard tool sets. These standard tool sets are used to define the general control capabilities supported by the device type. For example, when the device type is a camera, its corresponding standard tool set includes image stream acquisition, PTZ control, snapshot capture, and video recording control capabilities; when the device type is a temperature control device, its corresponding standard tool set includes temperature setting, temperature query, and mode switching capabilities.

[0052] Specifically, the semantic control system can first collect data on the brand devices to be connected to obtain their actual deployment information, including device type, device ID, device alias, deployment location, protocol parameters, etc. Then, the semantic control system can match the corresponding device type template from the device type template library based on the device type in the actual deployment information, and combine it with the standard tool set corresponding to the device type template to perform field mapping on the actual deployment information, convert the specific device parameters of the target device into the corresponding standard fields, and then generate a structured configuration file corresponding to the target device.

[0053] Subsequently, the semantic control system can load the generated structured configuration file and automatically generate the corresponding MCP tool description based on the standard tool set in the structured configuration file. This MCP tool description refers to a natural language interface description for a large language model, including the tool name, function description, parameter names, parameter requirements, and return definitions, allowing the large language model to accurately understand the control capabilities and calling rules of the brand's equipment. After constructing the MCP tool description, the semantic control system can also extract the device alias and device ID information from the tool description, establishing a mapping relationship between them based on an inverted index structure. Later, after the large language model parses the user control commands to obtain the device alias, it can quickly retrieve the corresponding device ID through this inverted index. Since the device alias includes information such as user-preferred names, device function names, and custom names, this application does not need to perform fuzzy traversal of the entire tool list, effectively improving the retrieval efficiency and matching accuracy of tool matching. Finally, the semantic control system stores the MCP tool description and inverted index into the MCP tool list, thus completing the registration process for a single brand of equipment.

[0054] In one embodiment, the process of converting each actual deployment information into a structured configuration file using a preset device type template in step S111 may include: S1111: For each actual deployment information, extract the device type, device ID, and protocol type from the actual deployment information, and obtain the protocol routing metadata of the protocol type.

[0055] S1112: Determine the standard tool name corresponding to the device type from the preset device type template, and concatenate the device ID and the standard tool name to obtain the MCP tool name.

[0056] S1113: Generate tool description text based on the device alias, location information, and device capabilities in the actual deployment information, and generate input parameter constraints based on the device capabilities.

[0057] S1114: Generate a structured configuration file based on protocol routing metadata, MCP tool name, tool description text, and input parameter constraints.

[0058] In this embodiment, for each actual deployment information, the semantic control system can extract the device type, device ID, and protocol type from the actual deployment information, obtain the protocol routing metadata of the protocol type, then determine the standard tool name corresponding to the device type from the preset device type template, and concatenate the device ID and the standard tool name to obtain the MCP tool name; then, the semantic control system can generate tool description text based on the device alias, location information, and device capabilities in the actual deployment information, and generate input parameter constraints based on the device capabilities, and generate a structured configuration file based on the protocol routing metadata, MCP tool name, tool description text, and input parameter constraints.

[0059] Specifically, the semantic control system can extract key fields such as device type, device ID, and protocol type from the actual deployment information, and obtain protocol routing metadata corresponding to the protocol type through a protocol routing mapping table to support protocol conversion and message forwarding of subsequent control commands. Subsequently, the semantic control system can determine the standard tool set corresponding to the device type from the device type template, and obtain the standard tool name for each tool in the set. Here, the device ID is used as the device instance identifier, and the standard tool name is used as the device capability identifier. Combining the two forms a unique MCP tool name, thus avoiding tool name conflicts when multiple similar devices are registered. Simultaneously, the semantic control system can generate tool description text based on the device alias, location information, and device capability information in the actual deployment information. Since device capabilities can include the control functions supported by the device, adjustable parameters, and status query capabilities, the semantic control system can also generate corresponding input parameter constraints based on the device capability information to limit the allowed parameter types, parameter ranges, and parameter formats when calling the MCP tool.

[0060] For example, for the set_temperature tool of a temperature control device, the semantic control system can generate input constraints with a temperature parameter range of 16°C to 30°C; for the set_brightness tool of a lighting control device, the semantic control system can generate input constraints with a brightness parameter range of 0 to 100.

[0061] Finally, the semantic control system can combine the acquired protocol routing metadata, the generated MCP tool name, the tool description text, and the input parameter constraints to generate a structured configuration file corresponding to the target device, which serves as a standardized storage carrier for device registration information.

[0062] In one embodiment, step S120, which involves semantically recognizing control commands based on the MCP tool list and generating a tool invocation request based on the recognition results, may include: S121: Extract the device alias and operation behavior from the control command, and index the device ID corresponding to the device alias from the MCP tool list.

[0063] S122: Extract the MCP tool description of the device ID, and generate a tool invocation request based on the MCP tool description and operation behavior.

[0064] In this embodiment, when the semantic control system receives a control command input by the user, it can first extract the device alias and operation behavior from the control command, and then index the device ID corresponding to the device alias from the MCP tool list. Then, it can extract the MCP tool description of the device ID and generate a tool call request based on the MCP tool description and operation behavior.

[0065] The device alias can be used to identify the target device that the user wants to control, and the operation behavior can be used to determine the control actions and parameters that the target device needs to perform. For example, when a user inputs "turn on the conference room lights and adjust the brightness to 80%", the semantic control system can resolve that the device alias is "conference room lights", and the operation behavior includes "turn on" and "adjust the brightness to 80%".

[0066] Specifically, the large language model can perform index matching in the MCP tool list based on the extracted device aliases to determine the device ID corresponding to the device aliases. Then, it can extract the MCP tool descriptions associated with the device ID from the MCP tool list to determine the control capabilities supported by the target device. Next, it can match the user's input operation with the standard tool capabilities in the MCP tool descriptions and generate a tool invocation request based on the matched MCP tool descriptions and user operation. For example, when the device type corresponding to the target device ID is a temperature control device, the large language model can determine from the device's MCP tool descriptions that it supports tool capabilities such as set_temperature, get_temperature, and set_mode, and match the corresponding temperature adjustment tool based on the user's input of "lower the temperature".

[0067] In one embodiment, the process of parsing the target device and semantic parameters in the tool invocation request in step S130 may include: S131: Respond to the tool invocation request and determine the MCP tool description and operation behavior in the tool invocation request.

[0068] S132: Parse the MCP tool name described in the MCP tool description to obtain the device ID, and search for the target device corresponding to the device ID from the device instance library.

[0069] S133: Based on the MCP tool description, perform semantic recognition on the operation behavior to obtain semantic parameters.

[0070] In this embodiment, after receiving a tool call request, the semantic control system can parse the MCP tool description and operation behavior in the tool call request, then parse the MCP tool name in the MCP tool description to obtain the device ID, and search for the target device corresponding to the device ID from the device instance library.

[0071] The device instance library is mainly used to store information on registered branded device instances, such as device ID, device brand, device model, device type, deployment location, communication protocol, and device access address. Through the mapping relationship between device ID and device instance information, the semantic control system can quickly locate the target device that actually needs to be controlled.

[0072] Specifically, after receiving a tool invocation request, the semantic control system first parses the request to obtain the MCP tool description and operational behavior contained within. Since the MCP tool name in the MCP tool description is obtained by concatenating the device ID with the standard tool name, the semantic control system can obtain the device ID by parsing the MCP tool name. Based on this device ID, the semantic control system can access a pre-built device instance library and perform an index query based on the target device ID to find the target device corresponding to that device ID. Subsequently, the semantic control system can combine the input parameter constraints defined in the MCP tool description to perform semantic extraction and format conversion on the parameter information contained in the operational behavior, ultimately obtaining standardized semantic parameters. This eliminates the differences in expression when different users input, providing a standardized input basis for subsequent message generation.

[0073] In one embodiment, the process of performing security verification on the target device and semantic parameters in step S130 may include: S134: Determine the target device's operating permissions and current operating mode, as well as the input parameters and input parameter constraints of the parsed semantic parameters.

[0074] S135: Determine whether the operation permission includes the user, the current operation mode is operable, and the input parameters are within the range of input parameter constraints.

[0075] S136: If so, then the security verification of the target device and semantic parameters is confirmed to be successful.

[0076] S137: If not, then the security verification of the target device and semantic parameters fails.

[0077] In this embodiment, after obtaining the target device and semantic parameters, the semantic control system can determine the operation permissions and current operation mode of the target device, as well as the input parameters and input parameter constraints of the semantic parameters obtained by parsing. Then, it can determine whether the operation permissions include the user, the current operation mode is operable, and the input parameters are within the range of the input parameter constraints. If so, it can confirm that the security verification of the target device and semantic parameters has passed; otherwise, it can confirm that the security verification of the target device and semantic parameters has failed.

[0078] Specifically, the semantic control system can compare the user information that initiates the control request with the target device's preset whitelist of operating permissions to confirm whether the user is qualified to operate the corresponding device; at the same time, it can read the target device's current operating status to determine whether the device is in an operable mode that allows remote control, and exclude abnormal requests from devices in uncontrollable states such as maintenance or offline; finally, it can compare the extracted input parameters with the predefined input parameter constraints in the MCP tool description to check whether the parameter type, parameter range, and parameter format meet the requirements.

[0079] Understandably, through multi-dimensional security verification, this application can effectively filter out unauthorized operations, invalid control while the device is offline, and parameter requests exceeding reasonable limits. This prevents abnormal issues such as accidental device startup and parameter mismatch from the source, improving the operational stability and security of the entire control system. Requests that fail verification will be directly blocked, and corresponding security prompts will be generated and returned to the user, informing them of the specific reasons for the verification failure, allowing the user to adjust control commands and re-initiate the request.

[0080] In one embodiment, the process of controlling the target device to perform operations according to semantic parameters and returning a response message after the execution is completed in step S140 may include: S141: Generate the operation steps for the target device based on semantic parameters; the operation steps include single steps and multiple steps.

[0081] S142: When the operation step is a single step, control the target device to execute the operation step and generate a response message based on the execution result.

[0082] S143: When the operation steps are multiple, the target device is controlled to execute each step in a preset execution order. If any step fails, the operation is rolled back to the step before the failed step and the operation is re-executed until all steps are executed successfully or the number of failures reaches the upper limit. Then, a response message is generated based on the execution result.

[0083] In this embodiment, the target device can confirm the operation steps it needs to perform through semantic parameters. These operation steps can include single steps and multiple steps. When the operation step is a single step, the target device is controlled to execute the operation step and generate a response message based on the execution result. When the operation step is a multiple step, the target device is controlled to execute each step in a preset execution order. If any step fails, the device rolls back to the step before the failed step and re-executes the operation until all steps are successfully executed or the number of failures reaches the upper limit. Then, a response message is generated based on the execution result.

[0084] As is understandable, semantic parameters include information such as the operation target, control actions, execution conditions, parameter values, and execution order. Therefore, after receiving the request message, the target device can parse the semantic parameters and determine whether to use a single-step execution mode or a multi-step execution mode based on the parsing result.

[0085] Specifically, when the operation step is a single step, the target device can directly execute the corresponding operation based on the semantic parameters and collect device status information in real time during the execution process. After completing the operation, the target device can generate a response message based on the actual execution result. For example, when the target device is an air conditioner and the semantic parameter indicates that the temperature is adjusted to 26 degrees Celsius, the target device can determine that it only contains one operation step: "temperature adjustment." In this case, it can directly execute the temperature adjustment operation and generate a response message containing information such as "air conditioner successfully adjusted" and "current temperature is 26 degrees Celsius" after the adjustment is completed.

[0086] When an operation involves multiple steps, the target device can orchestrate these steps according to the logical order of execution and execute them sequentially. For example, if the target device is an air conditioner, and the semantic parameters indicate that it is started, the temperature is adjusted to 24°C, and then the fresh air system is turned on, the target device can determine that these three sequentially executed steps only include "air conditioner start," "temperature adjustment," and "fresh air system turn on." In this case, it can execute these steps sequentially according to a preset logical order and generate a response message containing information such as "air conditioner started successfully," "air conditioner adjusted successfully," "current temperature is 24 degrees Celsius," and "fresh air system turned on successfully" upon successful execution.

[0087] Furthermore, after each operation step is completed, the target device can check the execution result of that step to determine whether the current step was successful. If the current step is successful, the target device can continue to execute the next operation step; if any operation step fails, the target device can record the execution status of the failed step and roll back to the previous step according to a preset rollback strategy to re-execute the failed step, thereby improving the success rate of multi-step task execution. In addition, when re-executing a failed step, the target device can also record the current number of failures and determine whether the number of failures has reached a preset upper limit; if the number of failures reaches the preset upper limit and the task is still not successful, the target device can stop the current multi-step task execution and generate a response message containing the reason for failure, the failed step, and device status information based on the current execution status, avoiding resource waste caused by repeated abnormal execution.

[0088] The following describes the heterogeneous device unified control device provided in the embodiments of this application. The heterogeneous device unified control device described below and the heterogeneous device unified control method described above can be referred to in correspondence.

[0089] In one embodiment, such as Figure 3 As shown, Figure 3 This application provides a schematic diagram of a unified control device for heterogeneous devices, as shown in an embodiment of the present application. The application also provides a unified control device for heterogeneous devices, including a device registration module 210, a request generation module 220, a message generation module 230, an operation execution module 240, and a result output module 250, specifically comprising the following: The device registration module 210 is used to identify multiple brands of devices that need to be controlled, and to register each brand of device to the MCP tool list using an inverted index pattern for use by the large language model. The request generation module 220 is used to perform semantic recognition on the control command input by the user based on the MCP tool list when the large language model receives the control command input by the user, and generate a tool call request based on the recognition result. The message generation module 230 is used to parse the target device and semantic parameters in the tool call request, to perform security verification on the target device and the semantic parameters, and to generate a request message containing the target device and the semantic parameters when the verification is successful. The operation execution module 240 is used to obtain the protocol route of the target device based on a preset multi-protocol mapping table, and send the request message to the target device through the protocol route, so as to control the target device to perform operation according to the semantic parameters, and return a response message after the execution is completed; The result output module 250 is used to generate natural language description text corresponding to the response message, and output the natural language description text through the large language model.

[0090] In the above embodiments, after identifying multiple brand devices that need to be controlled, an inverted index pattern can be used to register each brand device in the MCP tool list for use by the large language model, breaking down the access barriers between heterogeneous devices of different brands. After receiving the control commands input by the user, the large language model can perform semantic recognition on the control commands from the tool list, and then generate a tool call request based on the recognition results to unify the format differences of various control commands. Subsequently, it can parse the target device and semantic parameters in the tool call request, and generate a request message after the target device and semantic parameters have passed verification. This can filter out risky commands such as misoperation and unauthorized control, avoiding abnormal problems such as device misstart and parameter mismatch. Simultaneously, it can match the protocol route of the target device based on a preset multi-protocol mapping table, achieving automatic adaptation of heterogeneous communication protocols. The request message is then accurately sent to the target device through this protocol route, enabling the target device to perform operations according to the semantic parameters and return a response message after execution. Finally, it can generate a natural language description text corresponding to the response message and output the natural language description text through the large language model, realizing an automated control closed loop for different brand devices.

[0091] In one embodiment, the device registration module 210 may include: The file conversion submodule is used to obtain the actual deployment information of each brand of device and convert each actual deployment information into a structured configuration file through a preset device type template.

[0092] The description generation submodule is used to load various structured configuration files, automatically generate MCP tool descriptions for each brand of device, and build an inverted index between device aliases and device IDs in each MCP tool description.

[0093] The device registration submodule is used to register devices of various brands to the MCP tool list based on the MCP tool description and inverted index of each brand.

[0094] In one embodiment, the file conversion submodule may include: The information acquisition unit is used to extract the device type, device ID and protocol type from each actual deployment information, and to obtain the protocol routing metadata of the protocol type.

[0095] The name generation unit is used to determine the standard tool name corresponding to the device type from the preset device type template, and concatenate the device ID and the standard tool name to obtain the MCP tool name.

[0096] The text generation unit is used to generate tool description text based on the device alias, location information, and device capabilities in the actual deployment information, and to generate input parameter constraints based on the device capabilities.

[0097] The file generation unit is used to generate a structured configuration file based on protocol routing metadata, MCP tool name, tool description text, and input parameter constraints.

[0098] In one embodiment, the request generation module 220 may include: The ID index submodule is used to extract device aliases and operation behaviors from control commands, and index the device ID corresponding to the device alias from the MCP tool list.

[0099] The request generation submodule is used to extract the MCP tool description of the device ID and generate a tool invocation request based on the MCP tool description and operation behavior.

[0100] In one embodiment, the message generation module 230 may include: The request and response submodule is used to respond to tool call requests and determine the MCP tool description and operation behavior in the tool call request.

[0101] The device lookup submodule is used to parse the MCP tool name described by the MCP tool to obtain the device ID, and then search for the target device corresponding to the device ID from the device instance library.

[0102] The semantic recognition submodule is used to perform semantic recognition on operation behaviors based on the MCP tool description to obtain semantic parameters.

[0103] In one embodiment, the message generation module 230 may further include: The parameter parsing submodule is used to determine the target device's operating permissions and current operating mode, as well as the input parameters and input parameter constraints for parsing the semantic parameters.

[0104] The condition judgment submodule is used to determine whether the operation permission includes the user, the current operation mode is operable, and the input parameters are within the range of input parameter constraints.

[0105] The first confirmation submodule is used to confirm that the security verification of the target device and semantic parameters has passed if the condition is met.

[0106] The second confirmation submodule is used to confirm that the security verification of the target device and semantic parameters fails if no.

[0107] In one embodiment, the operation execution module 240 may include: The step generation submodule is used to generate operation steps for the target device based on semantic parameters; the operation steps include single steps and multi-step steps.

[0108] The first execution submodule is used to control the target device to execute the operation steps when the operation steps are single steps, and to generate a response message based on the execution result.

[0109] The second execution submodule is used to control the target device to execute each step in a preset execution order when the operation steps are multiple. If any step fails, it rolls back to the step before the failed step and re-executes the operation until all steps are executed successfully or the number of failures reaches the upper limit. Then, it generates a response message based on the execution result.

[0110] In one embodiment, this application also provides a storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the heterogeneous device unified control method as described in any of the above embodiments.

[0111] In one embodiment, this application also provides a computer device storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the heterogeneous device unified control method as described in any of the above embodiments.

[0112] Indicatively, such as Figure 4 As shown, Figure 4 This is a schematic diagram of the internal structure of a computer device 300 provided in an embodiment of this application. The computer device 300 can be provided as a server. (Refer to...) Figure 4 The computer device 300 includes a processing component 302, which further includes one or more processors, and memory resources represented by memory 301 for storing instructions, such as application programs, that can be executed by the processing component 302. The application programs stored in memory 301 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 302 is configured to execute instructions to perform the heterogeneous device unified control method of any of the above embodiments.

[0113] The computer device 300 may also include a power supply component 303 configured to perform power management of the computer device 300, a wired or wireless network interface 304 configured to connect the computer device 300 to a network, and an input / output (I / O) interface 305. The computer device 300 may operate on an operating system stored in memory 301, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or similar.

[0114] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0115] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0116] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0117] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A heterogeneous device unified control method, characterized by, The method includes: Identify multiple brands of devices that need to be controlled, and register each brand of device in the MCP tool list using an inverted index pattern for use by the large language model; When the user inputs a control command through the large language model, the control command is semantically recognized based on the MCP tool list, and a tool invocation request is generated based on the recognition result. The target device and semantic parameters in the tool call request are parsed to perform security verification on the target device and the semantic parameters, and a request message containing the target device and the semantic parameters is generated when the verification passes. The protocol route of the target device is obtained by matching based on a preset multi-protocol mapping table, and the request message is sent to the target device through the protocol route to control the target device to perform operations according to the semantic parameters, and return a response message after the execution is completed; Generate a natural language description text corresponding to the response message, and output the natural language description text through the large language model.

2. The unified control method for heterogeneous devices according to claim 1, characterized in that, The method of registering various brand devices to the MCP tool list using an inverted index pattern includes: Obtain the actual deployment information of each brand of equipment, and convert each actual deployment information into a structured configuration file through a preset device type template; Load each structured configuration file, automatically generate an MCP tool description for each brand of device, and build an inverted index between device alias and device ID in each MCP tool description; Each brand of device is registered in the MCP tool list based on its MCP tool description and inverted index.

3. The unified control method for heterogeneous devices according to claim 2, characterized in that, The process of converting each actual deployment information into a structured configuration file using a preset device type template includes: For each actual deployment information, extract the device type, device ID, and protocol type from the actual deployment information, and obtain the protocol routing metadata of the protocol type; Determine the standard tool name corresponding to the device type from the preset device type template, and concatenate the device ID and the standard tool name to obtain the MCP tool name; Based on the device alias, location information, and device capabilities in the actual deployment information, tool description text is generated, and input parameter constraints are generated based on the device capabilities; A structured configuration file is generated based on the protocol routing metadata, the MCP tool name, the tool description text, and the input parameter constraints.

4. The unified control method for heterogeneous devices according to claim 1, characterized in that, The step of semantically recognizing the control commands based on the MCP tool list and generating a tool invocation request based on the recognition results includes: Extract the device alias and operation behavior from the control command, and index the device ID corresponding to the device alias from the MCP tool list; Extract the MCP tool description of the device ID, and generate a tool invocation request based on the MCP tool description and the operation behavior.

5. The unified control method for heterogeneous devices according to claim 1, characterized in that, The parsing process yields the target device and semantic parameters from the tool invocation request, including: In response to the tool invocation request, determine the MCP tool description and operation behavior in the tool invocation request; The MCP tool name described in the MCP tool description is parsed to obtain the device ID, and the target device corresponding to the device ID is searched from the device instance library; Based on the MCP tool description, semantic recognition is performed on the operation behavior to obtain semantic parameters.

6. The unified control method for heterogeneous devices according to claim 1, characterized in that, The security verification of the target device and the semantic parameters includes: Determine the target device's operating permissions and current operating mode, and parse the input parameters and input parameter constraints of the semantic parameters; Determine whether the operation permission includes the user, whether the current operation mode is operable, and whether the input parameters are within the range of the input parameter constraints; If so, then the security verification of the target device and the semantic parameters is confirmed to be successful; If not, then the security verification of the target device and the semantic parameters is confirmed to have failed.

7. The unified control method for heterogeneous devices according to claim 1, characterized in that, The control of the target device to perform operations according to the semantic parameters and return a response message after the execution is completed includes: The operation steps for the target device are generated based on the semantic parameters; the operation steps include single steps and multiple steps. When the operation step is a single step, the target device is controlled to execute the operation step, and a response message is generated based on the execution result; When the operation steps are multiple, the target device is controlled to execute each step in a preset execution order. If any step fails, the process is rolled back to the step before the failed step and the operation is re-executed until all steps are successfully executed or the number of failures reaches the upper limit. Then, a response message is generated based on the execution result.

8. A unified control device for heterogeneous equipment, characterized in that, include: The device registration module is used to identify multiple brands of devices that need to be controlled, and uses an inverted index pattern to register each brand of device into the MCP tool list for use by the large language model. The request generation module is used to perform semantic recognition on the control command input by the user based on the MCP tool list when the large language model receives the control command input by the user, and generate a tool call request based on the recognition result. The message generation module is used to parse the target device and semantic parameters in the tool call request, to perform security verification on the target device and the semantic parameters, and to generate a request message containing the target device and the semantic parameters when the verification is successful. The operation execution module is used to obtain the protocol route of the target device based on a preset multi-protocol mapping table, and send the request message to the target device through the protocol route, so as to control the target device to perform operation execution according to the semantic parameters, and return a response message after the execution is completed; The result output module is used to generate natural language description text corresponding to the response message, and output the natural language description text through the large language model.

9. A storage medium, characterized in that: The storage medium stores computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the heterogeneous device unified control method as described in any one of claims 1 to 7.

10. A computer device, characterized in that, include: One or more processors, and memory; The memory stores computer-readable instructions, which, when executed by the one or more processors, perform the steps of the heterogeneous device unified control method as described in any one of claims 1 to 7.

Images