Equipment management method and device based on large language model, electronic equipment and storage medium

Abstract

The invention discloses an equipment management method and device based on a large language model, electronic equipment and a storage medium. The device management method provided by the embodiment of the invention mainly comprises the following steps: acquiring a device operation request from user equipment, wherein the device operation request carries a natural language fragment input by a user; calling a large language model to perform intention recognition on the natural language fragment and generating a first function calling signal; adjusting the first operation parameter set according to the natural language segment to generate a second function call signal; calling a service interface by using a second function calling signal so as to execute a specified operation on one or more to-be-managed devices and obtain an operation result; and returning an operation result to the user equipment. The intelligent device can be efficiently managed in a natural language mode.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence application technology, and in particular to a device management method, apparatus, electronic device, and storage medium based on a large language model. Background Technology

[0002] With the widespread adoption of smart devices and the development of remote control technology for smartphones, users' needs for managing their mobile devices are growing. Existing device management systems typically employ traditional command-based interaction methods, requiring users to learn and memorize specific operating instructions, making the process cumbersome and lacking in intuitiveness. Summary of the Invention

[0003] To address the aforementioned technical problems, this disclosure aims to provide a device management method, apparatus, electronic device, and storage medium based on a large language model.

[0004] According to a first aspect of this disclosure, a device management method based on a large language model is provided, the method being applied to a device management server in a device management system, the method comprising:

[0005] Obtain a device operation request from the user device, the device operation request carrying a natural language fragment input by the user;

[0006] A large language model is invoked to perform intent recognition on the natural language segment and generate a first function call signal. The first function call signal includes a first device operation instruction and a first operation parameter set. The first operation parameter set includes a combination of device operation parameters corresponding to the first device operation instruction.

[0007] The first set of operation parameters is adjusted according to the natural language segment to generate a second function call signal. The second function call signal contains a second device operation instruction and a second set of operation parameters. The second device operation instruction is of the same type as the first device operation instruction and each second device operation instruction corresponds to a device to be managed. The second set of operation parameters contains a combination of device operation parameters corresponding to each second device operation instruction.

[0008] The second function call signal is used to invoke the service interface to perform the corresponding device operation instructions for one or more devices to be managed as indicated by the device operation parameters and to obtain the operation results;

[0009] The operation result is returned to the user equipment.

[0010] In some embodiments of the first aspect of this disclosure, adjusting the device operation parameters in the first function call signal according to the natural language fragment includes one or more of the following:

[0011] Verify the device identifier in the first operation parameter set based on the preset identifier set;

[0012] Multi-device segmentation is performed based on a pre-set identifier set and the natural language segments;

[0013] Parameters are expanded by combining contextual data and the natural language fragments;

[0014] Quantitative information is extracted from the natural language segment;

[0015] Mark the missing device operation parameters.

[0016] In some embodiments of the first aspect of this disclosure, the method further includes: invoking the large language model to perform one or more rounds of natural language-based user dialogue and intent recognition based on the missing marker to obtain currently missing device operation parameters; and filling the currently missing device operation parameters into the corresponding device operation parameter combination in the second operation parameter set.

[0017] In some embodiments of the first aspect of this disclosure, the multi-device segmentation based on a preset identifier set and the natural language segment includes: identifying a predetermined delimiter in the natural language segment; extracting multiple device identifiers from the natural language segment according to the predetermined delimiter; matching the multiple device identifiers with the preset identifier set to determine the device identifier of each device to be managed; and splitting a first device operation instruction corresponding to the multiple devices to be managed into multiple second device operation instructions corresponding to the multiple devices to be managed.

[0018] In some embodiments of the first aspect of this disclosure, the device to be managed is a virtual device and / or a physical device.

[0019] In some embodiments of the first aspect of this disclosure, returning the operation result to the user equipment includes: returning the operation result to the user equipment in the form of a natural language streaming response.

[0020] In some embodiments of the first aspect of this disclosure, the method further includes: after calling a large language model to perform intent recognition on the natural language segment, if the first device operation instruction is not obtained, calling the large language model to execute a natural language-based user dialogue to prompt the user.

[0021] According to a second aspect of this disclosure, a device management apparatus based on a large language model is provided, the apparatus being applied to a server of a device management system, the apparatus comprising:

[0022] The acquisition module is used to acquire device operation requests from user devices, wherein the device operation requests carry natural language fragments input by the user;

[0023] The intent recognition module is used to call a large language model to perform intent recognition on the natural language segment and generate a first function call signal. The first function call signal includes a first device operation instruction and a first operation parameter set. The first operation parameter set includes a combination of device operation parameters corresponding to the first device operation instruction.

[0024] The parameter adjustment module is used to adjust the first set of operation parameters according to the natural language fragment to generate a second function call signal. The second function call signal contains a second device operation instruction and a second set of operation parameters. The second device operation instruction is of the same type as the first device operation instruction and each second device operation instruction corresponds to a device to be managed. The second set of operation parameters contains a combination of device operation parameters corresponding to each second device operation instruction.

[0025] The business invocation module is used to invoke the business interface using the second function invocation signal to perform the corresponding device operation instructions for one or more devices to be managed as indicated by the device operation parameters and to obtain the operation results.

[0026] The response module is used to return the operation result to the user equipment.

[0027] According to a third aspect of this disclosure, an electronic device is provided, comprising: one or more processors; and a memory storing a computer program that, when executed by the processor, causes the processor to perform the methods described above.

[0028] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, causes the processor to perform the methods described above.

[0029] Through the embodiments of this disclosure, users can achieve personalized management of diverse devices by using everyday language for human-computer interaction, such as specifying operations for a particular device or different operations for multiple specific devices. Users do not need to learn and memorize specific operation instructions, making the operation simple, efficient, and significantly improving the user experience. Attached Figure Description

[0030] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0031] Figure 1This is an exemplary deployment structure diagram of the device management system to which this disclosure applies.

[0032] Figure 2 This is a schematic diagram of the layered architecture of the device management server in the device management system to which this disclosure applies;

[0033] Figure 3 This is a flowchart illustrating a device management method based on a large language model provided in an exemplary embodiment of this disclosure.

[0034] Figure 4 This is an example diagram of the device operation interface involved in this disclosure;

[0035] Figure 5 This is a schematic diagram of the structure of a device management apparatus based on a large language model provided in another exemplary embodiment of this disclosure.

[0036] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an exemplary embodiment of this disclosure. Detailed Implementation

[0037] Hereinafter, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present disclosure, and not all embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0038] As mentioned earlier, the equipment management system of related technologies usually adopts the traditional command-based interaction method, which requires users to learn and memorize specific operation instructions, making the operation cumbersome and not intuitive.

[0039] Currently, there are two main methods for managing equipment related to this technology:

[0040] 1) Graphical interface-based device management methods: Operating devices by clicking buttons, menus, etc., such as remote control software like TeamViewer and AirDroid. While intuitive, these methods are inefficient, especially when managing multiple devices, as users need to repeat the same steps.

[0041] 2) Command-line based device management methods: These methods control the device by entering commands in a specific format, such as Android Debug Bridge (ADB). These methods require users to memorize specific command syntax, resulting in a high learning curve and a poor user experience.

[0042] In addition, the equipment management methods also have the following problems:

[0043] 1) Lacking natural language understanding capabilities, users need to operate through specific interfaces, buttons, or commands, resulting in an unnatural and unintelligent interaction method.

[0044] 2) It has limited functionality, lacks contextual understanding capabilities, and cannot provide personalized management services for diverse device management scenarios.

[0045] 3) It lacks accurate parameter extraction capabilities and cannot accurately identify the operation object and instructions from the user's natural language expression.

[0046] 4) It typically uses a request-response model, lacks streaming response capabilities, and has a poor user experience.

[0047] 5) Insufficient scalability, making it difficult to flexibly adapt to new business scenarios and functional requirements.

[0048] 6) Insufficient batch device management capabilities; users cannot manage multiple devices simultaneously using simple commands.

[0049] In view of this, the present disclosure provides the following device management method, apparatus, electronic device and computer storage medium based on a large language model to solve the technical problems in related device management methods such as unnatural user interaction, inaccurate parameter extraction, low efficiency of batch device management and insufficient system scalability, so as to achieve the goal of efficiently managing multiple devices through natural language.

[0050] This disclosure is applicable to application scenarios requiring remote management and personalized control of various smart devices. For example, it can be applied to the remote management of various smart devices in smart home scenarios, such as smart curtains, smart lights, smart tables and chairs, smart sofas, and smart monitoring equipment. As another example, it can be applied to the management and control of virtual devices such as cloud phones, cloud computers, and cloud servers. This disclosure does not limit the specific application scenarios or application areas of these embodiments.

[0051] To facilitate understanding, a brief description of the system to which the embodiments of this disclosure apply will be given first.

[0052] Figure 1 A schematic diagram of the structure of a device management system applicable to embodiments of this disclosure is shown. For example... Figure 1 As shown, the device management system applicable to the embodiments of this disclosure may include: user equipment and device management server, and the user equipment and device management server can communicate.

[0053] In practical applications, user devices can be, but are not limited to, electronic devices such as mobile phones, smartwatches, laptops, desktop computers, and client devices. The device management server can be implemented as a physical server or a cluster thereof. User devices and the device management server can communicate via various methods such as Wi-Fi, mobile networks, Bluetooth, IoT, Ethernet, serial communication, and USB.

[0054] The user equipment can respond to the user's operation on the specified interface, receive the user's input description statement, and send a device management request containing the description statement to the device management server. The device management server has a built-in large language model. After receiving the device management request from the user equipment, the device management server executes the device management method provided in the embodiments of this disclosure, and completes device management by parsing the description statement.

[0055] Further, see Figure 1 The device management system may also include a business server, which is responsible for executing specific business operations, while the device management server is responsible for process scheduling and task distribution. For example, the device management server and the business server can be deployed as a single unit, meaning they can be implemented using the same server or its cluster. Alternatively, the device management server and the business server can be deployed separately, meaning they can be implemented using independent servers or their clusters, and the device management server and the business server can communicate with each other. For instance, the device management server can be implemented as a cloud component, while the business server can be implemented as an edge device, and the device management server can invoke the business server through various applicable methods.

[0056] The device management system manages virtual devices and / or physical devices. Virtual devices can be, but are not limited to, virtual cloud phones, virtual machines, and virtual servers. Physical devices can be, but are not limited to, mobile phones, tablets, and smart home devices.

[0057] Figure 2 This diagram illustrates the layered architecture running on the device management server. See also... Figure 2 As shown, the layered architecture of the device management server can include a controller layer, a service layer, a workflow engine layer, a model integration layer, and a storage layer, with adjacent layers capable of interaction. The model integration layer is configured with access addresses for large language models or external large language models, while the storage layer can be used to store the device management server's source data, business data, and / or various types of data.

[0058] In practical applications Figure 1 The device management system shown can be implemented in various forms, including but not limited to: smartphone cloud control platform, cloud phone management system, remote device operating system, virtual device management platform, etc.

[0059] Figure 3 A flowchart illustrating a device management method provided in an embodiment of this disclosure is shown. This device management method can be executed through the aforementioned device management server. See also... Figure 3 Equipment management methods may include:

[0060] Step 301: Obtain a device operation request from the user device, the device operation request carrying a natural language fragment input by the user;

[0061] Step 302: Call the large language model to perform intent recognition on the natural language segment and generate a first function call signal. The first function call signal includes a first device operation instruction and a first operation parameter set. The first operation parameter set includes a combination of device operation parameters corresponding to the first device operation instruction.

[0062] Step 303: Adjust the first operation parameter set according to the natural language fragment to generate a second function call signal. The second function call signal contains a second device operation instruction and a second operation parameter set. The second device operation instruction is of the same type as the first device operation instruction and each second device operation instruction corresponds to a device to be managed. The second operation parameter set contains a combination of device operation parameters corresponding to each second device operation instruction.

[0063] Step 304: Use the second function call signal to call the business interface to execute the corresponding device operation instructions for one or more devices to be managed as indicated by the device operation parameters and obtain the operation results;

[0064] Step 305: Return the operation result to the user device.

[0065] Through the device management method provided in the embodiments of this disclosure, users can achieve personalized management of diverse devices by using everyday language for human-computer interaction, such as specifying operations for a specific device or different operations for multiple specific devices. There is no need to learn and memorize specific operation instructions, making the operation simple and greatly improving the user experience.

[0066] In step 301, the device management server can interact with the user device through various applicable communication protocols to obtain device operation requests from the user device. The message format of the device operation request is related to the communication protocol. This embodiment of the disclosure does not limit the specific form of the device operation request.

[0067] Figure 4 This image shows a sample diagram of the device management interface displayed on a user's device. See also: Figure 4 The device management interface is displayed to the user through the user equipment. The device management interface contains only one dialog box for user interaction. The user clicks the dialog box, enters a natural language fragment in the dialog box, and clicks the send button. The user equipment can then receive the natural language fragment entered by the user and send the aforementioned device operation request to the device management server. The device operation request carries the natural language fragment entered by the user.

[0068] Therefore, the embodiments of this disclosure can perform the device management operations required by the user by providing a device management interface that only contains dialog boxes. The operation is simple and the interface is intuitive.

[0069] The large language model disclosed in this embodiment can be implemented using various applicable deep learning models. For example, the large language model can be implemented using deep learning neural networks, etc. The large language model can use existing publicly available models, or it can be obtained by pre-training a deep learning model in a device management-related manner.

[0070] In step 302, the first device operation instruction belongs to a predefined device operation command system. Specifically, a device operation command system can be predefined, and the large language model can be trained to generate a first device operation instruction belonging to this predefined device operation command system through intent recognition. For example, the device operation command system may include, but is not limited to, the following device operation instructions:

[0071] 1) Basic device operation commands: such as screenshotPhone (device screenshot command), rebootPhone (device restart command), etc.;

[0072] 2) Product operation commands: For example, buyPhone (purchase device command), renewalPhone (renewal device command), etc.;

[0073] 3) Device management commands: such as replacePhone (device replacement command), etc.

[0074] 4) Cloning operation commands: such as createPhoneSnapshot (create device snapshot command), phoneSnapshotReduction (device snapshot restoration command), etc.;

[0075] 5) File operation commands: For example, installUploadFileApp (install upload file application command), etc.

[0076] In step 302, the first function call signal can be a JSON format data packet, and the first function call signal can be used to trigger the parameter adjustment in step 303.

[0077] The first function call signal may contain one or more first device operation instructions and a first operation parameter set. Each first device operation instruction represents a type of predefined device operation command. The first operation parameter set contains a combination of device operation parameters corresponding to each first device operation instruction. This combination of device operation parameters may lack necessary device operation parameters or may contain all necessary device operation parameters required for the operation in step 304.

[0078] For example, the first function call signal can be represented as follows:

[0079] {"command": "rebootPhone"}

[0080] "Params": {

[0081] "phoneName": My device 3

[0082] "phoneNo": "paramNotFound"

[0083] "userPhoneId":"paramNotFound"

[0084] }

[0085] }

[0086] Here, rebootPhone is the first device operation command, a predefined command to restart the device, and "My Device 3" is the first set of operation parameters corresponding to this first device operation command. "paramNotFound" indicates that device operation parameters such as phoneNo and userPhoneId are missing.

[0087] Furthermore, the first function call signal may also include: call logic information, which can be used to indicate whether there are currently any missing device operation parameters, so that in step 303, the corresponding parameter adjustment can be performed according to the call logic information.

[0088] Specifically, in steps 301 to 302, with Figure 2 Taking the system architecture shown as an example, the control layer of the device management server receives device operation requests from user devices, constructs a RunWorkFlowRequest object and passes it to the service layer. Under the trigger of the RunWorkFlowRequest object, the service layer calls the pre-built intelligent semantic processing engine "intentRecognition" in the workflow engine layer. The intelligent semantic processing engine "intentRecognition" executes the pre-built Large Language Model (LLM) node to input the natural language fragments carried in the device operation request into the Large Language Model for processing. The Large Language Model performs user intent recognition on the natural language fragments to generate the first function call signal.

[0089] The RunWorkFlowRequest object is equivalent to a device management task, which can be used to trigger device management processes.

[0090] The intelligent semantic processing engine can be built using a declarative workflow definition method. The workflow configuration of the intelligent semantic processing engine can be built through WorkflowBuilder. This workflow configuration can be used to set various node types such as Large Language Model (LLM) nodes, condition judgment nodes, code execution nodes, and HTTP call nodes.

[0091] LLM nodes can be used to invoke large language models to parse natural language, files, or images to perform tasks such as intent recognition and content generation. LLM nodes can invoke large language models for intent recognition to analyze user-input natural language fragments and output a structured JSON-formatted first function call signal.

[0092] Step 303 may include one or more of the following: 1) verifying device identifiers in the first set of operation parameters based on a preset identifier set; 2) performing multi-device segmentation based on the preset identifier set and natural language segments; 3) expanding parameters by combining context data and the natural language segments; 4) extracting quantity information from the natural language segments; 5) determining the currently missing device operation parameters and generating missing markers.

[0093] The preset identifier set contains device information for each user, including but not limited to one or more device identifiers, device addresses, etc. Each user's device information in the preset identifier set can be stored as an array; for example, a user's device identifier array (i.e., the Phone group) can include device information for all manageable devices of that user. In specific applications, the preset identifier set can be pre-configured or set by the user.

[0094] The device identifiers in the preset identifier set may include, but are not limited to, device name (phoneName), device number (phoneNo), user device ID (userPhoneId), etc.

[0095] Specifically, verifying the device identifier in the first operation parameter set based on the preset identifier set may include: matching the device identifier in the first operation parameter set with the preset identifier set; if the match is successful, it indicates that the device identifier has passed verification and the device identifier can be added to the second operation parameter set; if no match is found, it indicates that the device identifier verification has failed and the device identifier can be marked as missing.

[0096] Specifically, multi-device segmentation based on a preset identification set and a natural language segment may include: identifying a predetermined delimiter in the natural language segment, extracting multiple device identifiers from the natural language segment according to the predetermined delimiter, matching the multiple device identifiers with the preset identification set one by one to determine the device identifiers of each device to be managed, and splitting a first device operation instruction corresponding to the multiple devices to be managed into multiple second device operation instructions corresponding to the multiple devices to be managed one by one.

[0097] The predetermined delimiter may be, but is not limited to, a Chinese comma, a Chinese semicolon, a space, a connector, a conjunction. The conjunction may include, but is not limited to, "and", "with", "and so on". The connector may include, but is not limited to, "&" or other similar connectors. For example, assume that the natural language segment input by the user is "My device 9 and my device 5 new machine at one click". Through the multi-device segmentation in step 303, it can be split into "My device 9" and "My device 5". If the first device operation instructions corresponding to these devices are the same, the first device operation instruction is further split to generate second device operation instructions corresponding to each device, that is, the first device operation instruction is split into a second device operation instruction corresponding to "My device 9" and a second device operation instruction corresponding to "My device 5".

[0098] In a specific application, a multi-device segmentation rule may be pre-configured, and the above-mentioned predetermined delimiter is configured in the multi-device segmentation rule. In step 303, the multi-device segmentation rule may be directly called to perform the above-mentioned multi-device segmentation.

[0099] Through multi-device segmentation, each device that the user needs to operate can be correctly segmented and identified, realizing parallel operation of multiple devices, meeting the user's need to manage multiple intelligent devices simultaneously, and improving the device management efficiency.

[0100] Furthermore, the matching of the device identifier with the preset identification set may be implemented by using a pre-configured matching rule. For example, a matching rule may be pre-configured as an exact matching rule, that is, the device identifier is considered to match successfully only when it is exactly equal to the same type of device identifier recorded in the preset identification set. The matching of the device identifier with the preset identification set is performed by loading this matching rule.

[0101] Extracting quantity information from the natural language segment can obtain quantity information for use in the operation in step 304. For example, in a scenario involving commodity operations, corresponding commodity operations may be performed based on this quantity information.

[0102] Considering that the intent recognition of the large language model in step 302 may not be able to completely extract all device operation parameters from the natural language fragment, to avoid omissions, parameters can be expanded in step 303 by combining context data and natural language fragments to further improve the device operation parameters in the second parameter operation set. For example, if the first operation parameter set extracted in step 302 only contains the device name, then the user device ID, device number, and other device operation parameters corresponding to the device name can be obtained by querying the context data. These device operation parameters can be added to the corresponding device operation parameter combinations in the second operation parameter set, thereby expanding the device operation parameters.

[0103] Contextual data may include, but is not limited to, user interaction data, environmental awareness data, user profile data, and external databases. Contextual data can be stored externally or in a local database. It should be noted that all data potentially related to user privacy disclosed herein was obtained with the user's authorization.

[0104] Before step 303, the necessary parameter types of each device operation instruction can be predefined while predefining the device operation command system. In step 303, the combination of device operation parameters corresponding to a certain device operation instruction in the second operation parameter set can be compared with the necessary parameter types of the device operation instruction to determine whether the combination of device operation parameters corresponding to the device operation instruction in the second operation parameter set contains the necessary parameters of the device operation instruction. If yes, it indicates that the parameters are complete and without missing ones. If no, a missing mark can be made according to the currently missing necessary parameter type.

[0105] For example, a predefined flag such as "paramNotFound" can be used to mark missing parameters. The missing parameter flag can also record the types of necessary parameters that are currently missing, so that they can be retrieved later through user dialogue.

[0106] Furthermore, after step 303, the method of this embodiment may further include: invoking a large language model to perform one or more rounds of user dialogue and intent recognition based on natural language according to the missing marker to obtain the currently missing device operation parameters; and filling the currently missing device operation parameters into the corresponding device operation parameter combination in the second operation parameter set. Thus, necessary parameters can be supplemented in real time through natural language interaction, which is simple to operate, highly efficient, and provides a good user experience.

[0107] Here, the specific implementation process of calling the large language model to perform one or more rounds of user dialogue and intent recognition based on natural language according to the missing marker to obtain the currently missing device operation parameters is basically the same as the process of steps 301 to 303 mentioned above. The difference is that in the user dialogue, it is necessary to call the large language model to generate natural language prompts. The content of these natural language prompts is related to the necessary parameter types in the missing marker. The content of the natural language prompts can be pre-configured or generated in real time based on pre-configured rules. This disclosure embodiment does not limit the generation method and specific content of the natural language prompts.

[0108] Specifically, with Figure 2 Taking the system structure shown as an example, in step 303, the parameter extension engine "parameterExtraction" pre-built in the workflow engine layer is triggered by the first function call signal generated by the intelligent semantic processing engine to perform the parameter adjustment in step 303. The parameter extension engine can be set with, for example, condition judgment nodes, code execution nodes, HTTP call nodes, etc., and the aforementioned processing in step 303 is performed through these nodes.

[0109] In practical applications, the upper layers of the workflow engine, such as the intelligent semantic processing engine and the parameter expansion engine, can be implemented in Java, while the nodes within the engine can be implemented using a Java-Python integration method. Implementing workflow engines such as the intelligent semantic processing engine and the parameter expansion engine in the workflow engine layer using a Java-Python hybrid programming approach can fully leverage the advantages of both languages ​​and improve the efficiency of device management algorithm implementation.

[0110] When building workflow engines such as intelligent semantic processing engines and parameter expansion engines, different device management scenarios can use independent workflow configurations to achieve business isolation.

[0111] Therefore, the device management method of this disclosure embodiment, which is based on a workflow processing framework, can better support unified processing of different device management scenarios and improve the scalability and maintainability of the system.

[0112] The second function call signal is identical to the first function call signal in other details, except that the second device operation instruction in the second function call signal is associated with a single managed device, and the combinations of device operation parameters in the second operation parameter set are also associated with a single managed device. That is, the second function call signal can include combinations of device operation parameters corresponding to the second device operation instruction for each managed device. For example, the second device operation instructions in the second function call signal can exist in the form of a list containing the second device operation instructions for each managed device, and the corresponding combinations of device operation parameters in the second parameter set can also exist in the form of a list containing combinations of device operation parameters for each managed device.

[0113] For example, the second function call signal can be represented as follows:

[0114] / / First device

[0115] {"command":"screenshotPhone"

[0116] "Params": {

[0117] "phoneName": My device 1

[0118] "phoneNo": 13894567321

[0119] "userPhoneId": 123456

[0120] }

[0121] }

[0122] / / Second device

[0123] {"command":"screenshotPhone"

[0124] "Params": {

[0125] "phoneName": My Device 2

[0126] "phoneNo": 13894567321

[0127] "userPhoneId": 123457

[0128] }

[0129] }

[0130] Here, "screenshotPhone" represents the second device operation command, {MyDevice2, 13894567321, 123457} is the device operation parameter combination for the second device, and {MyDevice1, 13894567321, 123456} is the device operation parameter combination for the first device. {{MyDevice1, 13894567321, 123456}, {MyDevice2, 13894567321, 123457}} constitute the second operation parameter set.

[0131] Step 303 enables precise adjustment of device operation parameters, which can extract the parameters required for operation from natural language fragments and contextual data input by the user, and supports batch operation of multiple devices.

[0132] In step 304, the business interface can be, but is not limited to, a business application programming interface (API). By calling the business API, the device operation instructions and the second set of operation parameters in the second function call signal can be passed to the business server. The business server controls the corresponding managed device to perform the required operation according to the combination of device operation parameters corresponding to each device operation instruction in the second set of operation parameters and returns the operation result.

[0133] In step 304, if multiple devices to be managed are involved, the operations of these devices can be executed in parallel to achieve synchronous management of multiple devices. When multiple devices to be managed are involved, these devices can be of the same or different types; they can be virtual devices such as virtual machines or cloud phones, or physical devices such as mobile phones or smart home devices. The operations of the devices to be managed can include, but are not limited to: taking screenshots of devices, restarting devices, creating device snapshots, installing specified applications, purchasing devices, renewing device subscriptions, and replacing devices. This disclosure does not limit the number, type, or specific operations of the devices to be managed.

[0134] The operation results may include prompts indicating whether the operation was successful or not, data generated by the operation (e.g., screenshots obtained when taking a screenshot of a device under management), and information about the device under management.

[0135] In step 305, the operation result can be returned to the user device in the form of natural language via a streaming response. In specific applications, the streaming response of the operation result can be implemented based on the Spring WebFlux and Reactor frameworks.

[0136] Specifically, the operation result can be converted into a natural language prompt, and this prompt can be returned to the user's device via a streaming response, informing the user of the requested device operation result through natural language interaction. For example, the operation result can be converted into a natural language prompt based on pre-configured rules and / or templates, or it can be converted into a natural language prompt based on a dynamic generation method using deep learning, or it can be converted into a natural language prompt by calling a large language model.

[0137] By using streaming response to return operation results, real-time response to device operations can be achieved, and the initial response time can be shortened to an average of less than 500ms.

[0138] Furthermore, the device management method in this embodiment may further include: after calling the large language model to perform intent recognition on the natural language segment in step 302, if the first device operation instruction is not obtained, then calling the large language model to execute a user dialogue based on natural language to prompt the user. Thus, prompting the user through natural language interaction facilitates user understanding, further reduces the operational difficulty of device management, and improves the user experience.

[0139] Here, the process of calling the large language model to execute a natural language-based user dialogue to prompt the user is the same as the aforementioned user dialogue method, and will not be repeated here.

[0140] The specific implementation process of the device management method of this disclosure embodiment is illustrated below with specific examples.

[0141] Suppose that the natural language fragment entered by the user in the device operation request obtained in step 301 is as follows: "Renew my cloud phone 1 and cloud phone 4 for 1 month of GVIP, create a template for cloud phone 2, and install Honor of Kings on cloud phone 3".

[0142] In step 302, since the predefined device operation command system does not include a device operation instruction for "installing Honor of Kings," but does include a device renewal instruction and a template creation instruction, the first function call signal generated by calling the large language model to perform intent recognition on the above natural language fragment contains two first device operation instructions. The first operation parameter set contains the device operation parameter combinations corresponding to these two first device operation instructions. These two first device operation instructions are: a device renewal instruction and a template creation instruction. The device operation parameter combination corresponding to the device renewal instruction is {phoneName: Cloud Phone 1 and Cloud Phone 4; Time: 1 month; Renewal Type: gvip}, and the device operation parameter combination for the template creation instruction is {phoneName: Cloud Phone 2}.

[0143] In step 303, the first set of operation parameters in the first function call signal is adjusted, that is,

[0144] 1) Based on the preset identifier set, the device identifier "phoneName" (i.e., cloud phone 1, cloud phone 2 and cloud phone 4) in the first operation parameter set are verified, and all are verified successfully;

[0145] 2) The natural language segments are segmented into multiple devices. The renewal device instructions for "Cloud Phone 1" and "Cloud Phone 4" are segmented into renewal device instructions for "Cloud Phone 1" and renewal device instructions for "Cloud Phone 4". Three second device operation instructions and corresponding device operation parameter combinations for these three second device operation instructions are generated. These three device operation parameter combinations form a second operation parameter set. The three second device operation instructions are the renewal device instruction for Cloud Phone 1, the template creation instruction for Cloud Phone 2, and the renewal device instruction for Cloud Phone 4. The corresponding three device operation parameter combinations are: {phoneName: Cloud Phone 1; Time: 1 month; Renewal type: gvip}, {phoneName: Cloud Phone 2}, {phoneName: Cloud Phone 4; Time: 1 month; Renewal type: gvip}.

[0146] 3) Based on contextual data and natural language fragments, parameters were supplemented for each device operation parameter combination in the second operation parameter set. The "phoneNo" and "userPhoneId" device operation parameters were added to each of the three device operation parameter combinations, resulting in the following device operation parameter combinations:

[0147] {phoneName: Cloud Phone 1; phoneNo: 13578695432; userPhoneId: 123467; Time: 1 month; renewal type: gvip};

[0148] {phoneName: Cloud Phone 2; phoneNo: 13843215678; userPhoneId: 123468};

[0149] {phoneName: Cloud Phone 4; phoneNo: 13800310032; userPhoneId: 123470; Time: 1 month; renewal type: gvip}.

[0150] 4) Parameter missing detection was performed on each device operation parameter combination in the second operation parameter set. The detection revealed that the device operation parameter combination corresponding to the "template creation instruction" was missing the following parameters: "template type" and "template address". These were marked as missing using "paramNotFound", i.e., {phoneName: Cloud Phone 2; phoneNo: 13843215678; userPhoneId: 123468; template type: paramNotFound; template address: paramNotFound}. All other device operation parameter combinations were complete.

[0151] 5) Obtain the specific values ​​of "template type" and "template address" through one or more rounds of user dialogue and intent recognition, and fill them into the device operation parameter combination corresponding to "template creation instruction" in the second operation parameter set, to obtain: {phoneName: Cloud Phone 2; phoneNo: 13843215678; userPhoneId: 123468; template type: first file; template address: http...}.

[0152] 6) Generate a second function call signal, which includes the aforementioned three second device operation instructions and their corresponding device operation parameter combinations; or, the second function call signal includes two second device operation instructions with complete parameters (i.e., the renewal device instruction of cloud phone 1 and the renewal device instruction of cloud phone 4) and their corresponding device operation parameter combinations.

[0153] In step 304, the second function call signal is used to call the business API to pass the second device operation instruction and its corresponding device operation parameter combination to the business server. The business server controls cloud phone 1 and cloud phone 4 to perform the operation of renewing the GVIP for one month, controls cloud phone 2 to create the first file at the specified address, and returns the corresponding operation results to the device management server through the business API.

[0154] In step 305, the device management server converts the above operation results into natural language prompts and returns them to the user device through a streaming response, so that the user can intuitively see the natural language prompts describing the operation results through the user device.

[0155] The device management method provided in this disclosure enables personalized device management through natural language interaction. Users do not need to learn specific commands; they can operate the device directly using everyday language, which can significantly improve interaction efficiency and user experience. Tests show that compared to the traditional command-line method, the operation time of the method in this disclosure is reduced by an average of 60%.

[0156] The embodiments disclosed herein accurately identify user intent using a large language model, achieving an accuracy rate of over 95%.

[0157] The embodiments disclosed herein can accurately adjust device operating parameters, and can maintain more than 90% accuracy in extracting device operating parameters even in scenarios with multiple devices and multiple parameters.

[0158] This disclosure supports a single natural language segment (i.e., user instruction) to operate multiple devices simultaneously, improving efficiency by more than 75% in batch operation scenarios.

[0159] Furthermore, the embodiments disclosed herein adopt a unified workflow framework, which facilitates the access of new business scenarios, reduces development costs, and can shorten the development time of new scenarios by more than 50%.

[0160] This disclosure also provides a device management apparatus based on a large language model. Figure 5 The structure of the device management unit 500 is shown. See also Figure 5 The device management apparatus of this disclosure embodiment may include:

[0161] The acquisition module 501 is used to acquire device operation requests from user devices, the device operation requests carrying natural language fragments input by the user;

[0162] The intent recognition module 502 is used to call a large language model to perform intent recognition on natural language segments and generate a first function call signal. The first function call signal includes a first device operation instruction and a first operation parameter set. The first operation parameter set includes a combination of device operation parameters corresponding to the first device operation instruction.

[0163] The parameter adjustment module 503 is used to adjust the first set of operation parameters according to the natural language fragment to generate a second function call signal. The second function call signal contains a second device operation instruction and a second set of operation parameters. The second device operation instruction is of the same type as the first device operation instruction and each second device operation instruction corresponds to a device to be managed. The second set of operation parameters contains a combination of device operation parameters corresponding to each second device operation instruction.

[0164] The business call module 504 is used to call the business interface using the second function call signal to perform the corresponding device operation instructions for one or more devices to be managed as indicated by the device operation parameters and to obtain the operation results.

[0165] The response module 505 is used to return the operation result to the user device.

[0166] Furthermore, the parameter adjustment module 503 can specifically be used to perform one or more of the following to adjust the device operation parameters in the first function call signal according to the natural language fragment:

[0167] 1) Verify the device identifier in the first set of operating parameters based on the preset identifier set;

[0168] 2) Multi-device segmentation based on a pre-set identifier set and natural language segments;

[0169] 3) Expand parameters by combining contextual data and natural language fragments;

[0170] 4) Extract quantitative information from natural language segments;

[0171] 5) Mark the missing equipment operation parameters.

[0172] Furthermore, the response module 505 can be specifically used to: return the operation result to the user device in the form of a natural language through a streaming response.

[0173] Other technical details of the device management device 500 based on the large language model described above can be found in the preceding method section and will not be repeated here. In specific applications, the device management device 500 based on the large language model can be implemented through software, hardware, or a combination thereof. For example, the device management device 500 based on the large language model can be implemented as software running on a device management server.

[0174] Figure 6 A block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0175] like Figure 6 As shown, the electronic device 60 includes one or more processors 61 and a memory 62.

[0176] The processor 61 may be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and may control other components in the electronic device 60 to perform desired functions.

[0177] The memory 62 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 61 may execute the program instructions to implement the dynamic sampling method and / or path planning method of the various embodiments of this disclosure described above, and / or other desired functions.

[0178] In one example, the electronic device 60 may further include an input device 63 and an output device 64, these components being interconnected via a bus system and / or other forms of connection mechanism (not shown). For example, the input device 63 may be a microphone or microphone array. Furthermore, the input device 63 may also include, for example, a keyboard, a mouse, etc. The output device 64 can output various information to the outside. The output device 64 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0179] Of course, for the sake of simplicity, Figure 6 Only some of the components of the electronic device 60 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 60 may include any other suitable components depending on the specific application.

[0180] In practical applications, the aforementioned electronic device 60 can be the aforementioned device management server.

[0181] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps of the device management method based on a large language model according to various embodiments of this disclosure as described in the method section above.

[0182] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0183] Furthermore, embodiments of this disclosure also provide a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform the steps of the device management method based on a large language model according to various embodiments of this disclosure as described in the method section above.

[0184] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0185] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0186] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0187] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.

[0188] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0189] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A device management method based on a large language model, characterized in that, The method is applied to an equipment management server in an equipment management system, and the method includes: Obtain a device operation request from the user device, the device operation request carrying a natural language fragment input by the user; The large language model is invoked to perform intent recognition on the natural language segment and generate a first function call signal. The first function call signal includes a first device operation instruction and a first operation parameter set. The first operation parameter set includes a combination of device operation parameters corresponding to the first device operation instruction. The first set of operation parameters is adjusted according to the natural language segment to generate a second function call signal. The second function call signal contains a second device operation instruction and a second set of operation parameters. The second device operation instruction is of the same type as the first device operation instruction and each second device operation instruction corresponds to a device to be managed. The second set of operation parameters contains a combination of device operation parameters corresponding to each second device operation instruction. The second function call signal is used to invoke the service interface to perform the corresponding device operation instructions for one or more devices to be managed as indicated by the device operation parameters and to obtain the operation results; The operation result is returned to the user equipment.

2. The method according to claim 1, characterized in that, Adjusting the device operation parameters in the first function call signal based on the natural language fragment includes one or more of the following: Verify the device identifier in the first operation parameter set based on the preset identifier set; Multi-device segmentation is performed based on a pre-set identifier set and the natural language segments; Parameters are expanded by combining contextual data and the natural language fragments; Quantitative information is extracted from the natural language segment; Mark the missing device operation parameters.

3. The method according to claim 2, characterized in that, The method further includes: The large language model is invoked to perform one or more rounds of natural language-based user dialogue and intent recognition based on the missing markers to obtain the currently missing device operation parameters. Fill the missing device operation parameters into the corresponding device operation parameter combinations in the second operation parameter set.

4. The method according to claim 2, characterized in that, The multi-device segmentation based on the preset identifier set and the natural language segment includes: identifying a predetermined delimiter in the natural language segment; extracting multiple device identifiers from the natural language segment according to the predetermined delimiter; matching the multiple device identifiers with the preset identifier set to determine the device identifier of each device to be managed; and splitting the first device operation instruction corresponding to the multiple devices to be managed into multiple second device operation instructions corresponding to the multiple devices to be managed.

5. The method according to claim 1, characterized in that, The devices to be managed are virtual devices and / or physical devices.

6. The method according to claim 1, characterized in that, Returning the operation result to the user device includes: returning the operation result to the user device in the form of natural language via a streaming response.

7. The method according to claim 1, characterized in that, The method further includes: After calling the large language model to perform intent recognition on the natural language segment, if the first device operation instruction is not obtained, the large language model is called to execute a natural language-based user dialogue to prompt the user.

8. A device management device based on a large language model, characterized in that, The device is used in the device management server of the device management system, and the device includes: The acquisition module is used to acquire device operation requests from user devices, wherein the device operation requests carry natural language fragments input by the user; The intent recognition module is used to call a large language model to perform intent recognition on the natural language segment and generate a first function call signal. The first function call signal includes a first device operation instruction and a first operation parameter set. The first operation parameter set includes a combination of device operation parameters corresponding to the first device operation instruction. The parameter adjustment module is used to adjust the first set of operation parameters according to the natural language fragment to generate a second function call signal. The second function call signal contains a second device operation instruction and a second set of operation parameters. The second device operation instruction is of the same type as the first device operation instruction and each second device operation instruction corresponds to a device to be managed. The second set of operation parameters contains a combination of device operation parameters corresponding to each second device operation instruction. The business invocation module is used to invoke the business interface using the second function invocation signal to perform the corresponding device operation instructions for one or more devices to be managed as indicated by the device operation parameters and to obtain the operation results. The response module is used to return the operation result to the user equipment.

9. An electronic device, comprising: One or more processors; as well as A memory storing a computer program that, when run by the processor, causes the processor to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, the computer program causing the processor to perform the method as claimed in any one of claims 1 to 7 when executed by a processor.

Images