Multi-intelligent-device collaborative decision-making method, device, equipment, medium and product

Abstract

The invention provides a multi-intelligent-device collaborative decision-making method and device, equipment, a medium and a product. The method comprises the following steps: acquiring event information reported by a plurality of sensing devices in a target scene; a decision demand is generated in response to a user instruction or rule triggering information, the rule triggering information is generated when the event information is matched with rules in a rule base, and the decision demand comprises equipment resource configuration information of the target scene, knowledge information of the target scene and a to-be-executed task corresponding to the user instruction or the rule triggering information; inputting the decision demand into an information dependence discrimination model to obtain task planning demand information including the to-be-executed task and a decision information judgment result of an acquisition source of the task planning demand information; and generating an execution decision of the to-be-executed task based on the decision information, the execution decision including sub-tasks of a plurality of intelligent devices, and issuing the sub-tasks to the corresponding intelligent devices for execution. According to the invention, cooperation of multiple intelligent devices can be realized, and complex tasks can be completed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a multi-intelligent-device cooperative decision-making method, device, equipment, medium and product. BACKGROUND

[0002] In the prior art, intelligent devices have been applied to the field of smart home. However, in the prior art, the intelligent devices used in the home scenario are still independent of each other and can only implement specific functions, and cannot form an effective cooperative mechanism with other intelligent devices to perform complex tasks. For example, after a smart camera detects that an old person has fallen, it can only implement the functions implemented by the smart camera, such as issuing a warning, and cannot form a linkage with other robots to perform complex tasks such as fetching medicine and first-aid equipment in parallel after detecting that an old person has fallen. SUMMARY

[0003] The present application provides a multi-intelligent-device cooperative decision-making method, device, equipment, medium and product to solve the defect that complex tasks cannot be performed cooperatively by multiple intelligent devices in the prior art, and to implement cooperative execution of complex tasks by multiple intelligent devices.

[0004] The present application provides a multi-intelligent-device cooperative decision-making method, comprising: obtaining event information reported by multiple sensing devices in a target scenario; generating a decision-making requirement in response to a user instruction or rule trigger information, the rule trigger information being generated when the event information matches a rule in a rule library, the decision-making requirement including device resource configuration information of the target scenario, knowledge information of the target scenario, and a to-be-executed task corresponding to the user instruction or the rule trigger information; inputting the decision-making requirement into an information dependency discrimination model to obtain a decision-making information judgment result output by the information dependency discrimination model, the decision-making information judgment result including task planning requirement information of the to-be-executed task and an acquisition source of the task planning requirement information; generating an execution decision of the to-be-executed task based on the decision-making information, the execution decision including sub-tasks of multiple intelligent devices, and the sub-tasks being executed by corresponding intelligent devices, the intelligent devices including the sensing devices and auxiliary function robots.

[0005] According to the multi-intelligent-device cooperative decision-making method provided by the present application, before the execution decision of the to-be-executed task is generated based on the decision-making information, the method comprises: when the acquisition source of the task planning requirement information in the decision-making information is a cloud, generating an information acquisition request based on the task planning requirement information and sending the information acquisition request to the cloud. receiving the task planning information returned by the cloud and adding the task planning information to the decision information.

[0006] According to the multi-intelligent-device cooperative decision method provided in the application, the event information is obtained by desensitizing the event data monitored by the sensing device. When the task planning requirement information in the decision information is obtained from the cloud, the task planning requirement information is desensitized information.

[0007] According to the multi-intelligent-device cooperative decision method provided in the application, after the task planning information returned by the cloud is received, the following steps are included: adding the task planning information returned by the cloud to the knowledge information of the target scene.

[0008] According to the multi-intelligent-device cooperative decision method provided in the application, before the subtask is executed by the corresponding intelligent device, the following steps are included: generating an execution disturbance strategy of the intelligent device based on the function of the intelligent device, the execution disturbance strategy reflecting a correction strategy of the intelligent device when an abnormal event occurs during execution of a task; When there is a permission requirement in the subtask, the corresponding permission is granted to the intelligent device executing the subtask based on the execution time of the subtask.

[0009] According to the multi-intelligent-device cooperative decision method provided in the application, after the subtask is executed by the corresponding intelligent device, the following steps are included: obtaining sensing information of the intelligent device during execution of the subtask, and determining a target disturbance strategy from a plurality of execution disturbance strategies based on the sensing information; controlling the intelligent device to execute the target disturbance strategy; optimizing the target disturbance strategy based on an execution result of the target disturbance strategy.

[0010] The application further provides a multi-intelligent-device cooperative decision device, which includes: an event acquisition module configured to acquire event information reported by a plurality of sensing devices in a target scene; a decision requirement generation module configured to generate a decision requirement in response to a user instruction or a rule trigger information, the rule trigger information being generated when the event information matches a rule in a rule library, the decision requirement including device resource configuration information of the target scene, knowledge information of the target scene, and a to-be-executed task corresponding to the user instruction or the rule trigger information; The information dependence determination module is configured to input the decision requirement into an information dependence determination model to obtain a decision information determination result output by the information dependence determination model, wherein the decision information determination result comprises task planning requirement information of the to-be-executed task and a source of the task planning requirement information. The task execution decision module is configured to generate an execution decision of the to-be-executed task based on the decision information, wherein the execution decision comprises sub-tasks of a plurality of intelligent devices, and the sub-tasks are sent to the corresponding intelligent devices for execution, wherein the intelligent devices comprise the sensing devices and the auxiliary function robots.

[0011] The application further provides an electronic device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the multi-intelligent-device cooperative decision method according to any one of the above when executing the computer program.

[0012] The application further provides a non-transitory computer-readable storage medium, which stores a computer program, and the computer program implements the multi-intelligent-device cooperative decision method according to any one of the above when executed by a processor.

[0013] The application further provides a computer program product, which comprises a computer program, and the computer program implements the multi-intelligent-device cooperative decision method according to any one of the above when executed by a processor.

[0014] The multi-intelligent-device cooperative decision method, device, equipment, medium and product provided by the application can obtain event information reported by a plurality of sensing devices in a target scene, and if the event information matches a rule in a rule library or a user instruction is received, a decision requirement of a to-be-executed task is generated, the decision requirement comprises device resource configuration information of the target occasion and knowledge information of the target scene, and the decision requirement is input into an information dependence determination model. Based on the information, the information dependence determination model outputs a decision information determination result, which comprises task planning requirement information of the to-be-executed task and a source of the task planning requirement information. Thus, the task planning requirement information can be obtained from the source of the task planning requirement information, and an execution decision of the to-be-executed task is generated, which comprises sub-tasks of a plurality of auxiliary function robots. The sub-tasks are sent to the corresponding auxiliary function robots for execution. In this way, when a user issues an instruction or a trigger condition of a complex task is met, the to-be-executed task can be divided into a plurality of sub-tasks, and each sub-task is sent to an intelligent device that can implement the sub-task for execution. Thus, the multi-intelligent-device cooperation can be implemented, and the complex task can be completed. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0016] Figure 1 is a flowchart of a method for collaborative decision-making of multiple intelligent devices provided by the application.

[0017] Figure 2 is a data processing process diagram of a decision-making process in the method for collaborative decision-making of multiple intelligent devices provided by the application.

[0018] Figure 3 is a structural diagram of a device for collaborative decision-making of multiple intelligent devices provided by the application.

[0019] Figure 4 is a structural diagram of an electronic device provided by the application. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of the application more clear, the technical solutions in the application will be described clearly and completely in the following with reference to the drawings in the application. Obviously, the described embodiments are some embodiments of the application, but not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the application.

[0021] It should be understood that when used in the specification and the appended claims, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0022] It should also be understood that the terms used in the specification of the application are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in the specification and the appended claims of the application, unless otherwise clear from the context, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0023] It should be further understood that the term "and / or" used in the specification and the appended claims of the application means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0024] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0025] The following is combined with Figures 1-2 Describe the multi-intelligent device collaborative decision-making method provided in this application. For example... Figure 1 As shown, the multi-intelligent-device collaborative decision-making method includes the following steps: S110. Obtain event information reported by multiple sensing devices within the target scene; S120. In response to user instructions or rule triggering information, generate decision requirements. Rule triggering information is generated when event information is matched with rules in the rule base. Decision requirements include the task to be executed corresponding to user instructions or rule triggering information, device resource configuration information of the target scenario, and knowledge information of the target scenario. S130. Input the decision requirements into the information dependency discrimination model to obtain the decision information judgment result output by the information dependency discrimination model. The decision information judgment result includes the task planning requirement information of the task to be executed and the source of the task planning requirement information. S140. Generate an execution decision for the task to be executed based on the decision information. The execution decision includes sub-tasks of multiple intelligent devices. The sub-tasks are sent to the corresponding intelligent devices for execution. The intelligent devices include sensing devices and assistive robots.

[0026] The multi-intelligent device collaborative decision-making method provided in this application can be executed by a central control unit in the target scene, which can be a robot or other intelligent device. The target scene also includes multiple sensing devices and multiple auxiliary function robots. The sensing devices are used to collect information within the target scene and detect events. The auxiliary function robots are robots capable of various functions, such as robots for delicate operations in corners of homes, bathrooms, or narrow spaces, or small transport robots with only weighing capabilities. Figure 2 As shown, taking a home setting as an example, such as... Figure 2 As shown in Part (a), one high-end robot can act as the main control center to execute the method provided in this application and make decisions on user operation requirements or spontaneous tasks; M sensors can collect multi-dimensional information in the entire space as sensing devices; and N small robots can act as auxiliary function robots, which are extensions of the capabilities of the high-end robot and supplement its boundary functions and collaborative work capabilities.

[0027] In the prior art, various robots or intelligent devices often work independently, and cannot implement complex tasks that require the cooperation of multiple robots or intelligent devices, such as the demand for multiple things in parallel, such as giving first aid to an old person while fetching an AED, the demand for high efficiency, such as picking up toys while sorting and placing them in a designated location, the demand for multi-space information cooperation, such as old people's whole house fall detection and first aid, the demand for functional complementation, such as cleaning the dead corners of the bathroom and the heavy oil stains in the kitchen, the demand for task relay, such as robots picking up clothes and then washing them according to the washing machine function, and the demand for task cooperation, such as multiple robots cooperating to move heavy household items.

[0028] The method provided in the present application, after obtaining the event information reported by the multiple sensing devices in the target scene, if the event information matches the rules in the rule library or a user instruction is received, a decision requirement of a to-be-executed task is generated, the decision requirement is input into an information dependency discrimination model, the decision requirement includes device resource configuration information of the target occasion and knowledge information of the target scene, based on these information, the information dependency discrimination model outputs a decision information judgment result, the decision information judgment result includes task planning requirement information of the to-be-executed task and an acquisition source of the task planning requirement information, so that the task planning requirement information can be acquired from the acquisition source of the task planning requirement information, and then an execution decision of the to-be-executed task is generated, the execution decision includes sub-tasks of multiple auxiliary functional robots, and the sub-tasks are issued to the corresponding auxiliary functional robots for execution. In this way, when a user issues an instruction or a trigger condition of a complex task is met, the task to be executed can be divided into multiple sub-tasks, and each sub-task is sent to an intelligent device that can implement the sub-task for execution, so that the cooperation of multiple intelligent devices can be realized, and the complex task can be completed.

[0029] In the method provided in the present application, the multiple sensing devices in the target scene can collect multi-dimensional information in the target scene, thereby generating events. For example, the sensing device can be a temperature sensor, which can collect temperature information, thereby reporting an event: "the indoor temperature is higher than 30 degrees". The sensing device can also be a smart camera, which can collect and analyze images in the target scene, and can report an event: "someone falls down". It should be noted that when the main control hub executing the multi-intelligent-device-cooperation decision method provided in the present application is a smart robot, the sensing device can also be a sensing device provided on the main control hub. The sensing device collects information of the target scene, generates event information, and reports it to the main control hub of the multi-intelligent-device-cooperation decision method provided in the present application.

[0030] The event information includes data of the event collected by the sensing device. In a possible implementation, in order to realize privacy protection, the sensing device includes desensitized event data in the reported event information. Specifically, the sensing device can access the master hub in advance, and transmit the function description of the sensing device to the master hub. The master hub extracts the function description of the sensing device and generates a privacy label, synchronously constructs a corresponding matrix of the function and the privacy label of the sensing device, and dynamically isolates a high-risk data flow. The master hub can extract the function description of the sensing device and generate a privacy label by using a large model, that is, input the function description of the sensing device into the large model, and obtain the function and the corresponding privacy label output by the large model. The privacy label corresponding to different functions of the sensing device reflects the disposal rule of the data corresponding to the function. Different privacy labels are used to perform different degrees of desensitization processing on the data, as shown in Table 1.

[0031] Table 1

[0032] After the master hub determines the function and the privacy label of the sensing device, the function and the corresponding privacy label of the sensing device can be sent to the sensing device. The sensing device sends data after desensitization processing based on the corresponding privacy label.

[0033] Further, similar to the sensing device, the assistive function robot can also access the master hub in advance, determine the function and the privacy label of the assistive function robot, and send data after desensitization processing based on the corresponding privacy label.

[0034] Similarly, when the master hub transmits data, the data is desensitized based on the privacy label of the data of different functions stored locally before transmission.

[0035] By setting a privacy label, different data is processed in different ways, which can effectively achieve privacy protection while multiple smart devices are working together. For example, in a home scenario, the diversity of the categories, quantities, changes, human factors, and operation tasks of the items within a limited space range brings great unpredictability. The sensing device needs to follow the privacy principle to continuously report information events to the main control center for fusion and judgment. Such information is difference information including the difference between the surrounding environment and the state of the self. In this process, a multi-level privacy event desensitization and hiding method is introduced. According to the "disposal rules" in Table 1, different levels of privacy labels are desensitized and hidden. The original sensor data (such as images and positions) are converted into high-dimensional feature representations (such as "fall probability = 0.92") at the edge end, realizing user privacy information hiding (such as identity). The main control center only receives the localized event abstraction chain, such as {event type: "fall", location: "living room", confidence: 0.95, privacy label: P3}, and converts the original data into calculable privacy data, avoiding the exposure of the original data, realizing data not out of the domain, and decision-making can be collaborative.

[0036] In a possible implementation, the data transmission between the main control center and the smart devices (including sensing devices and auxiliary function robots) can adopt a privacy channel dynamic isolation mechanism. Through communication protocol isolation, millisecond-level response of critical instructions is realized, breaking through the delay bottleneck of multi-device collaboration. Specifically, the data transmission between the main control center and the smart devices can be divided into two channels: a control channel and a data channel. The control channel has a higher priority, and adopts local encrypted MQTT (Message Queuing Telemetry Transport) + timestamp signature + initialization random generation of device unique ID to transmit critical instructions or privacy data (such as first aid instructions, fall detection data, etc.), ensuring low delay, integrity, and confidentiality. The priority of the data channel is lower, and HTTP (HyperText Transfer Protocol) is used to transmit non-private information (such as environmental temperature, etc.), supporting flexible access of multiple source devices and realizing multi-space privacy information collaboration.

[0037] By setting a privacy channel dynamic isolation, not only the isolation between control instructions and non-private data can be realized, but also the quick response of critical instructions can be realized. In addition, privacy data and non-private data can be isolated, realizing privacy protection when different smart device systems are working together.

[0038] After receiving the event information, the master control center can match the event information with the rules in the rule library to determine whether the event information needs to trigger the to-be-executed task. The rules in the rule library include the mapping relationship between the conditions and the tasks, that is, the rules are used to determine whether an event that triggers task execution occurs, and when the occurred event matches the task execution condition in the rule, the task in the rule is triggered for execution.

[0039] The rules in the rule library can be extracted based on the functions of the accessed intelligent devices. Specifically, in a possible implementation, after determining the functions of the intelligent devices, the corresponding rules corresponding to the corresponding functions of the intelligent devices can be determined based on a pre-created function-rule mapping library. In another possible implementation, after the intelligent devices access and send their own function descriptions, the alarm / potential problem event set of the intelligent devices can be identified by a large model to obtain the rules in the rule library. For example, the intelligent device is a smart camera, and its function is to detect the posture and face of a human body in an image. The obtained alarm / potential problem event corresponding to the intelligent device is the appearance of a stranger breaking in or someone falling down. These events and the corresponding execution tasks (for example, the execution task corresponding to the event of the appearance of a stranger breaking in is to call the house owner) are added to the rule library. When the event received by the master control center matches the rule in the rule library, it is determined that the task in the rule needs to be executed, thereby starting to make a decision on how to execute the to-be-executed task.

[0040] In a possible implementation, when determining the rules in the rule library, a hierarchical system can be set for the rules in the rule library, and different levels are set for the rules according to the urgency of the events. For example, a) first-level rules (high confidence): locally preset strong trigger events (such as "fall detection + abnormal heart rate"), directly wake up the master control decision; b) second-level rules (medium confidence): rules extracted from single-device function descriptions by the large model on the side (such as "toy cleaning number > 5 → trigger cleaning"), and the confidence threshold is dynamically updated according to the execution result and user satisfaction (such as successfully performing first aid to improve the weight of the "falling down-first aid" rule), to realize the self-evolution decision loop of the rule library; c) third-level rules (low confidence): generated by the large model on the side in combination with the cross-device function cooperation of multiple devices, such as sharing the mechanical model parameters (instead of the original data) when multiple robots cooperate to carry. If there are multiple rules in the matching result, the rule priority is first-level rule > second-level rule > third-level rule.

[0041] The trigger condition of the to-be-executed task can not be a single event, but a plurality of events occurring in a certain order, which requires the execution of a certain task, that is, the task execution condition included in the rule includes a plurality of events in a certain order. In a possible implementation, when matching the reported event information with the rules in the rule library, the event information is not matched with the rules in the rule library alone, but the event information is combined into an event comprehensive chain based on the event information and the timestamp of the event information, and the event comprehensive chain is matched with the rules in the rule library.

[0042] Matching the event information with the rules in the rule library can be implemented based on a rule model, which can output a matching result based on the similarity between the event information and the rule. In a possible implementation, if the rule model cannot output an accurate matching judgment result, for example, the confidence of the rule output by the rule model is relatively low, the large model of the master hub can be called, the prompt word sentence input formed by the event comprehensive chain composed of the localized event abstraction chain and the timestamp thereof is input into the large model on the terminal side for comprehensive judgment and analysis, and the prompt word sentence is deposited as a lightweight rule to supplement the rule library, which is directly matched and processed by the rule model in the subsequent process, thereby greatly improving the response speed.

[0043] By reporting the event information to the master hub after event detection in the sensing device, the master hub generates rule trigger information based on the matching of the rule library, which realizes efficient and differentiated judgment of whether the rule decision-making capability of the master hub needs to be triggered in a large amount of collected information, reduces the pressure and computing power demand of the master hub, and ensures the privacy of user information because the event information reported by the sensing device is desensitized.

[0044] When the time information matches the rule in the rule library, the task corresponding to the matched rule is determined as the to-be-executed task, and a decision requirement is generated, which reflects that the to-be-executed task needs to be decided and planned at present. For example, the rule is “fall + heart rate anomaly, then execute first aid”, then if the intelligent camera reports a fall event and the intelligent wearable watch reports a heart rate anomaly event, it is determined that the to-be-executed task is “first aid”, and a rule trigger information is generated, which includes the task of “first aid” that needs to be executed. The master hub starts to decide and plan the execution of the task. In a possible implementation, even if the rule trigger information is not generated, if the user issues an instruction to execute a certain task, the task corresponding to the user instruction is determined as the to-be-executed task, and decision planning is started, for example, the user issues an instruction to “clean the dead angle of the kitchen”, and the master hub starts to decide and plan the execution of the task of “cleaning the dead angle of the kitchen”.

[0045] Further, in addition to the task to be executed, the decision requirement also includes device resource configuration information of the target scene and knowledge information of the target scene. The device resource configuration information of the target scene reflects the functions of the intelligent devices configured in the target scene, and the knowledge information of the target scene includes inherent knowledge of the target scene, reflecting the normal state of the target scene. For example, the knowledge information of the target scene can include user habits in the target scene, a map of the target scene, an arrangement of articles in the target scene, and experience knowledge of other tasks executed in the target scene.

[0046] In a possible implementation, the device resource configuration information of the target scene not only includes the functions of the intelligent devices, but also includes the confidence of the intelligent devices, which reflects the reliability of the availability of the functions of the intelligent devices. The confidence of the intelligent devices can be reflected by the state of the intelligent devices in multiple dimensions. For example, in some embodiments, the confidence of the device can be evaluated from four sub-dimensions of motion state, device type, device power, and last task execution result, and the calculation formula can be as follows: ; ; ; ; ; wherein the number of micro-disturbance adjustments of the device is the number of times that the task of the device is adjusted by the master hub in the last task execution. The variable parameter is fine-tuned according to the specific task. respectively, are used to evaluate the confidence of the device in the four sub-dimensions of motion state factor, device type factor, device power factor, and last task execution result factor.

[0047] By setting the confidence of the device, the reliability information of the functions of the device can be provided when the task planning decision is made, so that a more accurate task decision result can be obtained.

[0048] The method provided in the application is executed at the terminal side, and the knowledge storage and computing resources at the terminal side are limited. For complex tasks, the task planning requirement information may need to be obtained from different sources. The method provided in the application does not directly generate the execution decision of the task to be executed based on the decision requirement, but first judges the source of the task planning requirement information of the task to be executed, so as to distinguish the task planning requirement information that cannot be obtained locally and needs to be obtained by relying on other sources, realize the acquisition of complete task planning requirement information needed to complete the task to be executed, provide more accurate information for subsequent execution decision, and improve the reliability of the execution decision.

[0049] Specifically, after obtaining the device resource configuration information including the target scene, the knowledge information of the target scene, and the decision requirement of the to-be-executed task, the decision requirement input information is input into the information dependency discrimination model to obtain a decision information judgment result output by the information dependency discrimination model, the decision information judgment result including task planning requirement information of the to-be-executed task and a source of the task planning requirement information. By referring to a hierarchical framework in scientific and cognitive psychology: an instinct reflex layer, an instinct drive layer, a mode processing layer, and a high-order decision layer, the hierarchical framework is introduced into a dynamic hierarchical decision engine, and an information dependency judgment system is constructed in combination with an end-cloud collaborative architecture, as shown in Table 2.

[0050] Table 2

[0051] To realize the discrimination of different sources of the above information, the information dependency discrimination model is used to output the task planning requirement information of the to-be-executed task and the source of the task planning requirement information. In a possible implementation manner, the information dependency discrimination model can be a language large model. Before the decision requirement input information is input into the information dependency discrimination model, the decision requirement can be converted into a prompt word based on an existing template, and the prompt word is input into the information dependency discrimination model.

[0052] Further, limited by the knowledge reserve and computing resources of the end side for executing the method provided in the application, the source of the task planning requirement information can be the cloud, and in this case, the end side needs to send part of the data to the cloud, so that the cloud returns the related task planning requirement information. In this process, data will be transmitted out of the target scene domain, thereby causing the risk of exposure of private data. In a possible implementation manner of the method provided in the application, the requirement of protecting the privacy of users to the greatest extent needs to be considered, and the information dependency discrimination model is specially trained, so that when the source of the task planning requirement information in the decision information output by the information dependency discrimination model is the cloud, the information dependency discrimination model also performs desensitization processing on the task planning requirement information. In this way, the desensitized task planning requirement information can be sent to the cloud, so that the private content is de-privatized and rewritten while the information that must be decided by the cloud is split according to the information dependency analysis result, thereby realizing the protection of private information such as the user's home environment, private articles, and living habits.

[0053] For example, if the task to be executed is "cleaning the home, especially the bathroom, kitchen, and other sanitary dead corners," the demand information can be constructed as follows: "You are a master robot with excellent home scene task planning capabilities, equipped with auxiliary robots and sensor devices such as

Device 1 | Function

Device N | Function

Device 1 | Detailed Information | Confidence Factor 1

Device N | Detailed Information N | Confidence Factor N

Habit 1

Habit N

Information 1

Information N

End-Side Knowledge Base

User Interaction Instruction, such as Help me clean the home, especially the bathroom, kitchen, and other sanitary dead corners

[0054] Based on the above example of demand information, the information dependency discrimination model can output the following thinking process: <think>Alright, I need to help the user plan a cleaning schedule for their home, specifically focusing on the bathroom and kitchen. The user mentioned they already have some small, separate auxiliary robots to help, so I need to consider how to effectively utilize these robots to increase efficiency. First, I need to identify the user's main needs: bathroom hygiene dead spots and odors, and kitchen heavy oil stains. Both areas are cleaning priorities, but the cleaning methods may differ. The bathroom may require mold and limescale treatment, while the kitchen requires oil stains and stubborn stains. Next, I need to consider the existing resources, which are the functions and reliability of the auxiliary robots. Then, I need to think about the cleaning process. It may be necessary to handle the bathroom first, then the kitchen, or do them simultaneously. Considering the efficiency of the robots, try to use multiple robots at the same time or in stages. For example, first use the water absorption function of the auxiliary robot to clean the bathroom water, then use the oil removal function of the auxiliary robot to clean the kitchen oil stains. Also, the user mentioned the need to consider the lack of information and judgment. For example, whether the robot can cover all corners, whether there is enough cleaning agent, whether manual assistance is needed, etc. These all need to be considered in the plan. In addition, safety is also a factor. During the cleaning process, whether the robot will cause damage to the items, or whether manual supervision is needed. Especially when using cleaning agents, it may be necessary to ensure that there is no slip or chemical reaction. I also need to consider the time arrangement, which I already know the user's habits and family information. I need to consider how long each step takes, and which time period is more suitable to avoid interference with daily life. For example, deep cleaning can be done when family members are not at home, or in the morning or evening. Finally, it may also be necessary to evaluate the cleaning effect. Whether it is necessary to check after cleaning, or whether there are areas that need to be cleaned repeatedly. This may involve a feedback mechanism to adjust the cleaning plan based on the results.< / think> .

[0055] The information dependency discrimination model can further output the decision information judgment result as follows, which includes 7 pieces of task planning information, and label is the acquisition source tag indicating the acquisition source of the task planning demand information.

[0056] Output result: In order to help the user plan an efficient cleaning scheme for the home, especially for the bathroom and kitchen, which are difficult and critical areas, I need to know the following information: Auxiliary robot functions:

label: already available

Device 1 | Function | Detailed Information | Confidence Factor 1

Device N | Function | Detailed Information N | Confidence Factor N

[0057] Specific conditions of cleaning areas:

label: to be observed and supplemented by myself

[0058] Available tools and cleaning agents:

label: already available

Information 1

Information N

[0059] Cleaning steps and procedures:

label: Lack of knowledge and methods, to be determined by the cloud

Privacy: Function 1,..., Function N

[0060] Time scheduling and schedule:

label: Already available

Habit 1

Habit N

[0061] Safety considerations:

label: Lack of knowledge and methods, to be determined by the cloud

Privacy: Feature 1,..., Feature N

[0062] Effect evaluation and feedback:

label: To be determined by myself

[0063] In the above example decision information judgment results, for label "lack of knowledge and methods, to be determined by the cloud", the task planning requirement information is obtained from the cloud, and the "Privacy" part is the result after desensitizing the privacy data.

[0064] In order to make the information dependence discrimination model output the task planning requirement information of the task to be executed and the acquisition source of the task planning requirement information, a DPO (Direct Preference Optimization) algorithm can be used to reinforce the training of a 3B language model to obtain an information dependence discrimination model, and end-side deployment can be implemented. The training loss function increases the accuracy loss of the label judgment in the output result based on the DPO loss function , and if further privacy protection is needed, the completeness loss of the rewritten privacy information Privacy can also be added to the loss function to improve the accuracy of the end-to-cloud decision link selection and the completeness of user privacy information protection. The complete loss function is as follows, thereby strengthening the self-judgment ability of the model on the information dependence situation.

[0065] ; ; ; wherein, is the DPO loss, N is the number of samples, K is the number of classes, is the true label of sample i (1 if belonging to class j, 0 otherwise), is the predicted probability. m represents the number of

Privacy:****

Privacy:****

[0066] When the acquisition source of the task planning requirement information in the decision information is the cloud, an information acquisition request is generated based on the task planning requirement information, the information acquisition request is sent to the cloud, and the task planning information returned by the cloud is received and added to the decision information.

[0067] When the acquisition source of the task planning requirement information in the decision information is the cloud, a prompt word can be generated based on the task planning requirement information, the prompt word is sent to the cloud, the cloud model outputs the task planning information and returns it to the end-side central control hub executing the method provided in the present application. Taking the above-mentioned "cleaning the home, especially the bathroom, kitchen and other sanitary dead corners" task as an example, based on the label output by the information dependence discrimination model, which is "lack of knowledge and method, waiting for cloud judgment", the acquisition source of the task planning requirement information is the cloud, and the prompt word can be generated as follows: You are a master robot with excellent task planning ability in the home scene. Currently, you are equipped with a master robot, an auxiliary robot and a sensor device with

function 1,..., function N

feature 1,..., feature N

user interaction instruction. For example, help me clean the home, especially the bathroom, kitchen and other sanitary dead corners

[0068] After the cloud receives the prompt word, the prompt word is input into the cloud large model, combined with network search and experience knowledge base of the same type of robot for planning, and the task planning information is output and returned to the end-side.

[0069] The end-side central control center performing the method provided in the application can add the task planning information sent by the cloud to the knowledge information of the target scene. Specifically, the task planning information and the related knowledge and judgment ideas on which the task planning information is based can be combined with the scene situation to be deposited as knowledge base information and stored in the end-side local knowledge base. Subsequent similar problems can be solved by taking the end-side deposited knowledge base as additional input and performing planning at the end side.

[0070] Then, the end-side local knowledge base information and the operation steps to be refined and the related supplementary information in the "output result" of the 3B end-side large model in the foregoing are converted into prompt words, and are re-input into the 3B end-side large model to make a final scheme decision. Through the end-side deposition of knowledge, the end-side model can gradually accumulate and learn knowledge, fully protect the user privacy, improve the response speed, and realize the direct execution of the decisions of the instinct reflex layer, such as emergency braking, the instinct tendency layer, such as charging priority in the case of insufficient power, and the mode processing layer (i.e., local planning) of the robot self-adaptation to the terrain cleaning, at each device end side, while the high-order decision layer, such as the user operation demand and the decision based on comprehensive information, enters the end-cloud collaborative decision process.

[0071] If it is judged that the end side is not dependent on the cloud, the end-side local knowledge base information and the decision information judgment result in the "output result" of the 3B end-side large model in the foregoing are directly input into the end-side large model to make a final scheme decision, and an execution decision of a task to be executed is obtained. The execution decision includes sub-tasks of multiple intelligent devices, which are sent to the corresponding intelligent devices for execution.

[0072] In a possible implementation manner, after the sub-tasks are sent to the corresponding intelligent devices for execution, the execution process of the intelligent devices is tracked, and abnormal events are handled in a timely manner. Taking a family scene as an example, the unpredictability in the actual execution process in a complex family scene requires real-time supervision of the intermediate state, actual position, task progress and the like of the robot / intelligent hardware, and timely discovery of execution sub-process errors, device abnormalities, emergencies and the like (for example, a user suddenly puts an obstacle on the original advancing route of the robot), so as to ensure the final completion of the task. At the same time, since some execution operations involve user privacy, the temporary permission of the corresponding execution robot needs to be limited to ensure the maximum protection of the user privacy.

[0073] In order to track the sub-tasks and ensure that the sub-tasks can be normally completed, in a possible implementation manner of the method provided in the application, before the sub-tasks are sent to the corresponding intelligent devices for execution, the method includes: Based on the function of the intelligent device, an execution disturbance strategy of the intelligent device is generated, the execution disturbance strategy reflecting a correction strategy of the intelligent device when an abnormal event occurs during the execution of the task. When there is a permission requirement in the subtask, the corresponding permission is granted to the intelligent device executing the subtask based on the execution time of the subtask.

[0074] In this implementation, when the intelligent device accesses the master hub, the abnormal events that may occur when the intelligent device implements the function can be determined in advance based on the function of the intelligent device, and the correction strategies when these abnormal events occur can be generated in advance. Specifically, when the device is newly accessed, the execution disturbance strategy is automatically generated and stored in the end-side knowledge base by combining the end-side large model with the specification and function introduction information synchronized by each device (for example, when the cleaning agent is used up during cleaning, the remaining cleaning agent is selected again).

[0075] Before the execution plan starts, the master hub robot dynamically allocates temporary access rights to necessary user privacy data information based on the usage of the last big innovation point "user home information" and the corresponding relationship between the subtasks executed by each master / auxiliary robot (such information is extracted and deposited as a user portrait from user interaction behavior according to interaction frequency / user active setting, etc.). The sensitive operation is completed in the limited time in the trusted execution environment of the master / auxiliary robot. For example, the master hub analyzes and recognizes that the old user takes antihypertensive drugs at 8 o'clock every day, and the small robot that delivers the medicine has access rights to the user's medication information / medicine location between 7:50 and 8:10, and the access rights are automatically cleared after a certain time.

[0076] After the subtask is assigned to the corresponding intelligent device for execution, it includes: Obtaining sensing information of the intelligent device when executing the subtask, determining a target disturbance strategy among a plurality of execution disturbance strategies based on the sensing information; Controlling the intelligent device to execute the target disturbance strategy; Optimizing the target disturbance strategy based on the execution result of the target disturbance strategy.

[0077] During the execution plan, the execution disturbance strategy, the received sensor information, and the execution disturbance strategy in the knowledge base are converted into text information in real time, input into a correction decision model (such as a multi-classification model based on a 3B end-side language large model) for comprehensive judgment, and the specific strategy for executing the execution disturbance strategy or re-planning the execution is selected. Two-layer scheme guarantee mechanism ensures the final completion of the task target.

[0078] In a possible implementation, the optimization of the target perturbation strategy based on the execution result of the target perturbation strategy is to set a confidence for each execution perturbation strategy, which reflects the effectiveness of the execution perturbation strategy. When the confidence score is set for each strategy, it is generated based on a scoring system constructed in a device-by-function manner. For example, if the strategy only involves single-function dimension adjustment of the current device, the initial confidence score is 0.8. If the strategy involves joint adjustment of multiple function dimensions of the current device, the initial confidence score is 0.7. If the strategy involves joint adjustment of a single function dimension of each device, the initial confidence score is 0.6. If the strategy involves joint adjustment of multiple function dimensions of each device, the initial confidence score is 0.5.

[0079] After each execution of the target perturbation correction strategy, the confidence score of the strategy is increased by 0.1*(0.9^correction times

the number of times is 1-5

the number of times is 1-5

[0080] In the above embodiments of the method provided in the application, the master hub dynamically allocates temporary access rights to necessary user privacy data information according to the use of the "user home information" in the foregoing end-to-cloud decision planning process and the correspondence between the sub-tasks executed by each intelligent device during the task execution process; at the same time, the system monitors the device state and environmental changes in real time, selects micro-perturbation correction or triggers re-planning through a lightweight correction decision model, and dynamically optimizes the strategy confidence according to the execution result, finally realizing privacy security and fault-tolerant self-adaptation of the execution process.

[0081] The multi-intelligent-device collaborative decision device provided in the application is described below, and the multi-intelligent-device collaborative decision device described below can be correspondingly referred to the multi-intelligent-device collaborative decision method described above. As shown in Figure 3 The multi-intelligent-device collaborative decision device provided in the application includes: The event acquisition module 310 is configured to acquire event information reported by a plurality of sensing devices in a target scene. The decision demand generation module 320 is configured to generate a decision demand in response to a user instruction or rule trigger information. The rule trigger information is generated when the event information matches a rule in a rule library. The decision demand includes device resource configuration information of the target scene, knowledge information of the target scene, and a to-be-executed task corresponding to the user instruction or the rule trigger information. The information dependency determination module 330 is configured to input the decision requirement into the information dependency determination model to obtain a decision information judgment result output by the information dependency determination model, wherein the decision information judgment result includes task planning requirement information of the to-be-executed task and a source of the task planning requirement information. The task execution decision module 340 is configured to generate an execution decision of the to-be-executed task based on the decision information, wherein the execution decision includes sub-tasks of multiple intelligent devices, and the sub-tasks are sent to corresponding intelligent devices for execution, and the intelligent devices include sensing devices and auxiliary function robots.

[0082] Figure 3 An example of an entity structure diagram of an electronic device is shown in FIG. 4. Figure 4 As shown in FIG. 4, the electronic device can include a processor 410, a communications interface 420, a memory 430, and a communications bus 440, wherein the processor 410, the communications interface 420, and the memory 430 can communicate with each other through the communications bus 440. The processor 410 can invoke a logical instruction in the memory 430 to execute a multi-intelligent device cooperative decision method, which includes: obtaining event information reported by multiple sensing devices in a target scene; generating a decision requirement in response to a user instruction or rule trigger information, wherein the rule trigger information is generated when the event information matches a rule in a rule library, and the decision requirement includes device resource configuration information of the target scene, knowledge information of the target scene, and a to-be-executed task corresponding to the user instruction or the rule trigger information; inputting the decision requirement into an information dependency determination model to obtain a decision information judgment result output by the information dependency determination model, wherein the decision information judgment result includes task planning requirement information of the to-be-executed task and a source of the task planning requirement information; generating an execution decision of the to-be-executed task based on the decision information, wherein the execution decision includes sub-tasks of multiple intelligent devices, and the sub-tasks are sent to corresponding intelligent devices for execution, and the intelligent devices include sensing devices and auxiliary function robots.

[0083] In addition, the logic instructions in the memory 430 described above can be implemented in the form of a software function unit and sold or used as an independent product, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or say the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0084] In another aspect, the present application also provides a computer program product, the computer program product comprising a computer program, the computer program being stored on a non-transitory computer readable storage medium, and the computer program being executable by a processor to cause a computer to perform the multi-intelligent device cooperative decision method provided by the above method, the method comprising: obtaining event information reported by a plurality of sensing devices in a target scene; generating a decision requirement in response to a user instruction or a rule trigger information, the rule trigger information being generated when the event information matches a rule in a rule library, the decision requirement including device resource configuration information of the target scene, knowledge information of the target scene, a to-be-executed task corresponding to the user instruction or the rule trigger information; inputting the decision requirement into an information dependency discrimination model to obtain a decision information judgment result output by the information dependency discrimination model, the decision information judgment result including task planning requirement information of the to-be-executed task and an acquisition source of the task planning requirement information; generating an execution decision of the to-be-executed task based on the decision information, the execution decision including sub-tasks of a plurality of intelligent devices, and the sub-tasks being distributed to corresponding intelligent devices for execution, the intelligent devices including sensing devices and auxiliary function robots.

[0085] In yet another aspect, the present application also provides a non-transitory computer-readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the multi-intelligent device cooperative decision-making method provided by any of the above methods, and the method comprises: acquiring event information reported by a plurality of sensing devices in a target scene; generating a decision requirement in response to a user instruction or rule trigger information, the rule trigger information being generated when the event information matches a rule in a rule library, the decision requirement including device resource configuration information of the target scene, knowledge information of the target scene, and a to-be-executed task corresponding to the user instruction or the rule trigger information; inputting the decision requirement into an information dependency discrimination model to obtain decision information judgment results output by the information dependency discrimination model, the decision information judgment results including task planning requirement information of the to-be-executed task and an acquisition source of the task planning requirement information; generating an execution decision of the to-be-executed task based on the decision information, the execution decision including sub-tasks of a plurality of intelligent devices, and the sub-tasks being distributed to corresponding intelligent devices for execution, the intelligent devices including sensing devices and auxiliary function robots.

[0086] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0087] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be implemented by means of software and necessary universal hardware platforms, and of course can also be implemented by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0088] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A collaborative decision-making method for multiple intelligent devices, characterized in that, include: Acquire event information reported by multiple sensing devices within the target scene; In response to user instructions or rule triggering information, a decision requirement is generated. The rule triggering information is generated when the event information is matched with the rules in the rule base. The decision requirement includes the device resource configuration information of the target scenario, the knowledge information of the target scenario, and the task to be executed corresponding to the user instruction or the rule triggering information. The decision requirements are input into the information dependency discrimination model to obtain the decision information judgment result output by the information dependency discrimination model. The decision information judgment result includes the task planning requirement information of the task to be executed and the source of the task planning requirement information. Based on the decision information, an execution decision is generated for the task to be executed. The execution decision includes sub-tasks of multiple smart devices. The sub-tasks are then sent to the corresponding smart devices for execution. The smart devices include the sensing devices and the assistive robots.

2. The multi-intelligent device collaborative decision-making method according to claim 1, characterized in that, Before generating the execution decision for the task to be executed based on the decision information, the process includes: When the source of the task planning requirement information in the decision information is the cloud, an information acquisition request is generated based on the task planning requirement information, and the information acquisition request is sent to the cloud. Receive the task planning information returned from the cloud and add it to the decision information.

3. The multi-intelligent device collaborative decision-making method according to claim 1, characterized in that, The event information is obtained by de-sensitizing the event data monitored by the sensing device; When the source of the task planning requirement information in the decision information is the cloud, the task planning requirement information is de-identified information.

4. The multi-intelligent device collaborative decision-making method according to claim 2, characterized in that, After receiving the task planning information returned by the cloud, the process includes: The task planning information returned from the cloud is added to the knowledge information of the target scene.

5. The multi-intelligent device collaborative decision-making method according to claim 1, characterized in that, Before the subtask is sent to the corresponding smart device for execution, the following steps are included: Based on the functions of the smart device, an execution perturbation strategy for the smart device is generated, which reflects the correction strategy of the smart device when an abnormal event occurs during task execution; When a subtask has permission requirements, the corresponding permissions are granted to the smart device executing the subtask based on the execution time of the subtask.

6. The multi-intelligent device collaborative decision-making method according to claim 5, characterized in that, After the subtask is dispatched to the corresponding smart device for execution, the following steps are included: Acquire sensor information of the intelligent device when executing the sub-task, and determine a target perturbation strategy among multiple execution perturbation strategies based on the sensor information; Control the intelligent device to execute the target perturbation strategy; The target perturbation strategy is optimized based on the execution result of the target perturbation strategy.

7. A multi-intelligent device collaborative decision-making device, characterized in that, include: The event acquisition module is used to acquire event information reported by multiple sensing devices in the target scene; The decision requirement generation module is used to generate decision requirements in response to user instructions or rule triggering information. The rule triggering information is generated when the event information is matched with the rules in the rule base. The decision requirement includes the device resource configuration information of the target scenario, the knowledge information of the target scenario, and the task to be executed corresponding to the user instruction or the rule triggering information. The information dependency discrimination module is used to input the decision requirements into the information dependency discrimination model and obtain the decision information judgment result output by the information dependency discrimination model. The decision information judgment result includes the task planning requirement information of the task to be executed and the source of the acquisition of the task planning requirement information. The task execution decision module is used to generate an execution decision for the task to be executed based on the decision information. The execution decision includes sub-tasks of multiple smart devices. The sub-tasks are sent to the corresponding smart devices for execution. The smart devices include the sensing devices and the assistive function robot.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the multi-intelligent device collaborative decision-making method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the multi-intelligent device collaborative decision-making method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the multi-intelligent device collaborative decision-making method as described in any one of claims 1 to 6.

Images