Equipment linkage control method and device, storage medium and electronic device

Abstract

The invention discloses an equipment linkage control method and device, a storage medium and an electronic device, and relates to the technical field of smart home, the equipment linkage control method comprises the following steps: receiving scheduling data uploaded by a target client, the scheduling data comprising: an equipment event for changing an equipment state corresponding to target equipment, the target client and the target device have a communication relationship according to the real-time device state of the target device and the real-time device state of other devices in the same network with the target device; under the condition that scene configuration associated with the target client is obtained, the scheduling data and the scene configuration are input into a large decision model, and estimated action information output by the large decision model is obtained; under the condition that the sub-application type allowing control operation on the target equipment is determined, the estimated action information is converted into an operation instruction corresponding to the sub-application type, the operation instruction is issued to a target client side, and the target client side is used for scheduling the sub-application type to conduct response control on the target equipment after the equipment event.

Description

Technical Field

[0001] The present application relates to the field of smart home technology, and more specifically, to a device linkage control method, device, storage medium and electronic device. Background Art

[0002] The control method of smart home appliance linkage is based on the combination of Internet of Things technology and artificial intelligence technology. By connecting various home appliances to a unified smart home control system, linkage and intelligent control between devices are realized. Smart devices of different ecosystems in consumers' homes cannot work together. In existing homes, most consumers do not make a comprehensive plan for whole-house intelligence before purchasing smart home devices, so there will be smart devices from multiple manufacturers' Internet of Things ecosystems in the home. Smart devices in the same ecosystem can collaborate with each other, but they cannot work across ecosystems, resulting in the inability to achieve scene linkage of whole-house devices.

[0003] Secondly, the linkage settings between devices are usually cumbersome and rigid. A powerful smart device usually has a wide range of functions. If users want to achieve scene linkage between multiple smart devices, they usually need to perform complex and lengthy settings in the APP and repeated trial and error to achieve satisfaction; however, when environmental factors such as external light, temperature, and wind speed change, the previous settings often cannot meet the current environment, resulting in a poor user experience. Therefore, it is necessary to learn user behavior, combine smart devices, refer to scene templates, and finally coordinate smart devices in multiple ecosystems through a large model to achieve the best user experience.

[0004] Therefore, in relation to the problem in related technologies that smart devices from different manufacturers cannot be linked together, resulting in high development costs and complex user operations, no effective solution has yet been proposed. Summary of the invention

[0005] The embodiments of the present application provide a device linkage control method, device, storage medium and electronic device to at least solve the problem in the related art that linkage between smart devices from different manufacturers cannot be achieved, resulting in high development costs and complex user operations.

[0006] According to an embodiment of the embodiments of the present application, a control method for device linkage is provided, including: receiving scheduling data uploaded by a target client, wherein the scheduling data includes: a device event that changes a device state corresponding to the target device, and the real-time device state of other devices in the same network as the target device, and a communication relationship exists between the target client and the target device; when a scene configuration associated with the target client is obtained, the scheduling data and the scene configuration are input into a decision-making model to obtain estimated action information output by the decision-making model; when a sub-application type that allows control operations to be performed on the target device is determined, the estimated action information is converted into an operation instruction corresponding to the sub-application type, and the operation instruction is sent to the target client, wherein the target client is used to schedule the sub-application type to perform response control on the target device after the device event.

[0007] In an exemplary embodiment, after receiving the scheduling data uploaded by the target client, the above method also includes: extracting text information corresponding to the device events in the scheduling data; performing semantic retrieval in the cloud knowledge base based on the text information to obtain target knowledge matching the text information, wherein the target knowledge includes instructions for use of the target device and operation question and answer data of the target device.

[0008] In an exemplary embodiment, before performing semantic retrieval in a cloud knowledge base based on text information to obtain target knowledge matching the text information, the method further includes: when the target client has completed coordinated configuration of multiple sub-application types, obtaining manufacturer data corresponding to each sub-application type through a communication channel between the target cloud and the Internet, wherein the manufacturer data includes data corresponding to at least two different devices; parsing the manufacturer data, and determining knowledge documents corresponding to different devices based on the parsing results, wherein the knowledge documents include at least one of the following: device instructions for different devices, and answers to device questions for different devices; vectorizing the knowledge documents, and aggregating the vectorization results of different devices to obtain a cloud knowledge base.

[0009] In an exemplary embodiment, when the scene configuration associated with the target client is obtained, the scheduling data and the scene configuration are input into the decision-making big model. Before obtaining the estimated action information output by the decision-making big model, the above method also includes: obtaining the historical scheduling data and historical scene configuration of the target client to coordinate and control the target device, and determining the device action set corresponding to the historical scheduling data and the historical scene configuration; establishing a data relationship between the historical scheduling data, the historical scene configuration, and the device action set; training the preset big model according to the data relationship to obtain the decision big model, wherein the preset big model is a deep learning model to be trained through a large amount of data relationships.

[0010] In an exemplary embodiment, the estimated action information is converted into an operation instruction corresponding to the sub-application type, including: determining the operation content corresponding to the estimated action information; determining the conversion instruction format based on the operation content and the function list included in multiple sub-application types that have been coordinated and configured by the target client; using the conversion instruction format to convert the operation content, and adding the operation condition to the conversion result to obtain the operation instruction.

[0011] In an exemplary embodiment, when determining a sub-application type that allows control operations on a target device, converting estimated action information into an operation instruction corresponding to the sub-application type, and before sending the operation instruction to the target client, the method further includes: when determining that there are at least two sub-applications of the sub-application type, determining the working status of each sub-application; and when the working status indicates that a first sub-application of at least two sub-applications is unable to operate, instructing the target client to schedule a second sub-application of at least two sub-applications to respond to and control the target device after receiving the operation instruction.

[0012] In an exemplary embodiment, when determining the sub-application type that allows control operations on the target device, converting the estimated action information into an operation instruction corresponding to the sub-application type, and sending the operation instruction to the target client, the above method also includes: obtaining the control result of the target device's response control after a device event occurs; when the control result complies with the operation instruction, allowing the target client to continue to control the target device; when the control result does not comply with the operation instruction, prohibiting the target client from continuing to control the target device, and sending an alarm message of control abnormality to the management object of the target device.

[0013] According to another embodiment of the embodiment of the present application, a control device for device linkage is also provided, including: a receiving module, used to receive scheduling data uploaded by a target client, wherein the scheduling data includes: a device event that changes the device state corresponding to the target device, and the real-time device state of other devices in the same network as the target device, and there is a communication relationship between the target client and the target device; an input module, used to input the scheduling data and the scene configuration into the decision model when the scene configuration associated with the target client is obtained, and obtain the estimated action information output by the decision model; a conversion module, used to convert the estimated action information into an operation instruction corresponding to the sub-application type when it is determined that the sub-application type is allowed to control the target device, and send the operation instruction to the target client, wherein the target client is used to schedule the sub-application type to respond to the device event of the target device.

[0014] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned device linkage control method when running.

[0015] According to another aspect of an embodiment of the present application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the device linkage control method through the computer program.

[0016] According to another embodiment of the present application, a computer program product is provided, including a computer program, which implements the steps in any of the above-mentioned device linkage control method embodiments when the computer program is executed by a processor.

[0017] In an embodiment of the present application, the scheduling data uploaded by the target client is received, wherein the scheduling data includes: device events that change the device state corresponding to the target device, and the real-time device state of other devices in the same network as the target device, and the target client has a communication relationship with the target device; when the scene configuration associated with the target client is obtained, the scheduling data and the scene configuration are input into the decision-making model to obtain the estimated action information output by the decision-making model; when the sub-application type that allows the control operation of the target device is determined, the estimated action information is converted into an operation instruction corresponding to the sub-application type, and the operation instruction is sent to the target client, wherein the target client is used to schedule the sub-application type to respond to the device event of the target device. The above technical solution solves the problem that it is impossible to realize the linkage between smart devices of different manufacturers, resulting in high development costs and complex user operations. Furthermore, by combining smart devices and referring to scene templates, the smart devices under multiple ecosystems are finally coordinated through a large model to achieve the best user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0020] Figure 1 It is a hardware environment schematic diagram of a device linkage control method according to an embodiment of the present application;

[0021] Figure 2 is a flow chart of a method for controlling device linkage according to an embodiment of the present application;

[0022] Figure 3 It is a flow chart of a method for collaborative scheduling of cross-vendor smart devices based on a large model according to an embodiment of the present application;

[0023] Figure 4 is a structural block diagram of a control device for device linkage according to an embodiment of the present application;

[0024] Figure 5 is a block diagram of a computer system structure of an electronic device according to an embodiment of the present application;

[0025] Figure 6 An electronic device for implementing a control method for device linkage according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] According to one aspect of an embodiment of the present application, a device linkage control method is provided. The device linkage control method is widely used in smart home (Smart Home), smart home, smart home device ecology, smart residential (Intelli gence House) ecology and other whole-house intelligent digital control application scenarios. Optionally, in this embodiment, the above-mentioned device linkage control method can be applied to Figure 1In the hardware environment composed of the terminal device 102 and the server 104 shown in FIG. Figure 1 As shown, Figure 1 This is a hardware environment diagram of a device linkage control method according to an embodiment of the present application. The server 104 is connected to the terminal device 102 through a network, and can be used to provide services (such as application services, etc.) for the terminal or a client installed on the terminal. A database can be set on the server or independently of the server to provide data storage services for the server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data computing services for the server 104.

[0029] The network may include but is not limited to at least one of the following: wired network, wireless network. The wired network may include but is not limited to at least one of the following: wide area network, metropolitan area network, local area network, and the wireless network may include but is not limited to at least one of the following: WIFI (Wireless Fidelity), Bluetooth. The terminal device 102 may be but is not limited to a PC, a mobile phone, a tablet computer, a smart air conditioner, a smart range hood, a smart refrigerator, a smart oven, a smart stove, a smart washing machine, a smart water heater, a smart washing device, a smart dishwasher, a smart projection device, a smart TV, a smart clothes drying rack, a smart curtain, a smart audio and video, a smart socket, a smart speaker, a smart fresh air device, a smart kitchen and bathroom device, a smart bathroom device, a smart sweeping robot, a smart window cleaning robot, a smart mopping robot, a smart air purification device, a smart steamer, a smart microwave oven, a smart kitchen treasure, a smart purifier, a smart water dispenser, a smart door lock, etc.

[0030] In this embodiment, a device linkage control method is provided, which can be applied to the Internet of Things cloud. Figure 2 : is a flow chart of a method for controlling device linkage according to an embodiment of the present application, and the flow includes the following steps:

[0031] Step S202, receiving the scheduling data uploaded by the target client, wherein the scheduling data includes: a device event that changes the device state corresponding to the target device, and the real-time device state of other devices in the same network as the target device, and the target client has a communication relationship with the target device;

[0032] Optionally, in a smart home system, users can establish communication with smart home devices through mobile phone apps and remotely control home lights, air conditioners and other devices. When users change the brightness of lights on the mobile phone app, this operation will be uploaded to the backend server and updated in real time to other devices in the same network as the lights, such as smart sockets, smart door locks and other devices, to keep the synchronization between devices. At the same time, users can also receive real-time device status feedback to understand the working status of each device, which is convenient for users to monitor and adjust at any time. By receiving the scheduling data uploaded by the target client, the smart home system can achieve automated control and improve user experience and home management efficiency.

[0033] Step S204, when the scenario configuration associated with the target client is obtained, the scheduling data and the scenario configuration are input into the decision-making model to obtain the estimated action information output by the decision-making model;

[0034] Optionally, the above-mentioned scene configuration refers to user scenario-based rule configuration and scene linkage template. Among them, user scenario-based rule configuration means that users set a series of rules to achieve automatic control through the linkage function of smart devices according to their own needs and habits. For example, users can set rules to let smart lights turn on automatically at night, let smart curtains close automatically in the morning, or let smart speakers play music at a specific time. Scene linkage template means that users can choose some already designed scene templates, and realize the linkage between multiple smart devices through simple operations. For example, users can choose "home mode" to turn on smart door locks, smart lights and smart air conditioners with one click to make the home warm and comfortable; or choose "away mode" to turn off all electrical appliances with one click to ensure the safety of the home.

[0035] Step S206, when determining the sub-application type that allows control operations on the target device, convert the estimated action information into an operation instruction corresponding to the sub-application type, and send the operation instruction to the target client, wherein the target client is used to schedule the sub-application type to respond to the target device after a device event.

[0036] For example, if a family has lighting devices from multiple manufacturers, the corresponding sub-application types include the lighting control application of manufacturer A and the lighting control application of manufacturer B. When the user needs to control the lighting, the estimated actions are performed on the different applications that need to be operated separately, making the control process cumbersome. The same type of applications between different manufacturers can be linked through the target client, so that the functions of the lighting control application of manufacturer B can also be realized through the lighting control application of manufacturer A, and then only one manufacturer's application can be used when controlling the same type of devices in the home, greatly reducing the complexity of user operations. Further, when the cloud corresponding to the smart home collaborative scheduling system receives the action information on the target client, such as adjusting the light brightness or changing the light color, the cloud can generate corresponding operation instructions based on these action information. For example, if the user wants to increase the light brightness, the cloud can generate a corresponding brightness increase instruction and send it to the target client, and the target client will immediately adjust the light brightness after receiving the instruction. Users do not need to operate personally, and can adjust the light through remote control, which improves the convenience and experience of users.

[0037] Through the above steps, the scheduling data uploaded by the target client is received, wherein the scheduling data includes: device events that change the device status corresponding to the target device, and the real-time device status of other devices in the same network as the target device, and the target client has a communication relationship with the target device; when the scene configuration associated with the target client is obtained, the scheduling data and the scene configuration are input into the decision-making model to obtain the estimated action information output by the decision-making model; when the sub-application type that allows the control operation of the target device is determined, the estimated action information is converted into an operation instruction corresponding to the sub-application type, and the operation instruction is sent to the target client, wherein the target client is used to schedule the sub-application type to respond to the device event of the target device. The above technical solution solves the problem that it is impossible to realize the linkage between smart devices of different manufacturers, resulting in high development costs and complex user operations. Furthermore, by combining smart devices and referring to scene templates, the smart devices under multiple ecosystems are finally coordinated through the big model to achieve the best user experience.

[0038] In an exemplary embodiment, after receiving the scheduling data uploaded by the target client, the above method also includes: extracting text information corresponding to the device events in the scheduling data; performing semantic retrieval in the cloud knowledge base based on the text information to obtain target knowledge matching the text information, wherein the target knowledge includes instructions for use of the target device and operation question and answer data of the target device.

[0039] Optionally, the target client uploads scheduling data, including device event information: for example, the client uploads a message showing "The air conditioner is faulty and cannot cool normally." The system automatically extracts text information from the device event: The system extracts the text information "The air conditioner is faulty and cannot cool normally" from the uploaded scheduling data. Perform semantic retrieval in the cloud knowledge base: The system performs semantic retrieval in the knowledge base based on the extracted text information to find the target knowledge that matches the text information, including the instructions for use of the air conditioner and related operational question and answer data. Obtain target knowledge and respond in real time: The system obtains the instructions for use and operational question and answer data of the air conditioner, and feeds back relevant information to the client to help customers solve air conditioner failure problems and achieve real-time response and problem solving.

[0040] In an exemplary embodiment, before performing semantic retrieval in a cloud knowledge base based on text information to obtain target knowledge matching the text information, the method further includes: when the target client has completed coordinated configuration of multiple sub-application types, obtaining manufacturer data corresponding to each sub-application type through a communication channel between the target cloud and the Internet, wherein the manufacturer data includes data corresponding to at least two different devices; parsing the manufacturer data, and determining knowledge documents corresponding to different devices based on the parsing results, wherein the knowledge documents include at least one of the following: device instructions for different devices, and answers to device questions for different devices; vectorizing the knowledge documents, and aggregating the vectorization results of different devices to obtain a cloud knowledge base.

[0041] It is understandable that, in the case where the target client has completed the coordinated configuration of multiple sub-application types, the manufacturer data corresponding to each sub-application type is obtained through the communication channel between the target cloud and the Internet, such as obtaining data of devices such as smart home devices, smartphones, and smart watches. Parse the manufacturer data to determine the knowledge documents corresponding to different devices, such as determining the device instructions based on the data of smart home devices, and determining the device problem answers based on the data of smartphones. Vectorize the knowledge documents and summarize the vectorization results of different devices to obtain a cloud knowledge base, such as summarizing the vectorization results of smart home devices, smartphones, and smart watches into a cloud knowledge base to establish a unified information platform to provide users with comprehensive device information and problem-solving methods.

[0042] In an exemplary embodiment, when the scene configuration associated with the target client is obtained, the scheduling data and the scene configuration are input into the decision-making big model. Before obtaining the estimated action information output by the decision-making big model, the above method also includes: obtaining the historical scheduling data and historical scene configuration of the target client to coordinate and control the target device, and determining the device action set corresponding to the historical scheduling data and the historical scene configuration; establishing a data relationship between the historical scheduling data, the historical scene configuration, and the device action set; training the preset big model according to the data relationship to obtain the decision big model, wherein the preset big model is a deep learning model to be trained through a large amount of data relationships.

[0043] For example, if you need to optimize the control strategy of the smart lighting and air conditioning in the living room, first collect the customer's scheduling data and scene configurations in different scenarios, such as the light brightness and air conditioning temperature settings when the customer is watching a movie, the light brightness and air conditioning temperature settings when the customer is resting, etc. Then determine the device action set of the smart lighting and air conditioning in the living room based on these data, such as adjusting the light brightness, adjusting the air conditioning temperature, etc. Then establish the data relationship between historical scheduling data, historical scene configuration, and device action set to ensure the integrity and accuracy of the data. Finally, train the preset large model based on the data relationship to obtain the decision large model, learn historical data through the deep learning model, optimize the control strategy of the smart lighting and air conditioning in the living room, and improve the intelligence level of smart home appliances.

[0044] In an exemplary embodiment, the estimated action information is converted into an operation instruction corresponding to the sub-application type, including: determining the operation content corresponding to the estimated action information; determining the conversion instruction format based on the operation content and the function list included in multiple sub-application types that have been coordinated and configured by the target client; using the conversion instruction format to convert the operation content, and adding the operation condition to the conversion result to obtain the operation instruction.

[0045] Exemplarily, a user uses a smart home system to coordinate the scheduling system, which includes: a target client and a cloud. When using the above system to control smart lamps, the system determines the action information of the user currently estimated to turn on the light. According to the action information, the operation content is determined to be "turn on the lamp". According to the configured sub-application type function list, the conversion instruction format is determined to be "device type + operation + status". In this example, the device type is "lamp", the operation is "turn on", and the status is "turn on". According to the conversion instruction format, the operation content is converted into an operation instruction: "lamp-turn on-turn on", and the operation conditions (such as time, light intensity, etc.) are added to obtain the final operation instruction. Through the above steps, the estimated action information of smart home appliances can be accurately converted into operation instructions corresponding to the sub-application type of smart home appliances, thereby realizing the intelligent control of the smart home system.

[0046] In an exemplary embodiment, when determining a sub-application type that allows control operations on a target device, converting estimated action information into an operation instruction corresponding to the sub-application type, and before sending the operation instruction to the target client, the method further includes: when determining that there are at least two sub-applications of the sub-application type, determining the working status of each sub-application; and when the working status indicates that a first sub-application of at least two sub-applications is unable to operate, instructing the target client to schedule a second sub-application of at least two sub-applications to respond to and control the target device after receiving the operation instruction.

[0047] Exemplarily, there are sub-application types for smart home appliances from two different manufacturers, that is, through the sub-application types corresponding to the manufacturer, multiple different types of manufacturer equipment can be controlled, such as two manufacturers producing air conditioners and washing machines at the same time. By determining the working status of each sub-application, it can be known that the current air conditioner is running in cooling mode and the washing machine is running in washing mode. If the first sub-application of the air conditioner cannot be operated, the second sub-application can be scheduled by the target client to respond and control the air conditioner, such as adjusting the temperature or wind speed. This ensures the normal operation of the equipment and improves the user experience. That is, through the target client, multiple manufacturers are bound to applications, and then during use, the current manufacturer's application can be scheduled to intelligently control the same type of equipment under other manufacturers except the current manufacturer, avoiding the occurrence of complex equipment control caused by too many control applications.

[0048] Optionally, users can conveniently control smart home appliances from different manufacturers, and when a sub-application of a smart home appliance from one manufacturer cannot be operated, the above system can automatically schedule other sub-applications to meet user needs, thereby improving the intelligence level of the smart home system.

[0049] In an exemplary embodiment, when determining the sub-application type that allows control operations on the target device, converting the estimated action information into an operation instruction corresponding to the sub-application type, and sending the operation instruction to the target client, the above method also includes: obtaining the control result of the target device's response control after a device event occurs; when the control result complies with the operation instruction, allowing the target client to continue to control the target device; when the control result does not comply with the operation instruction, prohibiting the target client from continuing to control the target device, and sending an alarm message of control abnormality to the management object of the target device.

[0050] For example, for a smart lighting device, the operation instruction is to turn on the light and set the brightness to 50%. When the system detects that the user sends the instruction to turn on the light through the APP, the system responds to control the light, verifies whether the light is on and adjusts the brightness to 50%. If the control result meets the operation instruction, the user is allowed to continue to control the light; if the control result does not meet the operation instruction, for example, the light is not turned on or the brightness is less than 50%, the user is prohibited from continuing to control and an alarm message is sent to the device management object.

[0051] In order to better understand the process of the control method of the above-mentioned device linkage, the implementation method flow of the control of the above-mentioned device linkage is described below in combination with an optional embodiment, but it is not used to limit the technical solution of the embodiment of the present application.

[0052] In related technologies, smart devices in the same ecosystem can collaborate with each other, but not across ecosystems, which makes it impossible to achieve scene linkage of all devices in the house. Secondly, the linkage settings between devices are usually cumbersome and rigid.

[0053] In order to solve the above problems, the optional embodiment of this application proposes a method for cross-manufacturer smart device collaborative scheduling based on a large model. Specifically, first, by vectorizing the device knowledge documents of the ecological manufacturers into the "ecological manufacturer knowledge base", when the consumer device status changes, the barrier-free service obtains the device event and uploads it to the cloud; then, through semantic retrieval, relevant knowledge is obtained, combined with the user rule configuration and scene template configuration, it is passed to the large model to make the next action suggestion; finally, the action is converted into an operation instruction and issued to the client, and the barrier-free service operates the APP of each manufacturer.

[0054] Optionally, the client realizes device intercommunication by monitoring the events of each APP client through barrier-free services to upload events and status. After receiving the decision instructions made by the big model, the barrier-free services are used to realize direct operation of each APP, thereby realizing the control of the devices of each manufacturer. Therefore, there is no need for intercommunication between devices of different manufacturers, and no need for cloud docking API (application programming interface). In theory, any smart device with APP client can be intercommunication.

[0055] Optionally, AI (artificial intelligence) is mainly reflected in that after the cloud receives device events, it does not simply generate device instructions through rules, but combines multiple information such as manufacturer knowledge base, scenario template rules, and user rule configuration to ultimately transform them into operational decisions. Therefore, even if the user has never configured rules, they can still have an intelligent scenario-based experience.

[0056] Optional, Figure 3 This is a flow chart of a method for cross-vendor smart device collaborative scheduling based on a large model according to an embodiment of the present application, which specifically includes the following steps:

[0057] Step 1: Use web crawler technology to crawl all knowledge documents about equipment instructions, question answers, etc. from all eco-manufacturers, and vectorize them into the "Eco-Manufacturer Knowledge Base". For example, if there are eco-manufacturer A, eco-manufacturer B, and eco-manufacturer C, the knowledge bases of the three manufacturers can be unified into one knowledge base through crawling, which is convenient for subsequent information scheduling and processing.

[0058] Step 2: When the status of the device in the user's home changes, the accessibility service can obtain the device event. That is, the accessibility service determines the real-time status of the current device and the target state to be adjusted by determining the control requirements input by the user on the APP. At the same time, it can also obtain the control instructions issued by the target object to the device, thereby determining the events to be executed and the control requirements of the device.

[0059] Step 3: The accessibility service of the smart home collaborative scheduling client (equivalent to the target client in the above embodiment) uploads the acquired device events and the status of other devices to the cloud corresponding to the smart home collaborative scheduling, so that the cloud can process them in combination with the ecological manufacturer knowledge base.

[0060] Step 4: Based on the events uploaded by the device, semantic retrieval is performed in the ecological manufacturer knowledge base to obtain relevant knowledge, that is, the knowledge content corresponding to the current event in the knowledge base of different manufacturers is determined respectively, so that the effective correspondence of the knowledge content of different manufacturers can be achieved based on the association of events.

[0061] Step 5: The cloud corresponding to the smart home collaborative scheduling searches and obtains the user rule configuration and scene template configuration related to the device event.

[0062] Step 6: Transmit the corresponding relationship between device events, other device states, user rule configuration, scenario template configuration, and knowledge content of different manufacturers to the big model, which will decide the next action recommendation. It should be noted that the big model is trained using the current user's historical device usage habits and device operation data, and then the big model can determine personalized action recommendations that meet the current user.

[0063] Step 7: Convert the next action corresponding to the above action suggestion into an operation instruction such as operating an APP or opening a button.

[0064] Step 8: Send the operation instruction to the smart home collaborative scheduling client.

[0065] Step 9, the barrier-free service in the smart home collaborative scheduling client operates the APPs of each manufacturer to synchronize the corresponding operation instructions between different manufacturers. Then, when the APP corresponding to one manufacturer cannot control the device to perform an operation, the same control is achieved through the APPs of other manufacturers.

[0066] Optionally, a scenario-based linkage of smart devices, the collaboration between smart devices occurs by the client simulating user operations. The specific process is as follows:

[0067] (1) When an event occurs on a device of a certain ecosystem manufacturer, a notification is pushed to the client app of the ecosystem manufacturer.

[0068] (2) Monitor notifications in the ecosystem vendor’s client app in real time and send the notifications to the cloud.

[0069] (3) The big model generates the linkage actions of the family’s devices and converts them into simulated user operations on the client.

[0070] (4) The client controls the client apps of other ecosystem vendors to automatically execute device actions.

[0071] Optionally, there are smart washing machines from manufacturer A and smart clothes drying racks from manufacturer B in the user's home smart devices, and the existing scenario template is "automatically lowering the clothes drying rack after washing is completed". When the washing machine of manufacturer A finishes washing, manufacturer A automatically pushes a notification to the "eco-manufacturer A APP". After this device automatically uploads the event to the cloud, the big model generates a decision, and the cloud sends the "lower the clothes drying rack" action to the "eco-manufacturer B APP". The barrier-free service automatically operates the APP to complete the scenario linkage.

[0072] Optionally, the user's home smart devices include a smart water pressure booster pump from manufacturer A and a smart washing machine from manufacturer B. The common sense of washing machine failure is known as "When the washing machine does not take in water, you need to check whether the water pressure is less than 0.1MPa". When the washing machine does not take in water, manufacturer A automatically pushes the fault notification "ERROR01" to the "Eco-Manufacturer A APP", and this device automatically uploads the event to the cloud. After the knowledge base semantic query, it is found that the water pressure is too low. After the big model makes a decision, it sends "start the booster pump" to the "Eco-Manufacturer B APP", and the barrier-free service automatically operates the APP to complete the scene linkage.

[0073] It should be noted that web crawler technology needs to pay attention to its legality and the risk of privacy infringement, ensuring that the knowledge documents obtained comply with legal provisions and respect user privacy. At the same time, ensure that barrier-free services can accurately obtain device events to avoid information loss or errors. In addition, device events and status uploaded to the cloud need to be encrypted and securely transmitted to protect user information from being leaked or tampered with. And when operating the APP of each manufacturer, it is necessary to pay attention to permissions and authorization issues to ensure that the operation of barrier-free services is legal and compliant, and avoid infringing on user rights or causing data leakage.

[0074] In summary, the above solution, by vectorizing device knowledge documents into the knowledge base of ecological manufacturers, realizes the convenience and efficiency of barrier-free services to obtain device events and upload them to the cloud when the device status changes. By obtaining relevant knowledge through semantic retrieval, and combining user rule configuration and scene template configuration, it can provide users with more accurate suggestions for the next action. Finally, the action is converted into an operation instruction and issued to the client, and the barrier-free service operates the APP of each manufacturer, realizing the intelligent device operation process and improving the user experience and operation efficiency. Furthermore, consumer users can purchase smart devices from any manufacturer according to their preferences, and realize scenario linkage through the coordinated scheduling of the above technical solutions. And consumer users can reuse the scenario linkage effect of template sharing without any cumbersome configuration. In addition, there is no need to search for solutions everywhere for device errors. The cloud knowledge base automatically retrieves and gives solutions, and can automatically solve the problem in combination with existing devices.

[0075] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0076] Figure 4 is a structural block diagram of a device linkage control device according to an embodiment of the present application; Figure 4 As shown, including:

[0077] The receiving module 42 is used to receive the scheduling data uploaded by the target client, wherein the scheduling data includes: a device event that changes the device state corresponding to the target device, and the real-time device state of other devices in the same network as the target device, and the target client has a communication relationship with the target device;

[0078] An input module 44 is used to input the scheduling data and the scenario configuration into the decision-making model when the scenario configuration associated with the target client is obtained, so as to obtain the estimated action information output by the decision-making model;

[0079] The conversion module 46 is used to convert the estimated action information into an operation instruction corresponding to the sub-application type when determining the sub-application type that allows the control operation on the target device, and send the operation instruction to the target client, wherein the target client is used to schedule the sub-application type to respond to the device event of the target device.

[0080] Through the above device, the scheduling data uploaded by the target client is received, wherein the scheduling data includes: device events that change the device status corresponding to the target device, and the real-time device status of other devices in the same network as the target device, and the target client has a communication relationship with the target device; when the scene configuration associated with the target client is obtained, the scheduling data and the scene configuration are input into the decision-making model to obtain the estimated action information output by the decision-making model; when the sub-application type that allows the control operation of the target device is determined, the estimated action information is converted into an operation instruction corresponding to the sub-application type, and the operation instruction is sent to the target client, wherein the target client is used to schedule the sub-application type to respond to the device event of the target device. The above technical solution solves the problem that it is impossible to realize the linkage between smart devices of different manufacturers, resulting in high development costs and complex user operations. Furthermore, by combining smart devices and referring to scene templates, the smart devices under multiple ecosystems are finally coordinated through the big model to achieve the best user experience.

[0081] In an exemplary embodiment, the above-mentioned device also includes: an extraction module, which is used to extract text information corresponding to device events in the scheduling data after receiving the scheduling data uploaded by the target client; perform semantic retrieval in the cloud knowledge base based on the text information to obtain target knowledge matching the text information, wherein the target knowledge includes instructions for use of the target device and operation question and answer data of the target device.

[0082] In an exemplary embodiment, the above-mentioned device also includes: a first acquisition module, which is used to perform semantic retrieval in a cloud knowledge base based on text information, and before obtaining the target knowledge matching the text information, when the target client has completed the coordinated configuration of multiple sub-application types, obtain the manufacturer data corresponding to each sub-application type through the communication channel between the target cloud and the Internet, wherein the manufacturer data includes at least data corresponding to two different devices; parse the manufacturer data, and determine the knowledge documents corresponding to different devices based on the parsing results, wherein the knowledge documents include at least one of the following: device instructions for different devices, and answers to device questions for different devices; vectorize the knowledge documents, and summarize the vectorization results of different devices to obtain a cloud knowledge base.

[0083] In an exemplary embodiment, the above-mentioned device also includes: a second acquisition module, which is used to input the scheduling data and scene configuration into the decision-making big model when the scene configuration associated with the target client is acquired, and before obtaining the estimated action information output by the decision-making big model, obtain the historical scheduling data and historical scene configuration of the target client to coordinate and control the target device, and determine the device action set corresponding to the historical scheduling data and historical scene configuration; establish a data relationship between the historical scheduling data, historical scene configuration, and the device action set; train the preset big model according to the data relationship to obtain the decision big model, wherein the preset big model is a deep learning model to be trained through a large amount of data relationships.

[0084] In an exemplary embodiment, the above-mentioned conversion module is also used to determine the operation content corresponding to the estimated action information; determine the conversion instruction format based on the operation content and the function list included in multiple sub-application types that have completed coordinated configuration on the target client; use the conversion instruction format to convert the operation content, and add operation conditions to the conversion result to obtain operation instructions.

[0085] In an exemplary embodiment, the above-mentioned device also includes: a determination module, which is used to convert the estimated action information into an operation instruction corresponding to the sub-application type when it is determined that the sub-application type allows control operations on the target device, and before sending the operation instruction to the target client, when it is determined that there are at least two sub-applications of the sub-application type, determine the working status of each sub-application; when the working status indicates that the first sub-application of at least two sub-applications is unable to operate, instruct the target client to schedule the second sub-application of at least two sub-applications to respond to the control of the target device after receiving the operation instruction.

[0086] In an exemplary embodiment, the above-mentioned device also includes: a third acquisition module, which is used to convert the estimated action information into an operation instruction corresponding to the sub-application type when it is determined that the sub-application type is allowed to control the target device, and after the operation instruction is sent to the target client, obtain the control result of the target device's response control after the device event occurs; when the control result complies with the operation instruction, allow the target client to continue to control the target device; when the control result does not comply with the operation instruction, prohibit the target client from continuing to control the target device, and send an alarm information of control abnormality to the management object of the target device.

[0087] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0088] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0089] An embodiment of the present application further provides a storage medium, which includes a stored program, wherein the program executes any of the above methods when it is run.

[0090] Optionally, in this embodiment, the storage medium may be configured to store program codes for executing the following steps:

[0091] S1, receiving scheduling data uploaded by a target client, wherein the scheduling data includes: a device event that changes a device state corresponding to a target device, and real-time device states of other devices in the same network as the target device, and the target client has a communication relationship with the target device;

[0092] S2, when the scenario configuration associated with the target client is obtained, inputting the scheduling data and the scenario configuration into a decision-making model to obtain estimated action information output by the decision-making model;

[0093] S3, when determining the sub-application type that allows control operations on the target device, converting the estimated action information into an operation instruction corresponding to the sub-application type, and sending the operation instruction to the target client, wherein the target client is used to schedule the sub-application type to perform response control on the target device after a device event.

[0094] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0095] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0096] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0097] S1, receiving scheduling data uploaded by a target client, wherein the scheduling data includes: a device event that changes a device state corresponding to a target device, and real-time device states of other devices in the same network as the target device, and the target client has a communication relationship with the target device;

[0098] S2, when the scenario configuration associated with the target client is obtained, inputting the scheduling data and the scenario configuration into a decision-making model to obtain estimated action information output by the decision-making model;

[0099] S3, when determining the sub-application type that allows control operations on the target device, converting the estimated action information into an operation instruction corresponding to the sub-application type, and sending the operation instruction to the target client, wherein the target client is used to schedule the sub-application type to perform response control on the target device after a device event.

[0100] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0101] Figure 5 The computer system structure block diagram for implementing the electronic device of the embodiment of the present application is schematically shown. It should be noted that: Figure 5 The computer system 500 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application. Figure 5As shown, the computer system 500 includes a central processing unit 501 (CPU), which can perform various appropriate actions and processes according to the program stored in the read-only memory 502 (ROM) or the program loaded from the storage part 508 to the random access memory 503 (RAM). Various programs and data required for system operation are also stored in the random access memory 503. The central processing unit 501, the read-only memory 502 and the random access memory 503 are connected to each other through a bus 504. An input / output interface 505 (I nput / Output interface, i.e., I / O interface) is also connected to the bus 504.

[0102] The following components are connected to the input / output interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 505 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 55 is also connected to the input / output interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 55 as needed so that a computer program read therefrom is installed into the storage section 508 as needed.

[0103] In particular, according to an embodiment of the present application, the process described in each method flow chart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program contains a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processor 501, various functions defined in the system of the present application are executed.

[0104] According to another aspect of the embodiment of the present application, an electronic device for implementing the control of the above-mentioned device linkage is also provided. Figure 6 As shown, the electronic device includes a memory 602 and a processor 604. The memory 602 stores a computer program, and the processor 604 is configured to execute the steps in any of the above method embodiments through the computer program.

[0105] Optionally, in this embodiment, the electronic device may be located in at least one of a plurality of network devices of a computer network. A person skilled in the art may understand that Figure 6 The structure shown is for illustration only, and the electronic device may also be a device including the above-mentioned flash memory. Figure 6 The structure of the electronic device is not limited. Figure 6 More or fewer components (such as network interfaces, etc.) as shown in, or with Figure 6 Different configurations are shown.

[0106] Among them, the memory 602 can be used to store software programs and modules, such as the program instructions / modules corresponding to the device linkage control and the device in the embodiment of the present application. The processor 604 executes various functional applications and data processing by running the software programs and modules stored in the memory 602, that is, to realize the above-mentioned device linkage control. The memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 602 may further include a memory remotely located relative to the processor 604, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 602 can specifically be, but is not limited to, used for information such as logs containing modeling data. As an example, Figure 6 As shown, the memory 602 may include but is not limited to the modules in the device for controlling the linkage of the devices. In addition, it may also include but is not limited to other module units in the device for controlling the linkage of the devices, which will not be described in detail in this example.

[0107] Optionally, the transmission device 606 is used to receive or send data via a network. Specific examples of the network may include a wired network and a wireless network. In one example, the transmission device 606 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 606 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0108] In addition, the electronic device further includes: a display 606; and a connection bus 610, which is used to connect various module components in the electronic device.

[0109] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.

[0110] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0111] The embodiments of the present application also provide a computer program, which includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of any one of the above method embodiments.

[0112] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0113] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0114] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A device linkage control method, characterized in that: include: Receiving scheduling data uploaded by a target client, wherein the scheduling data includes: a device event that changes a device state corresponding to a target device, and real-time device states of other devices in the same network as the target device, and the target client has a communication relationship with the target device; When the scene configuration associated with the target client is obtained, the scheduling data and the scene configuration are input into the decision model to obtain the estimated action information output by the decision model; when the sub-application type that allows control operations on the target device is determined, the estimated action information is converted into an operation instruction corresponding to the sub-application type, and the operation instruction is sent to the target client, wherein the target client is used to schedule the sub-application type to perform response control on the target device after a device event.

2. The device linkage control method according to claim 1, characterized in that: After receiving the scheduling data uploaded by the target client, the method further includes: Extracting text information corresponding to the device event in the scheduling data; Based on the text information, a semantic search is performed in a cloud knowledge base to obtain target knowledge matching the text information, wherein the target knowledge includes instructions for use of the target device and operational question and answer data of the target device.

3. The device linkage control method according to claim 2, characterized in that: Before performing semantic retrieval in a cloud knowledge base based on the text information to obtain target knowledge matching the text information, the method further includes: In the case where the target client has completed the coordinated configuration of multiple sub-application types, obtaining the manufacturer data corresponding to each sub-application type through the communication channel between the target cloud and the Internet, wherein the manufacturer data includes data corresponding to at least two different devices; Parse the manufacturer data, and determine knowledge documents corresponding to different devices according to the parsing results, wherein the knowledge documents include at least one of the following: device instructions for different devices, and device problem answers for different devices; The knowledge document is vectorized, and the vectorization results of the different devices are summarized to obtain a cloud knowledge base.

4. The device linkage control method according to claim 1, characterized in that: In the case where the scenario configuration associated with the target client is obtained, the scheduling data and the scenario configuration are input into the decision-making model, and before obtaining the estimated action information output by the decision-making model, the method further includes: obtaining the historical scheduling data and the historical scenario configuration of the target device coordinated and controlled by the target client, and determining the device action set corresponding to the historical scheduling data and the historical scenario configuration; Establishing a data relationship between the historical scheduling data, the historical scene configuration, and the device action set; The preset large model is trained according to the data relationship to obtain the decision large model, wherein the preset large model is a deep learning model to be trained through a large amount of data relationships.

5. The device linkage control method according to claim 1, characterized in that: Converting the estimated action information into an operation instruction corresponding to the sub-application type includes: Determining the operation content corresponding to the estimated action information; Determine the conversion instruction format according to the operation content and the function list included in the multiple sub-application types that have completed the coordinated configuration of the target client; The operation content is converted using the conversion instruction format, and an operation condition is added to the conversion result to obtain the operation instruction.

6. The device linkage control method according to claim 1, characterized in that: In the case of determining the sub-application type that allows the control operation to be performed on the target device, converting the estimated action information into an operation instruction corresponding to the sub-application type, and before sending the operation instruction to the target client, the method further includes: In the case where it is determined that the sub-application type has at least two sub-applications, determining a working state of each sub-application; When the working status indicates that the first sub-application of the at least two sub-applications cannot be operated, the target client is instructed to schedule the second sub-application of the at least two sub-applications to respond to and control the target device after receiving the operation instruction.

7. The device linkage control method according to claim 1, characterized in that: In the case of determining the sub-application type that allows the control operation to be performed on the target device, converting the estimated action information into an operation instruction corresponding to the sub-application type, and sending the operation instruction to the target client, the method further includes: Acquire a control result of the target device performing response control after a device event occurs; When the control result conforms to the operation instruction, allowing the target client to continue to control the target device; In the case that the control result does not conform to the operation instruction, the target client is prohibited from continuously controlling the target device, and an alarm message of control abnormality is sent to the management object of the target device.

8. A control device for equipment linkage, characterized in that: include: A receiving module, configured to receive the scheduling data uploaded by the target client, wherein the scheduling data includes: a device event that changes the device state corresponding to the target device, and the real-time device state of other devices in the same network as the target device, and the target client has a communication relationship with the target device; An input module, for inputting the scheduling data and the scenario configuration into a decision-making model when the scenario configuration associated with the target client is obtained, to obtain estimated action information output by the decision-making model; A conversion module is used to convert the estimated action information into an operation instruction corresponding to the sub-application type when determining a sub-application type that allows control operations on the target device, and send the operation instruction to the target client, wherein the target client is used to schedule the sub-application type to respond to the device event of the target device.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 7 when executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

Images