Method and device for completing AI task in smart home equipment

Abstract

The invention provides a method and device for completing an AI task in smart home equipment. The method comprises the following steps: a receiving step: a master device receives an AI task; an allocation step: the main device allocates the AI task to at least one edge device according to the computing power balance of the edge devices and the computing power required by the AI task; in the output step, the main device obtains the calculation result output by the intelligent home device processing the AI task, and the method solves the problem that in the prior art, the response time for the intelligent home device to complete the AI task is long.

Description

Technical Field

[0001] The present application relates to the field of washing machine scheduling, and more specifically, to a method for completing an AI task in a smart home device, a completion device, a computer-readable storage medium, a processor, and an electronic device. Background Art

[0002] Cloud computing is a model that provides computing resources and services through the Internet. Users can access and use various computing resources on demand without having to manage the underlying infrastructure. Edge computing is a distributed computing model that stores computing and data close to the data source, aiming to reduce latency, bandwidth requirements and data center burden, and improve application response speed and efficiency.

[0003] A large part of the existing smart home devices on the market use cloud server computing to infer the AI ​​model of AI functions, but the inference time is very long because they all need to connect to the Internet, then transmit data to the cloud server, and then download the server's inference results, resulting in very poor real-time performance. In addition, when smart home devices complete AI tasks, sometimes the computing pressure is high and it is difficult to start execution immediately, resulting in a long response time for AI tasks, which is difficult to meet user needs.

[0004] Therefore, there is an urgent need for a method that can reduce the response time of smart home devices to complete AI tasks. Summary of the invention

[0005] The main purpose of this application is to provide a method, a completion device, a computer-readable storage medium, a processor and an electronic device for completing AI tasks in a smart home device, so as to at least solve the problem of long response time of smart home devices in completing AI tasks in the prior art.

[0006] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for completing an AI task in a smart home device is provided, wherein there are multiple smart home devices and they are in a local area network, and the multiple smart home devices include a main device and at least one edge device, and the edge device is the smart home device in the local area network other than the main device. The method includes: a receiving step, in which the main device receives the AI ​​task; an allocating step, in which the main device allocates the AI ​​task to at least one of the edge devices according to the computing power balance of the edge device and the computing power required for the AI ​​task; and an outputting step, in which the main device obtains the calculation result output by the smart home device that processes the AI ​​task.

[0007] Optionally, the allocation step includes: at least calculating the sum of the computing power balances of the edge devices to obtain the sum of the computing power balances; dividing the sum of the computing power balances into N computing power units, wherein at least the computing power balance of each edge device is composed of an integer number of computing power units; dividing the AI ​​task into N subtasks according to the total computing power required by the AI ​​task, and obtaining the computing power required for each subtask as [M / N], wherein M is the computing power required for the AI ​​task, and the number of subtasks is the same as the number of computing power units; at least allocating the subtasks to the smart home devices with the computing power balance, and the number of subtasks allocated to at least some of the smart home devices is the same as the number of computing power units of the smart home devices with the computing power balance.

[0008] Optionally, at least the subtask is assigned to the smart home device with the computing power surplus, including: assigning the same number of subtasks among the N subtasks as the number of the computing power units of the smart home devices to the corresponding smart home devices, and the subtasks correspond to the computing power units one-to-one.

[0009] Optionally, at least the subtask is assigned to the smart home device with the computing power surplus, and it also includes: when M / N is not an integer, calculating the remaining required computing power n according to the formula n=M-[M / N]╳N, and the remaining required computing power is the remaining computing power n after the AI ​​task is divided into the subtasks; and allocating the remaining required computing power to the main device.

[0010] Optionally, allocating the remaining required computing power to the master device includes: adding the remaining required computing power to a task waiting list; and after the master device completes the corresponding subtask, allocating the remaining required computing power in the task waiting list to the master device.

[0011] Optionally, at least the sum of the computing power balances of the edge devices is calculated to obtain the sum of the computing power balances, including: calculating the sum of the computing power balances of the main device and the computing power balances of the edge devices to obtain the sum of the computing power balances.

[0012] Optionally, in the case that the obtained calculation result is abnormal, the method further includes: repeating the receiving step, the assigning step and the outputting step in sequence at least once until the calculation result is not abnormal.

[0013] According to another aspect of the present application, a device for completing an AI task in a smart home device is provided, wherein the smart home devices are multiple and in a local area network, the multiple smart home devices include a main device and at least one edge device, and the edge device is the smart home device in the local area network except the main device, and the device includes:

[0014] A receiving unit, configured to execute a receiving step, wherein the master device receives the AI ​​task;

[0015] an allocating unit, configured to execute an allocating step, wherein the master device allocates the AI ​​task to at least one of the edge devices according to the computing power balance of the edge device and the computing power required by the AI ​​task;

[0016] An output unit is used to perform an output step, wherein the main device obtains a calculation result output by the smart home device that processes the AI ​​task.

[0017] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is run, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned methods for completing AI tasks in smart home devices.

[0018] According to another aspect of the present application, a processor is provided, wherein the processor is used to run a program, wherein when the program is run, the program executes any one of the above-mentioned methods for completing an AI task in a smart home device.

[0019] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for completing an AI task in any one of the smart home devices described above.

[0020] Optionally, the electronic device is a washing machine.

[0021] By applying the technical solution of the present application, first, a main device in a smart home device receives an AI task; secondly, the main device distributes the AI ​​task to at least one of the edge devices according to the computing power balance of the main device, the computing power balance of the edge device, and the computing power required for the AI ​​task; finally, the main device obtains the calculation result output by the smart home device that processes the AI ​​task. Compared with the method in the prior art of uploading all task data and AI models to a cloud server, using the cloud server to process all tasks, and then transmitting the results of the AI ​​tasks to smart home appliances, in the present application, the main device receives the AI ​​task and assigns the AI ​​task to at least one edge device based on the computing power surplus of the edge device and the computing power required for the AI ​​task. After the edge device processes the assigned AI task, it sends the processing result to the main device, and the main device obtains all the processing results to complete the AI ​​task. In this way, when the computing pressure of the AI ​​task is high and the main device cannot execute it immediately, the main device transfers at least part of the AI ​​task to the edge device with computing power surplus, which alleviates the computing pressure of the main device, thereby greatly reducing the time to complete the AI ​​task, thereby solving the problem of long response time for smart home devices to complete AI tasks in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0023] Figure 1 A hardware structure block diagram of a mobile terminal for executing a method for completing an AI task in a smart home device provided in an embodiment of the present application is shown;

[0024] Figure 2 A flowchart of a method for completing an AI task in a smart home device provided according to an embodiment of the present application is shown;

[0025] Figure 3 A smart home device deployment diagram showing a method for completing an AI task in a smart home device provided in an embodiment of the present application is shown;

[0026] Figure 4 A process diagram showing the allocation of computing power of a master device according to a method for completing an AI task in a smart home device provided in an embodiment of the present application is shown;

[0027] Figure 5 A main device workflow diagram of a method for completing an AI task in a smart home device provided according to an embodiment of the present application is shown;

[0028] Figure 6A structural block diagram of a device for completing an AI task in a smart home device provided according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0029] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0030] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] As introduced in the background technology, the response time of smart home devices in the prior art to complete AI tasks is long. To solve the above problems, the embodiments of the present application provide a method for completing AI tasks in a smart home device, a completion device, a computer-readable storage medium, a processor and an electronic device.

[0033] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0034] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 1 is a hardware structure block diagram of a mobile terminal of a method for completing an AI task in a smart home device according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0035] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the display method of device information in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The above-mentioned specific examples of the network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0036] In this embodiment, a method for completing an AI task in a smart home device running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0037] Figure 2 1 is a flow chart of a method for completing an AI task in a smart home device according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0038] Step S201, a receiving step, the master device receives an AI task.

[0039] Specifically, the main device is set as a washing machine. There will be relatively more AI tasks in the washing machine, such as clothing classification and recognition, trajectory recognition, clothing trajectory segmentation, fault prediction, etc., and its own motion logic is also relatively complex, and its computing power is relatively high. Among them, clothing classification can use resnet53, trajectory recognition can use deepsort, segmentation can use yolov5-seg, and fault detection can use a machine learning classification model.

[0040] Specifically, using a static IP, the IP address can be fixed in the WIFI module of the home appliance at the factory.

[0041] Specifically, smart home appliances generally use ARM's cortex-A series chips, which belong to the ARMv7 or v8 architecture and can install the Linux system.

[0042] Step S202, an allocation step, the master device allocates the AI ​​task to at least one of the edge devices according to the computing power balance of the edge device and the computing power required for the AI ​​task.

[0043] Specifically, the main device is set as a washing machine. When the washing machine starts an artificial intelligence task (such as clothing classification recognition, trajectory recognition, fault prediction, etc.), the washing machine will act as the network manager (master) to access the working status of each node one by one, such as whether the TV is working properly and whether it is executing AI functions such as voice control; whether the oven is working properly and whether it is executing AI functions such as identifying whether the food is cooked; whether the refrigerator is working properly and whether it is executing AI functions such as identifying the freshness of stored items. The AI ​​functions of these smart home appliances are not executed all the time, so we can use these idle computing powers to increase the computing speed of the washing machine. If these smart home appliances are executing AI functions and there is no excess computing power, they will return not being called; if they are not being executed, or some AI functions are being executed, they will return to being callable and calculating the computing power balance.

[0044] Specifically, Figure 3 As shown, the master device master is set as a washing machine, which is divided into different working nodes. The different working nodes can be televisions, air conditioners, refrigerators, cameras, etc.

[0045] Step S203, an output step, the main device obtains the calculation result output by the smart home device that processes the AI ​​task.

[0046] Specifically, in a smart home network, after each node completes the assigned AI task calculation, a mechanism is needed to feed back the calculation results to the main device (i.e., the washing machine, as the master node). When multiple smart home devices execute AI tasks, the multiple calculation results output by multiple smart home devices are aggregated to generate the final calculation result, with one smart home device corresponding to one calculation result.

[0047] In the above embodiment, first, the main device in the smart home device receives the AI ​​task; secondly, the main device distributes the AI ​​task to at least one of the edge devices according to the computing power balance of the main device, the computing power balance of the edge device and the computing power required for the AI ​​task; finally, the main device obtains the calculation result output by the smart home device that processes the AI ​​task. Compared with the method in the prior art of uploading all task data and AI models to a cloud server, using the cloud server to process all tasks, and then transmitting the results of the AI ​​tasks to smart home appliances, in the present application, the main device receives the AI ​​task and assigns the AI ​​task to at least one edge device based on the computing power surplus of the edge device and the computing power required for the AI ​​task. After the edge device processes the assigned AI task, it sends the processing result to the main device, and the main device obtains all the processing results to complete the AI ​​task. In this way, when the computing pressure of the AI ​​task is high and the main device cannot execute it immediately, the main device transfers at least part of the AI ​​task to the edge device with computing power surplus, which alleviates the computing pressure of the main device, thereby greatly reducing the time to complete the AI ​​task, thereby solving the problem of long response time for smart home devices to complete AI tasks in the prior art.

[0048] In one embodiment of the present application, the allocation step includes: at least calculating the sum of the computing power surplus of the edge devices to obtain the sum of computing power surplus; dividing the sum of computing power surplus into N computing power units on average, wherein at least the computing power surplus of each edge device is composed of an integer number of computing power units; dividing the AI ​​task into N subtasks according to the total computing power required by the AI ​​task, and obtaining the required computing power of each subtask as [M / N], wherein M is the computing power required by the AI ​​task, and the number of subtasks is the same as the number of computing power units; at least allocating the subtasks to the smart home devices with the computing power surplus, and at least part of the smart home devices are allocated to the same number of subtasks as the number of computing power units of the smart home devices with the computing power surplus. In this method, by calculating the sum of the computing power surplus of the main device and the edge device, the AI ​​task is decomposed into multiple subtasks, and the subtasks are allocated to the smart home devices according to the computing power surplus of the devices, so that the computing power resources of all devices can be more fully utilized, so that the tasks can be completed more efficiently.

[0049] Specifically, for the calculation of the computing power balance of the main device: first, the washing machine needs to accurately evaluate its own computing power balance, which usually involves real-time monitoring of the resource consumption of all currently running tasks and applications, and subtracting the computing power required for the tasks it is executing or about to execute to obtain the current computing power balance.

[0050] Specifically, for edge device computing power balance calculation: the washing machine queries all smart home appliances (such as smart refrigerators, smart TVs, smart air conditioners, etc.) in the home network through network communication for their current working status and computing power usage. Each smart home appliance returns its computing power balance, and the washing machine aggregates this information to calculate the total computing power balance of all edge devices.

[0051] Specifically, the amount of data transmitted between home appliances is not large, but the real-time performance is high. The total amount of data transmitted is at most the number of AI tasks of a washing machine. This can be achieved without a router connected to the Internet. The amount of data transmitted is very different from the amount of data that can be transmitted by the local bandwidth of a normal router in the home, which is sufficient for data transmission.

[0052] In order to further ensure that each device can share the corresponding amount of tasks, thereby further ensuring that the entire task can be completed more efficiently, and further improving the efficiency of task execution, in one embodiment of the present application, at least the subtask is assigned to the smart home device with the computing power surplus, including: assigning the same number of subtasks as the number of computing power units of the smart home device among the N subtasks to the corresponding smart home devices, and the subtasks correspond to the computing power units one-to-one.

[0053] Specifically, Figure 4 As shown in the figure, the main device will combine its own computing power, add up all the computing power, and get N units of computing power as shown in the figure, and then divide the AI ​​task to be executed into N equal subtasks as much as possible, and evenly distribute them to each computing power unit. When each edge device node receives the computing power task it needs to complete, it will immediately execute the result and then feed back the calculation result to the washing machine. After this allocation, the AI ​​task to be executed will be completed at the fastest speed.

[0054] Specifically, Figure 5As shown in the figure, the main device is set as a washing machine, and the lightweight version of kubernetes (k3s) technology is used to schedule other edge devices in the home. The main workflow is as follows: the washing machine AI task (such as clothing classification recognition, trajectory recognition, fault prediction, etc.) starts; the washing machine, as a network manager (master), visits the working status of each node (node, i.e. edge device); the washing machine summarizes the working status and computing power balance of each node; the washing machine divides the AI ​​task into N jobs; the washing machine (master) combines its own computing power to assign computing tasks to each node (node) in the fastest way to solve the task; each node (node) jointly completes the computing power task; the washing machine (master) summarizes the computing results of each node (node); finally, the washing machine (master) outputs the calculation results.

[0055] In order to further improve the overall computing efficiency, thereby further reducing the task execution time, and further improving the performance of smart home devices, in one embodiment of the present application, at least the subtask is assigned to the smart home device with the computing power surplus, and also includes: when M / N is not an integer, calculating the remaining required computing power n according to the formula n=M-[M / N]╳N, and the remaining required computing power is the remaining computing power n after the AI ​​task is divided into the subtasks; and allocating the remaining required computing power to the main device.

[0056] In another implementation of the present application, the computing power required for all tasks can be divided by the computing power balance of all edge devices to obtain an integer part and a remainder part, wherein the integer part is the number of subtasks into which the AI ​​task is divided, and the remainder part is the computing power remaining after the subtasks are divided according to the integer part. According to the number of computing power units corresponding to the computing power balance of each edge device, the corresponding number of subtasks are allocated to the corresponding edge devices, and the remaining computing power of the AI ​​task corresponding to the remainder part is divided to the main device.

[0057] In another embodiment, allocating the remaining required computing power to the master device includes: adding the remaining required computing power to a task waiting list; after the master device completes the corresponding subtask, allocating the remaining required computing power in the task waiting list to the master device. In this method, by adding the remaining required computing power to the task waiting list, it can be further ensured that the master device can continue to process the next task after completing the current subtask without being idle. At the same time, allocating the remaining computing power to the master device can also improve the processing efficiency of the task, reduce the waiting time, and thus complete all AI tasks faster, thereby more effectively utilizing the computing power resources of the master device, and further ensuring that all AI tasks can be processed.

[0058] In another embodiment, at least the sum of the computing power balances of the edge devices is calculated to obtain the sum of the computing power balances, including: calculating the sum of the computing power balances of the main device and the computing power balances of the edge devices to obtain the sum of the computing power balances. In this method, the sum of the computing power balances of the main device and the edge devices is calculated to obtain the overall sum of the computing power balances, thereby better evaluating the computing power status of the entire system.

[0059] Specifically, the main device is set as a washing machine, and the artificial intelligence model running on the washing machine is directly used as a model with more parameters. For example, when detecting clothing, the models used for inference are yolov5n.pt, yolov5s.pt, yolov5m.pt, yolov5l.pt, and yolov5x.pt. The difference is that the number of parameters from yolov5n.pt to yolov5x.pt gradually increases, and more and more computing power is required, but the accuracy will also increase. Previously, due to the limited computing resources of edge devices, only the smallest yolov5n.pt model was generally used. This method can integrate and connect edge computing resources, so that a larger model can be used, making the recognition accuracy higher; it can also distribute the existing model computing tasks in the fastest way, making the prediction time shorter and improving work efficiency.

[0060] In another embodiment, when the calculation result is abnormal, the method further includes: repeating the receiving step, the allocating step and the outputting step at least once in sequence until the calculation result is normal. In this method, by repeatedly performing the receiving, analyzing and outputting steps until a normal calculation result is obtained, it can further ensure that the system can automatically retry when an abnormal situation occurs, thereby further ensuring that the correct calculation result is finally obtained.

[0061] Specifically, if a home appliance suddenly loses power, it will be fed back an error and the task will be reallocated.

[0062] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the method for completing AI tasks in the smart home device of the present application will be described in detail below in combination with specific embodiments.

[0063] This embodiment relates to a specific method for completing an AI task in a smart home device, comprising the following steps:

[0064] Step 1: The master device receives the AI ​​task and calculates the computing power required to complete the AI ​​task;

[0065] Step 2: Sum up the computing power balance of the main device and the computing power balance of the edge device to obtain the total computing power balance;

[0066] Step 3: Divide the total computing power balance into N computing power units on average, and divide the AI ​​task into N subtasks according to the total computing power required by the AI ​​task. The computing power corresponding to each subtask is M / N, where M is the computing power required to complete the AI ​​task.

[0067] Step 4: Allocate N subtasks to corresponding smart home devices according to the number of computing power units possessed by each smart home device with computing power surplus, wherein the number of subtasks allocated to each smart home device with computing power surplus is the same as the number of computing power units possessed by it;

[0068] Step 5: When [M / N] is not an integer, N subtasks are allocated to corresponding smart home devices according to the number of computing power units of each smart home device with computing power surplus, wherein the number of subtasks allocated to each smart home device with computing power surplus is the same as the number of computing power units it has, and the remaining computing power M-[M / N]╳N is allocated to the main device;

[0069] Step 6: The edge device with surplus computing power and assigned subtasks transmits the computing results to the main device after completing the subtasks;

[0070] Step 7: The main device receives the calculation results of all edge devices, aggregates them with the calculation results of the main device to generate the final calculation result, and outputs the final result.

[0071] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0072] The embodiment of the present application also provides a device for completing AI tasks in a smart home device. It should be noted that the device for completing AI tasks in a smart home device in the embodiment of the present application can be used to execute the method for completing AI tasks in a smart home device provided in the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and those that have been described will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0073] The following introduces the device for completing AI tasks in the smart home device provided in the embodiment of the present application.

[0074] Figure 6 Schematic diagram of a device for completing an AI task in a smart home device according to an embodiment of the present application. Figure 6 As shown, the device comprises:

[0075] The receiving unit 10 is used to execute the receiving step, and the above-mentioned main device receives the AI ​​task.

[0076] Specifically, the main device is set as a washing machine. There will be relatively more AI tasks in the washing machine, such as clothing classification and recognition, trajectory recognition, clothing trajectory segmentation, fault prediction, etc., and its own motion logic is also relatively complex, and its computing power is relatively high. Among them, clothing classification can use resnet53, trajectory recognition can use deepsort, segmentation can use yolov5-seg, and fault detection can use a machine learning classification model.

[0077] Specifically, using a static IP, the IP address can be fixed in the WIFI module of the home appliance at the factory.

[0078] Specifically, smart home appliances generally use ARM's cortex-A series chips, which belong to the ARMv7 or v8 architecture and can install the Linux system.

[0079] The allocation unit 20 is used to execute the allocation step, and the above-mentioned main device allocates the above-mentioned AI task to at least one of the above-mentioned edge devices according to the computing power balance of the above-mentioned edge devices and the computing power required by the above-mentioned AI task.

[0080] Specifically, the main device is set as a washing machine. When the washing machine starts an artificial intelligence task (such as clothing classification recognition, trajectory recognition, fault prediction, etc.), the washing machine will act as the network manager (master) to access the working status of each node one by one, such as whether the TV is working properly and whether it is executing AI functions such as voice control; whether the oven is working properly and whether it is executing AI functions such as identifying whether the food is cooked; whether the refrigerator is working properly and whether it is executing AI functions such as identifying the freshness of stored items. The AI ​​functions of these smart home appliances are not executed all the time, so we can use these idle computing powers to increase the computing speed of the washing machine. If these smart home appliances are executing AI functions and there is no excess computing power, they will return not being called; if they are not being executed, or some AI functions are being executed, they will return to being callable and calculating the computing power balance.

[0081] The output unit 30 is used to execute the output step, and the above-mentioned main device obtains the calculation result output by the above-mentioned smart home device that processes the above-mentioned AI task.

[0082] Specifically, in a smart home network, after each node completes the assigned AI task calculation, a mechanism is needed to feed back the calculation results to the main device (i.e., the washing machine, as the master node). When multiple smart home devices execute AI tasks, the multiple calculation results output by multiple smart home devices are aggregated to generate the final calculation result, with one smart home device corresponding to one calculation result.

[0083] In the above embodiment, first, the main device in the smart home device receives the AI ​​task; secondly, the main device distributes the AI ​​task to at least one of the edge devices according to the computing power balance of the main device, the computing power balance of the edge device and the computing power required for the AI ​​task; finally, the main device obtains the calculation result output by the smart home device that processes the AI ​​task. Compared with the method in the prior art of uploading all task data and AI models to a cloud server, using the cloud server to process all tasks, and then transmitting the results of the AI ​​tasks to smart home appliances, in the present application, the main device receives the AI ​​task and assigns the AI ​​task to at least one edge device based on the computing power surplus of the edge device and the computing power required for the AI ​​task. After the edge device processes the assigned AI task, it sends the processing result to the main device, and the main device obtains all the processing results to complete the AI ​​task. In this way, when the computing pressure of the AI ​​task is high and the main device cannot execute it immediately, the main device transfers at least part of the AI ​​task to the edge device with computing power surplus, which alleviates the computing pressure of the main device, thereby greatly reducing the time to complete the AI ​​task, thereby solving the problem of long response time for smart home devices to complete AI tasks in the prior art.

[0084] In one embodiment of the present application, the allocation unit includes a calculation subunit, a division subunit, a first allocation subunit and a second allocation subunit, wherein the calculation subunit is used to at least calculate the sum of the computing power surpluses of the edge devices to obtain the sum of the computing power surpluses; the division subunit is used to divide the sum of the computing power surpluses into N computing power units on average, wherein at least the computing power surplus of each of the edge devices is composed of an integer number of computing power units; the first allocation subunit is used to divide the AI ​​task into N subtasks on average according to the total computing power required by the AI ​​task, and obtain the computing power required for each of the subtasks as [M / N], wherein M is the computing power required for the AI ​​task, and the number of the subtasks is the same as the number of the computing power units; the second allocation subunit is used to at least allocate the subtasks to the smart home devices with the computing power surplus, and the number of the subtasks allocated to at least some of the smart home devices is the same as the number of the computing power units of the smart home devices with the computing power surplus. In this solution, the AI ​​task is decomposed into multiple subtasks by calculating the total computing power balance of the main device and the edge device, and the subtasks are assigned to smart home devices based on the computing power balance of the device, so that the computing power resources of all devices can be more fully utilized and the task can be completed more efficiently.

[0085] Specifically, for the calculation of the computing power balance of the main device: first, the washing machine needs to accurately evaluate its own computing power balance, which usually involves real-time monitoring of the resource consumption of all currently running tasks and applications, and subtracting the computing power required for the tasks it is executing or about to execute to obtain the current computing power balance.

[0086] Specifically, for edge device computing power balance calculation: the washing machine queries all smart home appliances (such as smart refrigerators, smart TVs, smart air conditioners, etc.) in the home network through network communication for their current working status and computing power usage. Each smart home appliance returns its computing power balance, and the washing machine aggregates this information to calculate the total computing power balance of all edge devices.

[0087] Specifically, the amount of data transmitted between home appliances is not large, but the real-time performance is high. The total amount of data transmitted is at most the number of AI tasks of a washing machine. This can be achieved without a router connected to the Internet. The amount of data transmitted is very different from the amount of data that can be transmitted by the local bandwidth of a normal router in the home, which is sufficient for data transmission.

[0088] In order to further ensure that each device can share the corresponding amount of tasks, thereby further ensuring that the entire task can be completed more efficiently, and further improving the efficiency of task execution, in one embodiment of the present application, the second allocation subunit includes a first allocation module, wherein the first allocation module is used to allocate the same number of subtasks among the N subtasks as the number of computing power units of the smart home devices to the corresponding smart home devices, and the subtasks correspond one-to-one to the computing power units.

[0089] In order to further improve the overall computing efficiency, thereby further reducing the task execution time, and further improving the performance of smart home devices, in one embodiment of the present application, the second allocation subunit also includes a first calculation module and a second allocation module, wherein the first calculation module is used to calculate the remaining required computing power n according to the formula n=M-[M / N]╳N when M / N is not an integer, and the remaining required computing power is the remaining computing power n after the AI ​​task is divided into the subtasks; the second allocation module is used to allocate the remaining required computing power to the main device.

[0090] In another implementation of the present application, the computing power required for all tasks can be divided by the computing power balance of all edge devices to obtain an integer part and a remainder part, wherein the integer part is the number of subtasks into which the AI ​​task is divided, and the remainder part is the computing power remaining after the subtasks are divided according to the integer part. According to the number of computing power units corresponding to the computing power balance of each edge device, the corresponding number of subtasks are allocated to the corresponding edge devices, and the remaining computing power of the AI ​​task corresponding to the remainder part is divided to the main device.

[0091] In another embodiment, the second allocation module includes a adding submodule and an allocating submodule, wherein the adding submodule is used to add the above-mentioned remaining required computing power to the task waiting list; the allocating submodule is used to allocate the above-mentioned remaining required computing power in the above-mentioned task waiting list to the above-mentioned main device after the above-mentioned main device completes the corresponding above-mentioned subtask. In this method, by adding the remaining required computing power to the task waiting list, it can be further ensured that the main device can continue to process the next task after completing the current subtask without being idle. At the same time, allocating the remaining computing power to the main device can also improve the processing efficiency of the task, reduce the waiting time, and thus complete all AI tasks faster, thereby more effectively utilizing the computing power resources of the main device, and further ensuring that all AI tasks can be processed.

[0092] In another embodiment, the computing subunit includes a second computing module, wherein the second computing module is used to calculate the sum of the computing power balance of the above-mentioned main device and the computing power balance of the above-mentioned edge device to obtain the above-mentioned computing power balance sum. In this solution, by calculating the sum of the computing power balance of the main device and the edge device, the overall computing power balance sum is obtained, thereby better evaluating the computing power status of the entire system.

[0093] Specifically, the main device is set as a washing machine, and the artificial intelligence model running on the washing machine is directly used as a model with more parameters. For example, when detecting clothing, the models used for inference are yolov5n.pt, yolov5s.pt, yolov5m.pt, yolov5l.pt, and yolov5x.pt. The difference is that the number of parameters from yolov5n.pt to yolov5x.pt gradually increases, and more and more computing power is required, but the accuracy will also increase. Previously, due to the limited computing resources of edge devices, only the smallest yolov5n.pt model was generally used. This method can integrate and connect edge computing resources, so that a larger model can be used, making the recognition accuracy higher; it can also distribute the existing model computing tasks in the fastest way, making the prediction time shorter and improving work efficiency.

[0094] In another embodiment, the output unit further includes a repeating module, wherein the repeating module is used to sequentially repeat the receiving step, the allocating step, and the outputting step at least once until the calculation result is normal. In this solution, by repeatedly performing the receiving, analyzing, and outputting steps until a normal calculation result is obtained, it can be further ensured that the system can automatically retry when an abnormal situation occurs, thereby further ensuring that the correct calculation result is finally obtained.

[0095] Specifically, if a home appliance suddenly loses power, it will be fed back an error and the task will be reallocated.

[0096] The device for completing the AI ​​task in the above-mentioned smart home device includes a processor and a memory. The above-mentioned receiving unit, dividing unit and output unit are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize the corresponding functions. The above-mentioned modules are all located in the same processor; or, the above-mentioned modules are located in different processors in any combination.

[0097] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set to solve the problem of long response time of smart home devices completing AI tasks in the prior art by adjusting kernel parameters.

[0098] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0099] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is running, a device where the computer-readable storage medium is located is controlled to perform a method for completing an AI task in the smart home device, the method comprising:

[0100] Step S201, a receiving step, the master device receives an AI task.

[0101] Step S202, an allocation step, the master device allocates the AI ​​task to at least one of the edge devices according to the computing power balance of the edge device and the computing power required for the AI ​​task.

[0102] Step S203, an output step, the main device obtains the calculation result output by the smart home device that processes the AI ​​task.

[0103] An embodiment of the present invention provides a processor, the processor is used to run a program, wherein when the program is run, a method for completing an AI task in the smart home device is executed, the method comprising:

[0104] Step S201, a receiving step, the master device receives an AI task.

[0105] Step S202, an allocation step, the master device allocates the AI ​​task to at least one of the edge devices according to the computing power balance of the edge device and the computing power required for the AI ​​task.

[0106] Step S203, an output step, the main device obtains the calculation result output by the smart home device that processes the AI ​​task.

[0107] An embodiment of the present invention provides a device, the device including a processor, a memory, and a program stored in the memory and executable on the processor, and when the processor executes the program, at least the following steps are implemented:

[0108] Step S201, a receiving step, the master device receives an AI task.

[0109] Step S202, an allocation step, the master device allocates the AI ​​task to at least one of the edge devices according to the computing power balance of the edge device and the computing power required for the AI ​​task.

[0110] Step S203, an output step, the main device obtains the calculation result output by the smart home device that processes the AI ​​task.

[0111] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0112] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program for initializing at least the following method steps:

[0113] Step S201, a receiving step, the master device receives an AI task.

[0114] Step S202, an allocation step, the master device allocates the AI ​​task to at least one of the edge devices according to the computing power balance of the edge device and the computing power required for the AI ​​task.

[0115] Step S203, an output step, the main device obtains the calculation result output by the smart home device that processes the AI ​​task.

[0116] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0117] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0118] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0119] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0120] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0121] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0122] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0123] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0124] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0125] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0126] In the above embodiment, first, the main device in the smart home device receives the AI ​​task; secondly, the main device distributes the AI ​​task to at least one of the edge devices according to the computing power balance of the main device, the computing power balance of the edge device and the computing power required for the AI ​​task; finally, the main device obtains the calculation result output by the smart home device that processes the AI ​​task. Compared with the method in the prior art of uploading all task data and AI models to a cloud server, using the cloud server to process all tasks, and then transmitting the results of the AI ​​tasks to smart home appliances, in the present application, the main device receives the AI ​​task and assigns the AI ​​task to at least one edge device based on the computing power surplus of the edge device and the computing power required for the AI ​​task. After the edge device processes the assigned AI task, it sends the processing result to the main device, and the main device obtains all the processing results to complete the AI ​​task. In this way, when the computing pressure of the AI ​​task is high and the main device cannot execute it immediately, the main device transfers at least part of the AI ​​task to the edge device with computing power surplus, which alleviates the computing pressure of the main device, thereby greatly reducing the time to complete the AI ​​task, thereby solving the problem of long response time for smart home devices to complete AI tasks in the prior art.

[0127] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for completing an AI task in a smart home device, characterized in that: There are multiple smart home devices in a local area network, the multiple smart home devices include a main device and at least one edge device, the edge device is the smart home device in the local area network except the main device, and the method includes: Receiving step, the master device receives the AI ​​task; an allocating step, wherein the master device allocates the AI ​​task to at least one of the edge devices according to the computing power balance of the edge devices and the computing power required by the AI ​​task; In the output step, the main device obtains the calculation result output by the smart home device that processes the AI ​​task.

2. The method for completing AI tasks in smart home devices according to claim 1, characterized in that: The allocation step comprises: At least a sum of the computing power balances of the edge devices is calculated to obtain a sum of the computing power balances; Divide the sum of the computing power balances into N computing power units on average, wherein at least the computing power balance of each edge device consists of an integer number of computing power units; According to the total computing power required by the AI ​​task, the AI ​​task is evenly divided into N subtasks, and the computing power required for each subtask is obtained as [M / N], where M is the computing power required by the AI ​​task, and the number of subtasks is the same as the number of computing power units; At least the subtasks are allocated to the smart home devices with the computing power surplus, and the number of the subtasks allocated to at least some of the smart home devices is the same as the number of computing power units possessed by the smart home devices with the computing power surplus.

3. The method for completing AI tasks in smart home devices according to claim 2, characterized in that: At least allocating the subtask to the smart home device having the computing power surplus includes: The subtasks of the N subtasks, the same number as the number of the computing power units of the smart home devices, are allocated to the corresponding smart home devices, and the subtasks correspond to the computing power units one by one.

4. The method for completing AI tasks in a smart home device according to claim 3, characterized in that: At least allocating the subtask to the smart home device having the computing power surplus further includes: When M / N is not an integer, the remaining required computing power n is calculated according to the formula n=M-[M / N]╳N, where the remaining required computing power is the remaining computing power n after the AI ​​task is divided into the subtasks; The remaining required computing power is allocated to the master device.

5. The method for completing AI tasks in a smart home device according to claim 4, characterized in that: Allocating the remaining required computing power to the master device includes: Add the remaining required computing power to the task waiting list; After the master device completes the corresponding subtask, the remaining required computing power in the task waiting list is allocated to the master device.

6. The method for completing AI tasks in a smart home device according to claim 2, characterized in that: At least the sum of the computing power balances of the edge devices is calculated to obtain the sum of the computing power balances, including: The sum of the computing power balance of the master device and the computing power balance of the edge device is calculated to obtain the sum of the computing power balance.

7. The method for completing AI tasks in smart home devices according to claim 1, characterized in that: In the case where the obtained calculation result is abnormal, the method further includes: The receiving step, the allocating step and the outputting step are sequentially repeated at least once until the calculation result is not abnormal.

8. A device for completing AI tasks in a smart home device, characterized in that: There are multiple smart home devices in a local area network, the multiple smart home devices include a main device and at least one edge device, the edge device is the smart home device in the local area network except the main device, and the device includes: A receiving unit, configured to execute a receiving step, wherein the master device receives the AI ​​task; an allocating unit, configured to execute an allocating step, wherein the master device allocates the AI ​​task to at least one of the edge devices according to the computing power balance of the edge device and the computing power required by the AI ​​task; An output unit is used to perform an output step, wherein the main device obtains a calculation result output by the smart home device that processes the AI ​​task.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for completing an AI task in a smart home device according to any one of claims 1 to 7.

10. A processor, characterized in that: The processor is used to run a program, wherein the program, when running, executes the method for completing an AI task in a smart home device as described in any one of claims 1 to 7.

11. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for completing an AI task in a smart home device as described in any one of claims 1 to 7.

12. The electronic device according to claim 11, characterized in that: The electronic device is a washing machine.

Images