Task processing method and device, equipment and medium

Abstract

The invention relates to the technical field of data processing, in particular to a task processing method and device, equipment and a medium. In the embodiment of the invention, the household robot can identify the complex task and split the complex task to obtain the simple sub-task and the complex sub-task, then the household robot executes the simple sub-task, the simple reasoning task is executed locally, the demand for computing resources of a cloud server is reduced, and the computing efficiency of the cloud server is improved. Therefore, the use cost of the cloud server is reduced. The overall operation cost of the household robot system is reduced, and especially for household robot equipment which needs to run for a long time, a large amount of cloud computing resource overhead is saved. The cloud server executes the complex subtasks, the load of the home robot is reduced, the utilization rate of computing resources is optimized, resource waste is avoided, meanwhile, it is ensured that the complex tasks can be supported by the powerful computing power of the cloud server, and the execution efficiency and accuracy of the complex tasks are improved.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and particularly to a task processing method, apparatus, device and medium. Background Art

[0002] With the development of technology, home robots are widely used. A large model is deployed inside the home robot, enabling it to respond to user tasks. That is, the user sends a task to the home robot, and after receiving the task, the home robot calls the large model to execute the task.

[0003] However, in the prior art, when there are complex tasks, the home robot needs to spend a lot of time processing them, resulting in a slow response speed. Moreover, due to the complexity of the tasks, the final inference results are not accurate either. Summary of the Invention

[0004] This application provides a task processing method, apparatus, device and medium to solve the problems of long processing time, slow response speed and inaccurate inference results when the existing home robot processes complex tasks.

[0005] In a first aspect, an embodiment of this application provides a task processing method, and the method includes:

[0006] If it is determined, according to a preset classification algorithm, that the received target task is a complex task, then decompose the target task to determine each subtask corresponding to the target task;

[0007] According to the classification algorithm, identify each complex subtask and each simple subtask in each of the subtasks;

[0008] Process each of the simple subtasks to obtain a first inference result; and send each of the complex subtasks to a cloud server for processing, and receive a second inference result of each of the complex subtasks sent by the cloud server;

[0009] Call the large model to fuse the first inference result and the second inference result, and determine and output a target inference result of the target task.

[0010] In a second aspect, an embodiment of this application further provides a task processing apparatus, and the apparatus includes:

[0011] A processing module, configured to, if it is determined, according to a preset classification algorithm, that the received target task is a complex task, then decompose the target task to determine each subtask corresponding to the target task; according to the classification algorithm, identify each complex subtask and each simple subtask in each of the subtasks;

[0012] A calling module, configured to process each of the simple subtasks to obtain a first inference result; and send each of the complex subtasks to a cloud server for processing, and receive a second inference result of each of the complex subtasks sent by the cloud server; call the large model to fuse the first inference result and the second inference result, and determine and output a target inference result of the target task.

[0013] In a third aspect, an embodiment of the present application further provides an electronic device, which at least includes a processor and a memory. The processor is configured to implement the steps of the task processing method as described in any one of the above when executing a computer program stored in the memory.

[0014] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. The computer program is configured to implement the steps of the task processing method as described in any one of the above when executed by a processor.

[0015] In an embodiment of the present application, if it is determined according to a preset classification algorithm that a received target task is a complex task, the large model is called to decompose the target task to determine each subtask corresponding to the target task; according to the classification algorithm, each complex subtask and each simple subtask in each subtask are identified; the large model is called to process each simple subtask to obtain a first inference result; and each complex subtask is sent to a cloud server for processing, and a second inference result of each complex subtask sent by the cloud server is received; the large model is called to fuse the first inference result and the second inference result, and determine and output a target inference result of the target task. In an embodiment of the present application, a home robot can identify a complex task, decompose the complex task to obtain simple subtasks and complex subtasks, and then the home robot executes the simple subtasks. By performing simple inference tasks locally, the demand for computing resources of the cloud server is reduced, thereby reducing the usage cost of the cloud server. The overall operating cost of the home robot system decreases, especially for home robot devices that need to run for a long time, saving a large amount of cloud computing resource overhead. The cloud server executes complex subtasks, reducing the load on the home robot, optimizing the utilization rate of computing resources, avoiding resource waste, and at the same time ensuring that complex tasks can be supported by the powerful computing ability of the cloud server, improving the efficiency and accuracy of complex task execution. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] To more clearly illustrate the technical solutions of the present application, the following briefly introduces the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0017] Figure 1 It is the architecture diagram of the intelligent robot in the embodiment of the present application;

[0018] Figure 2 It is the architecture diagram of the cloud-network-edge integrated intelligent robot system provided in the embodiment of the present application;

[0019] Figure 3 It is a schematic diagram of a task processing process provided in the embodiment of the present application;

[0020] Figure 4 It is the architecture diagram of the system provided in the embodiment of the present application;

[0021] Figure 5 It is a schematic diagram of the structure of a task processing device provided in the embodiment of the present application;

[0022] Figure 6 It is a schematic diagram of the structure of an electronic device provided in the embodiment of the present application. Detailed implementation manners

[0023] To make the objectives, technical solutions and advantages of the present application clearer and more understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope of protection of the present application. Without conflict, the embodiments in the present application and the features in the embodiments can be arbitrarily combined with each other. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0024] The terms "first" and "second" in the specification, claims and above-mentioned drawings of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products or devices. "Multiple" in the present application can represent at least two, for example, it can be two, three or more, and the embodiments of the present application do not make limitations.

[0025] The following describes exemplary embodiments of the present application in conjunction with the accompanying drawings. Various details of the embodiments of the present application are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the disclosure of the present application. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description. It should be noted that in the embodiments of the present application, some industry-existing solutions such as certain software, components, models, etc. may be mentioned, and they should be considered exemplary. The purpose is only to illustrate the feasibility in the implementation of the technical solution of the present application, but it does not mean that the applicant has already or necessarily used this solution.

[0026] As the most iconic tool in the digital economy era, robots are profoundly changing the production and lifestyle of humans, and "robotized" intelligent tools are everywhere. The emergence of home robots has improved people's daily life and work, brought convenience to people, and enhanced the quality of life of families.

[0027] Currently, there are rich and diverse technical solutions for home intelligent robots in the industry. Classified by technology:

[0028] 1. Focus on perception technology:

[0029] Visual perception: including cameras, image processing, and computer vision technologies, used for environmental perception and object recognition.

[0030] Auditory perception: including microphones, speech recognition, and natural language processing technologies, used for voice interaction.

[0031] Tactile perception: including tactile sensors, used for perceiving the shape, texture, and temperature of objects.

[0032] 2. Focus on motion control:

[0033] Manipulator and hand: used for fine operation and grasping objects.

[0034] Mobile platform: including wheeled, tracked, and walking robots, used for moving in the home environment.

[0035] 3. Focus on intelligence and human-machine interaction technology to enhance the cognitive ability of robots:

[0036] Machine learning and deep learning: used for pattern recognition, prediction, and decision-making.

[0037] Natural language processing: used for understanding and generating natural language for voice interaction.

[0038] Computer vision: used for image and video analysis, environmental perception, and object recognition.

[0039] Voice interaction: Interaction is carried out through speech recognition and synthesis.

[0040] Graphical user interface: Interaction is carried out through the screen and touch screen.

[0041] Gesture and motion recognition: Recognize the user's gestures and motions through the camera and sensors.

[0042] According to the technical architecture, the technical architecture of the household robot can be in the following ways:

[0043] 1. Centralized architecture: All calculations and decisions are completed in a central processing unit, which is suitable for simple tasks and small household environments.

[0044] 2. Distributed architecture: Calculations and decisions are distributed among multiple nodes, which is suitable for complex tasks and large household environments.

[0045] 3. Cloud architecture: Dependent on cloud computing resources for data processing and decision-making, which is suitable for tasks that require a large amount of computing resources and data storage.

[0046] Figure 1 This is the architecture diagram of the intelligent robot for the embodiments of this application. As Figure 1 shown, compared with traditional robots, the intelligent robot has expanded the interaction layer, application service layer, digital twin model layer, operation support layer, intelligent decision-making layer, execution layer, security guarantee, and energy guarantee.

[0047] Currently, the typical robot technical architecture forms the robot's "neural network" through a wireless 5G communication network and a secure high-speed backbone network to realize the connection between the robot body and the cloud brain. The cloud brain is continuously trained and evolved through artificial intelligence algorithms, enabling the intelligence of the front-end robot to increase rapidly. Therefore, the intelligent robot system architecture that combines the cloud, network, and terminal has stronger adaptability and scalability.

[0048] Figure 2 This is the architecture diagram of the cloud-network-terminal combined intelligent robot system provided for the embodiments of this application. As Figure 2 shown, there is interaction between machine intelligence, the artificial assistance system, and the robot role application and technology platform.

[0049] Disadvantages of the existing technical architecture:

[0050] 1. Due to the latency problem of large models, the user experience of household robots is poor. The large model has a large parameter scale and long inference time, which may cause the response speed of the household robot to slow down when performing tasks, affecting the user experience. For tasks that require real-time processing and response, such as security monitoring and emergency handling, the existing cloud architecture may not be able to provide a fast enough response speed.

[0051] 2. High computing resources and cost issues brought by the existing cloud architecture to support domestic robots. The training and inference of large models require a large amount of computing resources, and the computing resource cost of cloud services is relatively high, which may increase the usage cost of users. The allocation and management of cloud computing resources may not be efficient enough, resulting in resource waste or shortage, affecting the performance of domestic robots.

[0052] 3. Privacy and security issues of home private domain data in the existing architecture: In the existing cloud architecture, the data of domestic robots needs to be frequently uploaded to the cloud for processing, and the data may face the risks of being intercepted and tampered with during the transmission process. The data stored in the cloud may face the risks of being hacked and leaked, especially sensitive data involving user privacy. Cloud service providers may also abuse user data for unauthorized analysis and commercial applications.

[0053] 4. Personalization and localization issues. The existing cloud architecture is difficult to provide highly personalized services and cannot fully utilize local data and user preferences for personalized adjustment. For some tasks that require local processing, such as the processing of affairs unique to a single family, the cloud architecture may not provide sufficient support.

[0054] Based on this, the embodiments of the present application provide a task processing method, device, equipment and medium. In the embodiments of the present application, if it is determined according to a preset classification algorithm that the received target task is a complex task, the large model is called to decompose the target task to determine each subtask corresponding to the target task; according to the classification algorithm, each complex subtask and each simple subtask in each subtask are identified; the large model is called to process each simple subtask to obtain a first inference result; and each complex subtask is sent to the cloud server for processing, and the second inference result of each complex subtask sent by the cloud server is received; the large model is called to fuse the first inference result and the second inference result to determine and output the target inference result of the target task.

[0055] Embodiment 1:

[0056] Figure 3 It is a schematic diagram of a task processing process provided by the embodiments of the present application, and the process includes:

[0057] S301: If it is determined according to a preset classification algorithm that the received target task is a complex task, the target task is decomposed to determine each subtask corresponding to the target task; according to the classification algorithm, each complex subtask and each simple subtask in each subtask are identified.

[0058] A task processing method provided by the embodiments of the present application is applied to an electronic device, and the electronic device can be a domestic robot, a PC, a server, etc.

[0059] In the embodiments of the present application, the household robot interacts with the user through multiple modalities (such as voice, vision, touch, etc.), and collects user instructions and environmental information. Modal interaction can better understand user needs and environmental background, so as to provide more personalized and localized services.

[0060] Based on this, in the embodiments of the present application, the household robot can collect user instructions carrying target tasks input by the user, and obtain the target tasks carried in the user instructions. In order to be able to distinguish complex tasks, and thus process complex tasks separately to improve the processing speed and accuracy of complex tasks, in the embodiments of the present application, the household robot will classify the obtained target tasks according to a preset classification algorithm, so as to determine whether the target task is a simple task or a complex task.

[0061] If the household robot determines that the target task is a complex task, the household robot calls a large model to decompose the target task and determine each subtask corresponding to the target task. It should be noted that in the embodiments of the present application, a large model is deployed in the household robot, and the household robot can call the large model to decompose the target task by means of prompt words.

[0062] In the embodiments of the present application, ideally, each subtask decomposed by the large model should be a simple task, but there may be special cases.

[0063] Based on this, in the embodiments of the present application, the household robot will adopt a classification algorithm to identify each complex subtask and each simple subtask in each subtask. Among them, the classification process of each subtask by the household robot is the same as the classification process of the target task, which will not be elaborated here.

[0064] S302: Process each simple subtask to obtain a first inference result; and send each complex subtask to the cloud server for processing, and receive the second inference result of each complex subtask sent by the cloud server; call the large model to fuse the first inference result and the second inference result, and determine and output the target inference result of the target task.

[0065] In the embodiments of the present application, in order to improve the task processing speed and accuracy of the household robot, the household robot is only responsible for tasks other than simple tasks.

[0066] Based on this, after the home robot determines each complex subtask and each simple subtask, the home robot processes each simple subtask to obtain a first inference result; and sends each complex subtask to the cloud server for processing, so that the cloud server processes each complex subtask. The home robot receives the second inference result of each complex subtask sent by the cloud server.

[0067] Among them, the home robot can call a large model to process each simple subtask, and can also call a local process to process each simple subtask.

[0068] In the embodiment of the present application, after the home robot receives the second inference result sent by the cloud server, the home robot calls a large model to fuse the first inference result and the second inference result, and determines and outputs the target inference result of the target task.

[0069] In addition, in the embodiment of the present application, the home robot can also directly splice the first inference result and the second inference result, and then output the spliced result as the target inference result of the target task to the user.

[0070] In the embodiment of the present application, the home robot can identify complex tasks, split the complex tasks into simple subtasks and complex subtasks, then the home robot executes the simple subtasks, and the cloud server executes the complex subtasks, reducing the load of the home robot and improving the efficiency and accuracy of complex task execution.

[0071] Embodiment 2:

[0072] In order to improve the task processing speed and accuracy of the home robot, on the basis of the above embodiment, in the embodiment of the present application, the determination that the received target task is a complex task includes:

[0073] Input the target task into a pre-trained classification model. If the classification result output by the classification model is a complex category, determine that the target task is a complex task; or,

[0074] Obtain each saved historical task, and determine the target historical task most similar to the target task according to a preset similarity algorithm; if the target historical task is a complex task, determine that the target task is a complex task.

[0075] In the embodiment of the present application, various classification algorithms can be used to determine whether the target task is a complex task.

[0076] In a possible implementation manner, a trained classification model is configured in the home robot, and the classification model can output the category to which the input information belongs, and the category includes but is not limited to a simple category and a complex category.

[0077] Specifically, the domestic robot inputs the target task into a pre-trained classification model. If the classification result output by the classification model is a complex category, it is determined that the target task is a complex task. The training process of the classification model is the same as that of the prior art and will not be elaborated here.

[0078] In another possible implementation, the domestic robot also stores historical tasks and their corresponding categories, and the domestic robot uses the category of the target historical task that is most similar to the target task as the category of the target task.

[0079] Specifically, the domestic robot obtains each stored historical task, determines the similarity between the target task and each historical task according to a preset similarity algorithm, and screens out the target historical task that is most similar to the target task. If the target historical task is a complex task, it is determined that the target task is a complex task.

[0080] Among them, when determining the similarity between the target task and each historical task, a similarity algorithm can be used to determine the similarity between the text corresponding to the target task and the historical text corresponding to each historical task. This similarity algorithm can be a cosine similarity algorithm, etc., and will not be elaborated here.

[0081] Embodiment 3:

[0082] To improve the task processing speed and accuracy of the domestic robot, based on the above embodiments, in the embodiments of the present application, before sending the complex subtask to the cloud server for processing, the method further includes:

[0083] Obtain target data collected within a preset time range;

[0084] Send the target data to the cloud server so that the cloud server processes the complex subtask according to the target data.

[0085] In the embodiments of the present application, the cloud server may need relevant data when processing complex subtasks. Based on this, to enable the cloud server to execute complex subtasks, the domestic robot will also send the locally collected data to the cloud server while sending the complex subtask to the cloud server.

[0086] Among them, the earlier the collection time, the less likely the data is to be used by the cloud server. Therefore, to avoid excessive data leakage, in the embodiments of the present application, the domestic robot will obtain the target data collected within a preset time range and send the target data to the cloud server so that the cloud server processes the complex subtask according to the target data.

[0087] Generally, the preset time range is one day, one hour, etc., and there is no limitation here.

[0088] In order to improve the task processing speed and accuracy of the home robot, based on the above embodiments, in the embodiments of the present application, before sending the target data to the cloud server, the method further includes:

[0089] Using a preset encryption algorithm to encrypt the target data.

[0090] For data security and to avoid leakage of private data, in the embodiments of the present application, before sending the target data, the home robot encrypts the target data and then sends the encrypted target data to the cloud server.

[0091] Specifically, in the embodiments of the present application, complex subtasks need to be pushed to the cloud server for processing, and at the same time, at least one of the following encryption algorithms (such as differential privacy, federated learning, etc.) is used to protect the target data.

[0092] 1. Differential privacy protects the privacy of individual data points by adding noise to the data, so that even if an external observer obtains the processed data, it is impossible to infer the specific information of a single data point, such as Google's DifferentialPrivacy Library and Microsoft's SmartNoise.

[0093] 2. Federated learning: TensorFlow Federated, PySyft, FATE, etc., allows multiple devices or nodes to collaboratively train a shared machine learning model without uploading local data to the cloud server. Each device only uploads model parameter updates, rather than the original data.

[0094] 3. Homomorphic encryption allows computations to be performed on encrypted data without decrypting the data. This means that encrypted data can be processed on the cloud server without accessing the original data, such as Microsoft SEAL, IBM HELib, and PALISADE.

[0095] 4. Data masking protects privacy by hiding sensitive information in the data, so that even if the data is leaked, it is not easy to identify individuals. Such as Oracle Data Masking and Subsetting and IBM InfoSphere Optim.

[0096] 5. Secure multi-party computation allows multiple parties to collaboratively compute a function, and each party only knows its own input and output and cannot know the inputs of other parties. Such as Sharemind, SPDZ, and ABY.

[0097] It should be noted that regardless of the encryption algorithm used to encrypt the target data, the cloud server is pre-configured with the decryption algorithm corresponding to the encryption algorithm, so as to decrypt the encrypted target data received and execute complex subtasks based on the decrypted target data.

[0098] Embodiment 4:

[0099] In order to improve the task processing speed and accuracy of the household robot, based on the above embodiments, in the embodiments of the present application, the training process of the large model includes:

[0100] Sending a large model training instruction to the cloud server, so that the cloud server fine-tunes the candidate large model configured in the cloud according to the saved first sample data, and determines each parameter value of the candidate large model after fine-tuning;

[0101] Receiving each parameter value sent by the cloud server, and configuring the local original large model with each parameter value;

[0102] Obtaining the saved second sample data, and fine-tuning the configured original large model with the second sample data to obtain the large model.

[0103] In order to be able to train a suitable large model, and at the same time to protect privacy data and avoid privacy leakage, in the embodiments of the present application, the large model can be initially trained by the cloud server first. The cloud server sends the parameter values of the trained large model to the household robot, and then the household robot performs secondary training on the large model.

[0104] Among them, when the cloud server trains the large model, the first sample data used is the non-privacy data uploaded by each household robot. The large model trained by the cloud server has the ability to decompose complex tasks, etc., but when decomposing, factors such as user habits and application scenarios are not considered, which may lead to poor decomposition effects.

[0105] Based on this, in the embodiments of the present application, after the cloud server sends the parameter values of the trained large model to the household robot, the household robot will perform secondary training on the large model according to the second sample data saved locally.

[0106] Specifically, the household robot sends a large model training instruction to the cloud server, enabling the cloud server to fine-tune the candidate large model configured on the cloud according to the saved first sample data, and determining each parameter value of the fine-tuned candidate large model. The cloud server sends each parameter value to the household robot, and the household robot receives each parameter value sent by the cloud server and configures the local original large model with each parameter value. The household robot obtains the saved second sample data and fine-tunes the configured original large model with the second sample data to obtain a large model.

[0107] In a possible implementation, the cloud server uses a general large model or a large-scale public domain dataset to train the large model to ensure that the model has extensive knowledge and capabilities, has high accuracy and generalization ability when dealing with complex tasks, and thus provides reliable inference results when needed. The training by the cloud server can cover a variety of scenarios and user preferences, providing a basis for subsequent personalized adjustments.

[0108] The cloud server compresses and quantizes the trained large model to reduce its parameter scale and computing requirements. The technologies here include but are not limited to: model compression technologies (weight pruning reduces the model size by removing weights that contribute less to the model performance), knowledge distillation (knowledge distillation trains a smaller student model to imitate the behavior of a larger teacher model), low-rank decomposition (low-rank decomposition such as SVD reduces the computing requirements by decomposing the weight matrix into the product of multiple low-rank matrices), model quantization technologies (weight quantization reduces the model size and computing requirements by converting the weights of the model from high precision such as 32-bit floating-point numbers to low precision such as 8-bit integers), activation quantization (similar to weight quantization, but it targets the output of the activation function), and mixed-precision quantization (optimizes performance and resource usage by using weights and activation values with different precisions in the same model).

[0109] The compressed and quantized model is smaller and faster, can significantly reduce the inference time, and improve the response speed of the household robot. At the same time, reducing the model size and computing requirements reduces the computing resource consumption on the cloud and the edge side, thus reducing costs.

[0110] The cloud server pushes the compressed and quantized model to the household robot, reducing the dependence on the cloud server and improving the response speed. At the same time, since the local execution of the inference task reduces the demand for the computing resources of the cloud server, the usage cost is reduced. It is also helpful for data privacy protection: reducing the frequency of data uploaded to the cloud, thus reducing the privacy and security risks during data transmission.

[0111] Example 5:

[0112] In order to improve the task processing speed and accuracy of domestic robots, based on the above embodiments, in the embodiments of the present application, the large model further outputs the association relationships of each subtask;

[0113] Before fusing the first inference result and the second inference result, the method further includes:

[0114] For each simple subtask, if it is determined, according to the association relationships of each subtask, that there is no candidate complex subtask associated with the simple subtask, then the first inference result corresponding to the simple subtask is output.

[0115] Generally, the time required for the cloud server to process complex subtasks is longer than the time required for the domestic robot to process simple subtasks. Therefore, in order to avoid long waiting times for users, in the embodiments of the present application, the domestic robot will first output a part of the inference results.

[0116] Specifically, for each simple subtask, if it is determined, according to the association relationships of each subtask, that there is no candidate complex subtask associated with the simple subtask, then the first inference result corresponding to the simple subtask is output. Among them, if there is a candidate complex subtask associated with the simple subtask, it means that the first inference result corresponding to the simple subtask still needs to be used, and then the first inference result is not output.

[0117] In a possible implementation manner, the domestic robot collects the second inference result of the cloud server and fuses it with the first inference result locally to provide the final decision and service. By fusing the inference results of the local and cloud servers, more comprehensive and accurate services can be provided, and at the same time, the impact of cloud server latency can be alleviated through local quick response. The possible technical difficulties and solutions here are as follows:

[0118] 1. Result consistency: The local and cloud servers may use different versions of models or data, resulting in inconsistent inference results.

[0119] The solutions include version control, that is, ensuring that the local and cloud servers use the same version of models and data. This can be achieved through a model push mechanism and a data synchronization strategy; model synchronization: when the model is updated, ensure that the model of the cloud server is promptly pushed to the local, and the inference task is only carried out after the update is completed locally.

[0120] 2. Latency and synchronization: The return of the second inference result of the cloud server may have latency, affecting the real-time nature of result fusion.

[0121] The solutions include asynchronous updates: allowing the local first inference result to be used for quick response first, and the second inference result of the cloud server for subsequent updates. Asynchronous processing can be achieved through an event-driven architecture; caching mechanism: caching the second inference result of the cloud server locally for quick access when needed; predictive inference: performing predictive inference locally to prepare possible inference results in advance and reduce waiting time.

[0122] 3. The confidence levels of the inference results of the local and cloud servers may be different, and how to reasonably assign weights is a challenge. Perform confidence assessment on the inference results of the local and cloud servers through confidence assessment, and assign different weights according to the confidence levels. Statistical methods, machine learning models, etc. can be used for assessment. Additionally, different weights can be assigned according to the confidence levels through weighted fusion, and weighted average or weighted voting can be performed to obtain the final fusion result.

[0123] Embodiment 6:

[0124] To improve the task processing speed and accuracy of the household robot, based on the above embodiments, in the embodiments of the present application, the method further includes:

[0125] If it is determined that the target task is a simple task, then process the target task to obtain a third inference result;

[0126] Determine the third inference result as the target inference result and output it.

[0127] In the embodiments of the present application, if the household robot determines that the target task is a simple task, then process the target task to obtain a third inference result. The household robot determines the third inference result as the target inference result and outputs it.

[0128] Among them, the household robot can call a large model to execute the target task, or call a local thread to execute the target task.

[0129] In a possible implementation manner, the household robot interacts with the user through multiple modalities (such as voice, vision, touch, etc.) to collect user instructions and environmental information. Modal interaction can better understand user needs and environmental backgrounds, thereby providing more personalized and localized services.

[0130] The household robot or other end-side devices with strong computing power can load the large model pushed from the cloud and perform necessary fine-tuning and training based on local data. Through local training, the large model can better adapt to the needs and preferences of a single household. Local training reduces the need to upload user data to the cloud, thereby protecting user privacy.

[0131] Home robots distribute tasks according to their complexity, and simple tasks are performed locally. Performing simple tasks locally can significantly reduce response time and improve user experience. At the same time, local execution of tasks reduces the demand for cloud server computing resources, thereby reducing costs. In addition, the frequency of data upload is reduced, thereby reducing privacy and security risks. Typical technical difficulties and feasible solutions are as follows:

[0132] 1. Task complexity assessment: How to accurately assess the complexity of a task to determine whether the task is to be performed locally or by a cloud server. Task complexity can be classified according to task type and historical data. For example, simple tasks include speech recognition, basic environmental perception, and simple device control, while complex tasks include video analysis, multi-sensor fusion, and complex behavior prediction. At the same time, machine learning models or rule engines can be used to evaluate the computing requirements of tasks in real time. For example, evaluation is based on the input data size of the task, the complexity of the computational graph, the historical execution time, and so on. Finally, the task allocation strategy can be dynamically adjusted based on the system resource status (such as the CPU and memory usage of the local device).

[0133] 2. Capacity limitations of local models: Complex tasks can be decomposed into multiple subtasks, and some subtasks can be executed locally as much as possible to reduce the amount of computing uploaded to the cloud.

[0134] 3. Task scheduling and resource management: Efficiently schedule and manage local and cloud computing resources to ensure that tasks can be executed in a timely manner.

[0135] Task queues and priority scheduling can be used to design task queues and schedule tasks according to their priority. For example, security monitoring tasks have a higher priority than daily device control tasks. In a multi-device environment (such as multiple home robots), load balancing algorithms can be used to reasonably distribute tasks to different devices to avoid overloading a single device.

[0136] Among them, based on the above embodiment, some processes can be replaced, for example:

[0137] 1. About the replacement of system architecture:

[0138] Edge computing architecture: offload some computing tasks from cloud servers to edge nodes (such as local servers, etc.). Edge nodes can perform some reasoning tasks and work together with end devices (home robots) and cloud servers.

[0139] The reasons why this architecture is not suitable for home scenarios are as follows: 1. The end-side devices, including home robots, already have good computing power. The home robots themselves can also be used as edge devices, without the need to deploy edge devices. 2. Home robots can perform actions, while edge devices cannot, and cannot help families improve their quality of life.

[0140] 2. Fully distributed architecture: The computing tasks are fully distributed to multiple household devices, and each device is responsible for a part of the inference task. The overall task is completed through communication and collaboration among the devices.

[0141] The advantage is to make full use of the computing resources of multiple devices in the home, avoid the single-point computing pressure, and enhance the robustness of the system at the same time.

[0142] The disadvantage is that the architecture is complex and difficult to implement, and there is no obvious advantage. Especially the communication between the edge devices will bring unnecessary network overhead.

[0143] 3. Technical solution substitution for each module: There are various technologies that can directly substitute for task allocation, privacy protection, computing resource management, model training and inference, etc. The embodiments of the present application mainly focus on the edge-cloud inference architecture and interaction process design of the household robot, and the replacement of a single module does not affect the overall process.

[0144] The embodiments of the present application have the following advantages:

[0145] 1. Low-latency response and high user experience: It reduces the user's perception of system latency and improves the interaction fluency and user satisfaction of the household robot.

[0146] 2. Reduce cloud computing costs: By executing simple inference tasks locally, it reduces the demand for the computing resources of the cloud server, thereby reducing the usage cost of the cloud server. The overall operation cost of the household robot system decreases, especially for the household robot devices that need to run for a long time, saving a large amount of cloud computing resource overhead.

[0147] 3. Data privacy and security protection: By reducing the data upload frequency and combining privacy protection technologies such as differential privacy, homomorphic encryption, and federated learning, it effectively protects the privacy and security of household private domain data.

[0148] 4. Personalized and localized services: The household robot can perform model fine-tuning and inference according to the user's personalized needs and environmental characteristics, and provide services that better meet the user's needs. Ultimately, it improves the personalization and localization level of the services, enabling the household robot to better adapt to the special needs of different families and users.

[0149] 5. Efficient resource utilization and task allocation: Through task complexity evaluation and dynamic task allocation, it reasonably utilizes the computing resources of the local and cloud servers to ensure that the tasks are executed in the best environment. It optimizes the utilization rate of computing resources, avoids resource waste, and at the same time ensures that complex tasks can be supported by the powerful computing capabilities of the cloud server.

[0150] 6. Fault Tolerance and High Reliability: A fault tolerance mechanism and a redundant backup strategy are designed to ensure that the home robot can still provide services relying on local inference results when the cloud server is unavailable. Therefore, the reliability and stability of the system are improved, ensuring that the home robot can still work properly in various situations (such as unstable network or cloud server failure).

[0151] Figure 4 This is the system architecture diagram provided by the embodiments of the present application. As shown in Figure 4 this figure, the system includes an edge side (home robot) and a cloud side (cloud server). Among them, large model training, model compression and quantization, model synchronization and distribution, and complex task inference are completed on the cloud side, and multi-modal interaction, local model training, task distribution, local inference, privacy protection, and inference result fusion are completed on the edge side.

[0152] Embodiment 7:

[0153] Based on the same inventive concept, an embodiment of the present application provides a task processing device Figure 5 This is a schematic structural diagram of a task processing device provided by an embodiment of the present application, which is applied to a display terminal device. Please refer to Figure 5 this figure. The device includes:

[0154] A processing module 501, configured to decompose the target task to determine each subtask corresponding to the target task if it is determined according to a preset classification algorithm that the received target task is a complex task; identify each complex subtask and each simple subtask in each of the subtasks according to the classification algorithm;

[0155] A calling module 502, configured to process each simple subtask to obtain a first inference result; send each complex subtask to the cloud server for processing, and receive a second inference result of each complex subtask sent by the cloud server; call the large model to fuse the first inference result and the second inference result, and determine and output a target inference result of the target task.

[0156] In a possible implementation manner, the processing module 501 is specifically configured to input the target task into a pre-trained classification model, and determine that the target task is a complex task if the classification result output by the classification model is a complex category; or, obtain each saved historical task, and determine the target historical task most similar to the target task according to a preset similarity algorithm; if the target historical task is a complex task, determine that the target task is a complex task.

[0157] In a possible implementation, the calling module 502 is specifically configured to obtain target data collected within a preset time range; send the target data to the cloud server, so that the cloud server processes the complex subtask according to the target data.

[0158] In a possible implementation, the calling module 502 is further configured to encrypt the target data by using a preset encryption algorithm.

[0159] In a possible implementation, the processing module 501 is further configured to send a large model training instruction to the cloud server, so that the cloud server fine-tunes a candidate large model configured in the cloud according to the saved first sample data to determine each parameter value of the candidate large model after fine-tuning; receive each parameter value sent by the cloud server, and configure the local original large model by using each parameter value; obtain the saved second sample data, and fine-tune the configured original large model by using the second sample data to obtain the large model.

[0160] In a possible implementation, the large model further outputs the association relationship of each subtask;

[0161] The calling module 502 is further configured to, for each simple subtask, if it is determined according to the association relationship of each subtask that there is no candidate complex subtask associated with the simple subtask, output a first inference result corresponding to the simple subtask.

[0162] In a possible implementation, the calling module 502 is further configured to, if it is determined that the target task is a simple task, process the target task to obtain a third inference result; determine the third inference result as the target inference result and output it.

[0163] Embodiment 8:

[0164] Based on the same inventive concept, an embodiment of the present application provides an electronic device, which can implement the steps of the screen mirroring method described above. Figure 6 The following is a schematic structural diagram of an electronic device provided by an embodiment of the present application, as Figure 6 shown, including: a processor 601, a communication interface 602, a memory 603, and a communication bus 604. Among them, the processor 601, the communication interface 602, and the memory 603 complete mutual communication through the communication bus 604;

[0165] A computer program is stored in the memory 603. When the program is executed by the processor 601, the processor 601 is caused to execute the following steps:

[0166] If it is determined that the received target task is a complex task according to a preset classification algorithm, the target task is decomposed to determine each subtask corresponding to the target task;

[0167] According to the classification algorithm, each complex subtask and each simple subtask in each of the subtasks are identified;

[0168] Each simple subtask is processed to obtain a first inference result; and each complex subtask is sent to a cloud server for processing, and a second inference result of each complex subtask sent by the cloud server is received;

[0169] The large model is called to fuse the first inference result and the second inference result, and the target inference result of the target task is determined and output.

[0170] In a possible implementation manner, the determining that the received target task is a complex task includes:

[0171] The target task is input into a pre-trained classification model. If the classification result output by the classification model is a complex category, it is determined that the target task is a complex task; or,

[0172] Each historical task saved is obtained, and according to a preset similarity algorithm, a target historical task most similar to the target task is determined; if the target historical task is a complex task, it is determined that the target task is a complex task.

[0173] In a possible implementation manner, before sending the complex subtask to the cloud server for processing, the method further includes:

[0174] Target data collected within a preset time range is obtained;

[0175] The target data is sent to the cloud server, so that the cloud server processes the complex subtask according to the target data.

[0176] In a possible implementation manner, before sending the target data to the cloud server, the method further includes:

[0177] The target data is encrypted using a preset encryption algorithm.

[0178] In a possible implementation manner, the training process of the large model includes:

[0179] A large model training instruction is sent to the cloud server, so that the cloud server fine-tunes a candidate large model configured in the cloud according to the saved first sample data to determine each parameter value of the candidate large model after fine-tuning;

[0180] Receive each of the parameter values sent by the cloud server, and configure the local original large model with each of the parameter values;

[0181] Obtain the saved second sample data, and fine-tune the configured original large model with the second sample data to obtain the large model.

[0182] In a possible implementation manner, the large model further outputs the association relationship of each subtask;

[0183] Before fusing the first inference result and the second inference result, the method further includes:

[0184] For each simple subtask, if it is determined according to the association relationship of each subtask that there is no candidate complex subtask associated with the simple subtask, then output the first inference result corresponding to the simple subtask.

[0185] In a possible implementation manner, the method further includes:

[0186] If it is determined that the target task is a simple task, process the target task to obtain a third inference result;

[0187] Determine the third inference result as the target inference result and output it.

[0188] Since the principle of the above electronic device for solving problems is similar to the task processing method, the implementation of the above electronic device can refer to the embodiments of the method, and the repeated parts will not be described again.

[0189] The communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, only a thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface 602 is used for communication between the above electronic device and other devices. The memory may include a Random Access Memory (RAM), and may also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0190] The above-mentioned processor may be a general-purpose processor, including a central processing unit, a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit, a field-programmable gate array, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0191] Embodiment 9:

[0192] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium, in which a computer program executable by a processor is stored. When the program runs on the processor, the processor is caused to execute any one of the task processing methods described above. Since the principle of solving problems by the above computer-readable storage medium is similar to that of the task processing method, the implementation of the above computer-readable storage medium can refer to the implementation of the method, and the repeated parts will not be described again.

[0193] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0194] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the flows and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the specified functions in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0195] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured product including an instruction device, and the instruction device implements the specified functions in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0196] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are executed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions for implementing the functions specified in one process or a plurality of processes and / or blocks Figure 1 in one block or a plurality of blocks Figure 1 in the steps.

[0197] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims

1. A task processing method, characterized in that: The method comprises: If the received target task is determined to be a complex task according to the preset classification algorithm, the target task is decomposed to determine each subtask corresponding to the target task; According to the classification algorithm, identifying each complex subtask and each simple subtask in each subtask; Processing each of the simple subtasks to obtain a first reasoning result; and sending each of the complex subtasks to a cloud server for processing, and receiving a second reasoning result of each of the complex subtasks sent by the cloud server; The large model is called to fuse the first reasoning result and the second reasoning result, and a target reasoning result of the target task is determined and output.

2. The method according to claim 1, characterized in that Determining that the received target task is a complex task includes: Input the target task into a pre-trained classification model, and if the classification result output by the classification model is a complex category, then determine that the target task is a complex task; or, Obtain each saved historical task, and determine the target historical task that is most similar to the target task according to a preset similarity algorithm; if the target historical task is a complex task, determine that the target task is a complex task.

3. The method according to claim 1, characterized in that Before sending the complex subtask to the cloud server for processing, the method further includes: Obtain target data collected within a preset time range; The target data is sent to the cloud server, so that the cloud server processes the complex subtask according to the target data.

4. The method according to claim 3, characterized in that: Before sending the target data to the cloud server, the method further includes: The target data is encrypted using a preset encryption algorithm.

5. The method according to claim 1, characterized in that The training process of the large model includes: Sending a large model training instruction to the cloud server, so that the cloud server fine-tunes the candidate large model configured in the cloud according to the saved first sample data, and determines each parameter value of the candidate large model after fine-tuning; Receiving each parameter value sent by the cloud server, and configuring the local original large model using each parameter value; The saved second sample data is obtained, and the configured original large model is fine-tuned using the second sample data to obtain the large model.

6. The method according to claim 1, characterized in that The large model also outputs the association relationship of each subtask; Before fusing the first reasoning result with the second reasoning result, the method further includes: For each simple subtask, if it is determined according to the association relationship of each subtask that there is no candidate complex subtask associated with the simple subtask, then the first reasoning result corresponding to the simple subtask is output.

7. The method according to claim 1, characterized in that The method further comprises: If it is determined that the target task is a simple task, the target task is processed to obtain a third reasoning result; The third inference result is determined as the target inference result and outputted.

8. A task processing device, characterized in that: The device comprises: A processing module, configured to, if it is determined according to a preset classification algorithm that the received target task is a complex task, decompose the target task to determine each subtask corresponding to the target task; and identify each complex subtask and each simple subtask in each subtask according to the classification algorithm; A calling module is used to process each simple subtask to obtain a first reasoning result; and send each complex subtask to a cloud server for processing, and receive a second reasoning result of each complex subtask sent by the cloud server; call the large model to fuse the first reasoning result with the second reasoning result, and determine and output the target reasoning result of the target task.

9. An electronic device, characterized in that: The electronic device comprises at least a processor and a memory, and the processor is used to implement the steps of the task processing method according to any one of claims 1 to 7 when executing a computer program stored in the memory.

10. A computer-readable storage medium, characterized in that: It stores a computer program, which, when executed by a processor, implements the steps of the task processing method as claimed in any one of claims 1 to 7.

Images