A task processing method and apparatus

Abstract

The application discloses a kind of task processing method and device, it is related to data processing technical field.The specific embodiment of the method includes: receiving the computing task transmitted by user end, obtains data processing requirement from computing task to determine data characteristics;Intelligent agent is called in advance in edge node and private cloud center deployment, and real-time computing resource information is obtained;Preset machine learning model is called, and the computing resource requirement information of edge node and private cloud center in future preset time period is predicted;According to data processing requirement, data characteristics, real-time computing resource information and computing resource requirement information, the object of processing computing task is determined;The computing task is sent to object for processing, and the processing result returned by object is received and returned to user end.The embodiment breaks the bottleneck that traditional computing task is limited to cloud center platform processing, effectively promotes the localization, customized deployment of computing task, ensures service quality and response speed, improves user experience.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing in big data, and in particular to a task processing method and device. BACKGROUND

[0002] In existing cloud computing, cloud resource scheduling mainly adopts a cloud center driving mode, that is, all computing tasks are submitted to a cloud center platform for processing, which cannot realize customized local computing, resulting in that the cloud resource scheduling cannot meet the needs of some users for resource customization and low delay, and cannot intelligently select whether to process at an edge node or a cloud center platform. The customized local computing refers to, in an edge computing environment, customizing and locally processing computing tasks according to specific application scenarios and user needs, that is, processing specific tasks at an edge node where data is generated, rather than transmitting data to a cloud center platform for processing. SUMMARY

[0003] Therefore, the embodiments of the present application provide a task processing method and device, which can at least solve the problem that the prior art cannot realize customized local computing and cannot intelligently select whether to process at an edge node or a cloud center platform.

[0004] To achieve the above object, according to an aspect of the embodiments of the present application, a task processing method is provided, which comprises: receiving a computing task transmitted by a user end, obtaining data processing requirements from the computing task, and determining data characteristics according to the data processing requirements; calling an intelligent agent pre-deployed at an edge node and a private cloud center, and respectively obtaining real-time computing resource information of the edge node and the private cloud center; calling a preset machine learning model, and respectively predicting computing resource demand information of the edge node and the private cloud center in a future preset time period; determining an object for processing the computing task according to the data processing requirements, the data characteristics, the real-time computing resource information and the computing resource demand information; wherein the object is one of the edge node, the private cloud center and a cloud center platform; sending the computing task to the object for processing, receiving a processing result returned by the object and returning the processing result to the user end.

[0005] Optionally, the method further comprises: receiving historical computing resource information and collection time points of the edge node and the private cloud center collected by the intelligent agent in a historical preset time period; registering and preprocessing the historical computing resource information to obtain computing resource index data; training the machine learning model by using the collection time points and the computing resource index data to obtain a trained machine learning model; wherein the machine learning model is iteratively updated based on the real-time computing resource information and the collection time points.

[0006] Optionally, the determining the object for processing the computing task according to the data processing requirement, the data characteristic, the real-time computing resource information and the computing resource demand information comprises: determining the object for processing the computing task according to the data processing requirement, the data characteristic and the real-time computing resource information; in the case that the real-time computing resource information is less than the computing resource demand information, calculating the difference between the two, recycling the computing resource information of the difference to a preset personalized private cloud resource pool; wherein the preset personalized private cloud resource pool is formed based on idle computing resources; in the case that the real-time computing resource information is greater than the computing resource demand information, allocating the computing resource of the difference from the preset personalized private cloud resource pool to the object; or if the object is an edge node or a private cloud center, transferring the computing task to a cloud center platform for processing.

[0007] Optionally, after the receiving the processing result returned by the object and returning to the user end, the method further comprises: in response to the object processing the computing task ending, recycling the computing resource allocated to the object to a preset personalized private cloud resource pool.

[0008] Optionally, applied to an intermediate agent platform, in the case that the object is an edge node, the sending the computing task to the object for processing, receiving the processing result returned by the object and returning to the user end comprises: sending a deployment instruction to the edge node, so that the edge node pulls the image of the computing task from the intermediate agent platform for processing; wherein the edge node uses container or virtual machine technology to allocate an independent computing environment and resources for the computing task; receiving the processing result returned by the edge node and returning the processing result to the user end.

[0009] Optionally, the computing task is generated according to the data processing requirement configured by the user through a development kit, and the development kit is arranged in the user end.

[0010] To achieve the above object, according to another aspect of the embodiment of the present application, a task processing device is provided, comprising: a receiving module configured to receive a computing task transmitted by a user terminal, obtain data processing requirements from the computing task, and determine data characteristics according to the data processing requirements; an obtaining module configured to call intelligent agents pre-deployed at edge nodes and private cloud centers, and obtain real-time computing resource information of the edge nodes and the private cloud centers, respectively; a prediction module configured to call a preset machine learning model, and predict computing resource requirement information of the edge nodes and the private cloud centers in a future preset time period, respectively; a determination module configured to determine an object for processing the computing task according to the data processing requirements, the data characteristics, the real-time computing resource information, and the computing resource requirement information; wherein the object is one of the edge nodes, the private cloud centers, and a cloud center platform; and a processing module configured to send the computing task to the object for processing, receive a processing result returned by the object, and return the processing result to the user terminal.

[0011] Optionally, the device further comprises a training module configured to: receive historical computing resource information of the edge nodes and the private cloud centers collected by the intelligent agents in a historical preset time period and collection time points; register and preprocess the historical computing resource information to obtain computing resource index data; and train a machine learning model using the collection time points and the computing resource index data to obtain a trained machine learning model; wherein the machine learning model is iteratively updated based on the real-time computing resource information and the collection time points.

[0012] Optionally, the determination module is configured to: determine the object for processing the computing task according to the data processing requirements, the data characteristics, and the real-time computing resource information; in a case where the real-time computing resource information is less than the computing resource requirement information, calculate a difference value between the two, and recycle the computing resource information of the difference value to a preset personalized private cloud resource pool; wherein the preset personalized private cloud resource pool is formed based on idle computing resources; in a case where the real-time computing resource information is greater than the computing resource requirement information, allocate the computing resource of the difference value from the preset personalized private cloud resource pool to the object; or if the object is an edge node or a private cloud center, transfer the computing task to a cloud center platform for processing.

[0013] Optionally, the device further comprises a resource recycling module configured to: in response to the object ending processing the computing task, recycle the computing resource allocated to the object to the preset personalized private cloud resource pool.

[0014] Optionally, applied to the intermediate agent platform, in the case that the object is an edge node, the processing module is configured to: send a deployment instruction to the edge node, so that the edge node pulls the image of the computing task from the intermediate agent platform for processing; wherein the edge node allocates an independent computing environment and resources for the computing task by using container or virtual machine technology; and receive the processing result returned by the edge node and return the processing result to the user terminal.

[0015] Optionally, the computing task is generated according to the data processing requirement configured by the user through the development kit, and the development kit is arranged in the user terminal.

[0016] To achieve the above object, according to another aspect of the embodiment of the present application, a task processing electronic device is provided.

[0017] The electronic device of the embodiment of the present application comprises: one or more processors; a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the task processing method described above.

[0018] To achieve the above object, according to another aspect of the embodiment of the present application, a computer readable medium is provided, which stores a computer program, and the program is executed by a processor to implement the task processing method described above.

[0019] To achieve the above object, according to another aspect of the embodiment of the present application, a computing program product is provided. The computing program product of the embodiment of the present application comprises a computer program, and the program is executed by a processor to implement the task processing method provided by the embodiment of the present application.

[0020] According to the scheme provided by the present application, one embodiment of the above application has the following advantages or beneficial effects: according to the task characteristics, real-time computing resource information and estimated resource demand information, the execution position of the computing task can be intelligently decided: or processed by the cloud center platform, or executed in the edge node or private cloud environment. This mechanism successfully breaks the bottleneck of traditional computing tasks limited to cloud center processing, effectively promotes the localization and customization of computing tasks, especially for those users who have strict requirements for low delay and highly personalized services, providing an ideal solution to ensure service quality and response speed and improve user experience.

[0021] The further effects of the above-mentioned non-conventional optional mode will be described in the following combined with the specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0022] The accompanying drawings are used to better understand the present application, and do not constitute an improper limitation on the present application. Among them:

[0023] Figure 1 FIG. 1 is a main flowchart of a task processing method according to an embodiment of the present application;

[0024] Figure 2 FIG. 2 is a flowchart of training of a machine learning model according to an embodiment of the present application;

[0025] Figure 3 FIG. 3 is a flowchart of an optional task processing method according to an embodiment of the present application;

[0026] Figure 4 FIG. 4 is a flowchart of another optional task processing method according to an embodiment of the present application;

[0027] Figure 5 FIG. 5 is a schematic diagram of an edge node structure;

[0028] Figure 6 FIG. 6 is an architecture diagram of the entire implementation process;

[0029] Figure 7 FIG. 7 is a main module diagram of a task processing apparatus according to an embodiment of the present application;

[0030] Figure 8 FIG. 8 is an exemplary system architecture diagram to which embodiments of the present application can be applied;

[0031] Figure 9 FIG. 9 is a structural diagram of a computer system of a mobile device or server suitable for implementing embodiments of the present application. DETAILED DESCRIPTION

[0032] Exemplary embodiments of the present application are described below with reference to the accompanying drawings, which include various details of the embodiments of the present application to assist in understanding, which should be considered in a descriptive sense only and not limiting. Thus, it will be apparent to those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Also, for the sake of brevity and clarity, descriptions of well-known functions and constructions are omitted from the following description.

[0033] It should be noted that in the embodiments of the present application, some software, components, models and other prior art solutions can be mentioned, which should be considered as exemplary and only for the purpose of illustrating the feasibility of the technical solutions of the present application, but do not mean that the applicant has or will necessarily use the solutions.

[0034] In the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The acquisition, transmission, storage, use, processing and the like of data in the technical solutions of the present application comply with the relevant provisions of national laws and regulations, are used for legal and reasonable purposes, are not shared, disclosed or sold outside these legal uses and the like, and are subject to the supervision and management of the regulatory authorities.

[0035] For user information, necessary measures should be taken for user personal information to prevent illegal access to such personal information data, to ensure that personnel with access to personal information data comply with the provisions of relevant laws and regulations, and to ensure the security of user personal information. Once these user personal information data are no longer needed, the risk should be minimized by limiting or even prohibiting data collection and / or deleting data. When applicable, including in certain relevant applications, user privacy is protected by de-identification of data, such as by removing specific identifiers (e.g., date of birth, etc.) when applicable, controlling the amount or specificity of data stored (e.g., collecting location data at the city level rather than at the specific address level), controlling how data is stored, and / or other methods of de-identification.

[0036] Edge devices and edge nodes are different. Edge devices generally refer to terminal devices located at the edge of the network, such as smartphones, tablets, industrial devices, etc. They can collect data, perform local computing and processing tasks. Edge nodes refer to centralized nodes of computing and storage resources located at the edge of the network, providing computing, storage and network services for edge devices. They usually have stronger computing and storage capabilities. Multiple edge nodes are usually set under an edge device, and there are many advantages in implementing customized local computing in edge nodes:

[0037] 1. Reduce latency: In some application scenarios (such as autonomous driving, industrial Internet of Things, smart cities), the requirement for latency is very high. By performing localized processing in edge nodes, the latency of data transmission can be significantly reduced, and the response speed can be improved.

[0038] 2. Reduce bandwidth consumption: Transmitting a large amount of data from edge nodes to cloud centers will occupy a large amount of bandwidth. Customized local computing can perform preliminary processing and filtering in edge nodes, reducing frequent data transmission, thereby effectively reducing bandwidth consumption.

[0039] 3. Enhance data privacy and security: In some sensitive applications (such as medical health, financial services), data privacy and security are very important. By performing localized computing in edge nodes, sensitive data can be avoided from being transmitted to cloud centers, reducing the risk of data leakage.

[0040] 4. Improve the reliability and flexibility of the system: edge nodes can handle data independently, even if the connection with the cloud center is interrupted, to ensure basic computing processing capability and improve the reliability and flexibility of the system.

[0041] 5. Realize personalized and customized services: Different users and application scenarios may have different computing needs. By customizing local computing, computing tasks can be optimized according to specific needs to provide personalized services.

[0042] Referring to Figure 1 , a main flowchart of a task processing method provided by an embodiment of the application is shown, including the following steps:

[0043] S101: receiving a computing task transmitted by a user terminal, obtaining data processing requirements from the computing task, and determining data characteristics according to the data processing requirements;

[0044] S102: calling an intelligent agent pre-deployed on an edge node and a private cloud center to obtain real-time computing resource information of the edge node and the private cloud center, respectively;

[0045] S103: calling a preset machine learning model to predict computing resource demand information of the edge node and the private cloud center in a future preset time period, respectively;

[0046] S104: determining an object for processing the computing task according to the data processing requirements, the data characteristics, the real-time computing resource information, and the computing resource demand information; wherein the object is one of an edge node, a private cloud center, and a cloud center platform;

[0047] S105: sending the computing task to the object for processing, receiving a processing result returned by the object, and returning the processing result to the user terminal.

[0048] In the above embodiment, for step S101, the user terminal is the ultimate user of the system and serves users who need computing. The development kit is a tool set for developing customized computing tasks. The user can set the development kit in the user terminal and describe the requirements of the computing task in the development kit, such as computing amount and response time. Then the user initiates a customized computing task to the system intermediate agent platform by using the development kit.

[0049] The intermediate agent platform receives the computing task initiated by the user terminal through the development kit. The data processing requirements are obtained from the computing task, such as 1) computing amount: how much computing resource the task needs, to determine whether it should be processed in the edge node or the cloud center. 2) Response time: the demand of the task for response speed, which affects whether it should be processed in the edge node closer to the user terminal.

[0050] According to the above data processing requirements, the intermediate agent platform infers the characteristics of the data involved, such as data volume, sensitivity, and real-time demand. 1) Data volume: the size of the data involved in the task determines whether it should be processed in a cloud center with larger bandwidth. 2) Sensitivity: whether the data involved in the task is sensitive, whether it should be processed in a more secure and controllable edge node. 3) Real-time demand: the real-time requirement of the task for data, which affects whether it should be processed in an edge node with lower delay.

[0051] For step S102, in addition to inferring the characteristics of the data involved according to the data processing requirements, the intermediate agent platform also monitors its own running state in real time, such as delay, bandwidth, resource utilization, etc., as a basis for decision-making. The present scheme pre-deploys intelligent agents in edge nodes to collect real-time computing resource information of edge nodes, including but not limited to workload information, resource usage, energy consumption data, etc. In actual operation, in addition to collecting real-time computing resource information of edge nodes, real-time computing resource information of private cloud centers can also be collected. The private cloud center here is opposite to the cloud center platform, and the cloud center platform is the server cloud platform. The private cloud center can be the user's own private cloud. The private cloud is a resource pool formed by computing resources, and the private cloud center is a centralized computing resource pool located in a relatively centralized data center. The cloud center platform is equivalent to the server side and can be connected to the user's edge node and also to the user's private cloud. Therefore, the final object of processing and computing tasks can be the cloud center platform or the edge node or the private cloud center.

[0052] For step S103, the present scheme also provides a machine learning model for predicting the computing resource demand information of edge nodes and private cloud centers in the future preset time period, such as the computing resources needed in the next three months, such as the computing resources needed during the big promotion, such as CPU, kernel, energy consumption, etc. Referring to Figure 2 The training process of the machine learning model is shown. After collecting the real-time computing resource information of the edge nodes, the intelligent agent returns the collected real-time computing resource information and the collection time point to the intermediate agent platform for registration and preprocessing, which is used for subsequent resource scheduling decisions.

[0053] The purpose of registration includes but is not limited to: 1) centralized management of computing resources. By registering in the intermediate agent platform, the computing resources scattered in edge nodes and private cloud centers can be centrally managed and monitored, such as generating a resource information table. 2) Resource information standardization. After registration in the intermediate agent platform, various indicators of computing resources (CPU, memory, bandwidth, etc.) will be uniformly preprocessed and formatted to facilitate subsequent analysis and scheduling. 3) Dynamic perception of resource state. The intermediate agent platform can dynamically perceive the resource usage and load state of each edge node and private cloud center to provide a basis for scheduling decisions.

[0054] The preprocessing includes, but is not limited to, processing of technical indicators such as CPU, memory, network bandwidth, energy consumption data, workload information, etc. 1) Processing of CPU and memory indicators: aggregate, average, etc. Process the original CPU occupancy, memory usage, etc. indicators, convert them into more meaningful technical indicators, and provide clearer and more comparable data basis for subsequent resource scheduling decisions. 2) Network bandwidth indicator processing, such as analyzing and processing network performance indicators such as network throughput and delay, to extract more representative bandwidth utilization, network delay, etc. indicators. 3) Processing of energy consumption data, such as classifying and counting the original energy consumption data to obtain the overall energy consumption level and trend information of each computing node. 4) Processing of workload information, such as analyzing the types of services and load conditions currently undertaken by each edge node to extract indicators that reflect the performance and usage state of the node.

[0055] Then use the collection time point and the computing resource indicator data to train the machine learning model, and the machine learning model is used to predict future resource demand changes. The machine learning model can be trained according to historical computing resource information, and dynamically adjusted according to real-time computing resource information to improve the accuracy of the prediction.

[0056] For steps S104 and S105, the present scheme determines the location of task processing according to the data processing requirements, data characteristics, current system state (such as delay, bandwidth, resource utilization, etc.) and computing resource demand information under the premise of ensuring data security, and intelligently selects whether to process in the edge node or the cloud center platform. See the subsequent description. Figure 3 For example, tasks with large computational complexity and high real-time requirements may be more suitable for execution on edge nodes with low latency. Tasks involving large data volumes and high security requirements may be more suitable for processing on cloud center platforms with sufficient resources.

[0057] Meanwhile, the intermediate agent platform is also responsible for taking measures such as encryption and permission control to ensure data security. For example, 1) encrypt user data. Symmetric encryption algorithm can be used, and symmetric key is shared by user equipment and edge node / cloud center for encryption and decryption, or dynamic key management is introduced. The symmetric key is dynamically allocated by the key management service to avoid key leakage. 2) Re-encryption during transmission. User equipment to edge node uses TLS (Transport Layer Security) protocol for transmission layer encryption, and edge node to cloud center also uses TLS protocol. 3) Strict host and access control is set for edge node and cloud center platform, and only authorized cloud platform components are allowed to access and process data. 4) Data transmission inside edge node or cloud center platform also needs to be encrypted to avoid risks caused by internal network leakage. 5) Support setting access control list to control which tasks or components can access specific data in a fine-grained manner.

[0058] The method provided by the above embodiment can intelligently decide the execution position of the computing task according to the task characteristics, real-time computing resource information and estimated resource demand information: or hand over to the cloud center platform for processing, or downlink to the edge node or private cloud environment for execution. This mechanism successfully breaks the bottleneck that traditional computing tasks are limited to cloud center processing, effectively promotes the localization and customization of computing task deployment, especially for user groups with strict requirements for low latency and highly personalized services, providing an ideal solution to ensure service quality and response speed and improve user experience.

[0059] Referring to Figure 3 , an optional task processing method flowchart according to an embodiment of the application is shown, which includes the following steps:

[0060] S301: determining an object for processing the computing task according to the data processing requirement, the data characteristics and the real-time computing resource information;

[0061] S302: calculating the difference between the real-time computing resource information and the computing resource demand information;

[0062] S303: in the case that the real-time computing resource information is less than the computing resource demand information, recycling the difference of the computing resource information to a preset personalized private cloud resource pool; wherein the preset personalized private cloud resource pool is formed based on idle computing resources;

[0063] S304: in the case that the real-time computing resource information is greater than the computing resource demand information, allocating the difference of the computing resource from the preset personalized private cloud resource pool to the object; or if the object is an edge node or a private cloud center, transferring the computing task to the cloud center platform for processing.

[0064] In the above embodiments, for steps S301-S304, the key steps for determining the task processing location are as follows:

[0065] 1. The user initiates a computing task to the intermediate agent platform through the development kit, and describes the data processing requirements (such as computing amount, response time, etc.).

[0066] 2. Under the premise of ensuring data security, the intermediate agent platform determines whether the task is processed in the edge node, the private cloud center, or the cloud center platform according to the data processing requirements of the task, the data characteristics, and the current system state (delay, bandwidth, resource utilization, etc.).

[0067] 3. The intermediate agent platform evaluates the real-time computing resource information of the registered edge nodes and private cloud centers, including CPU, memory, network bandwidth, etc. indicators. In combination with the machine learning model, the future computing resource demand of the edge nodes and private cloud centers is predicted.

[0068] 4. The intermediate agent platform uses a resource scheduling algorithm to dynamically adjust resource allocation according to the current system state (delay, bandwidth, resource utilization, etc.) and the future predicted resource demand, in order to optimize the overall resource utilization. If the actual computing resource information is lower than the predicted computing resource demand information, the intermediate agent platform can timely recover the excess edge computing resources to provide services for other tasks. At the same time, the intermediate agent platform can also continuously optimize and adjust the machine learning model according to the actual situation to improve the accuracy of resource demand prediction. However, in the case of insufficient resources, that is, the real-time computing resource information is greater than the computing resource demand information, the intermediate agent platform can consider transferring part of the task to the cloud center platform for execution to meet the business requirements. Or directly allocate the difference in computing resources to the object from the personalized private cloud resource pool; the personalized private cloud resource pool can be generated based on idle computing resources, such as idle resources of edge nodes.

[0069] 5. The intermediate agent platform selects a suitable object to execute the computing task and sends a deployment instruction to the object. After receiving the computing result returned by the object, the intermediate agent platform recovers the edge computing resources to the preset personalized private cloud resource pool for easy rescheduling. It should be noted that the present scheme uses various open cloud interface APIs (Application Programming Interface, Application Programming Interface) to manage and allocate computing tasks to meet the needs of customized computing. For example, RESTful API, gRPC API, WebSocket API, which are related to resource management, data transmission, task control, etc. interfaces.

[0070] The existing resource management scheme has limitations in the dynamic nature and rapid response of resource allocation, especially when facing sudden high load or demand fluctuations, it cannot quickly adjust the resources, and often ignores the importance of real-time data and system state in dynamic allocation. To solve this problem, the scheme introduces an intermediate agent platform to integrate the analysis of data processing needs, the current state of the system and the predicted future resource demand, with the help of a dynamic resource scheduling algorithm, it can respond to any changes in resource demand in real time, greatly enhancing the flexibility and efficiency of resource allocation. This aims to tailor low-latency computing service experience for users, while efficiently utilizing otherwise idle resources, ensuring both optimized resource utilization and precise satisfaction of diverse business needs.

[0071] Referring to Figure 4 , another optional task processing method flowchart according to an embodiment of the present application is shown, comprising the following steps:

[0072] S401: Send deployment instructions to the edge node, so that the edge node pulls the image of the computing task from the intermediate agent platform for processing; wherein the edge node uses container or virtual machine technology to allocate independent computing environment and resources for the computing task;

[0073] S402: Receive the processing result returned by the edge node and return the processing result to the user end.

[0074] In the above embodiment, for step S401, when the object of processing the computing task is the edge node, the intermediate agent platform sends deployment instructions to the selected edge node, and the edge device pulls up the user-customized computing task image to start computing. The reasons for using images here are mainly as follows:

[0075] 1. Container or virtual machine technology can create an independent running environment for each task. Using images can quickly create and deploy these independent environments, improving deployment efficiency.

[0076] 2. The image contains a complete set of operating systems, dependent libraries, code, and other running environments required by the task. The edge node only needs to pull the corresponding image to quickly start the task running environment.

[0077] 3. Using images can ensure that each task has a consistent running environment, avoiding compatibility problems caused by environment differences. This can improve the stability and reliability of task processing.

[0078] 4. Image technology can also achieve fast migration and expansion and contraction of tasks. When the edge node's resources are insufficient, the task image can be quickly migrated to other edge nodes or the cloud for execution.

[0079] For step S402, the edge node allocates an independent computing environment and resources for each task using container or virtual machine technology, as shown in Figure 5 As shown in FIG. 1, the virtualization layer is a software technology that allows the creation and operation of multiple independent virtual machines on a physical hardware platform. This layer abstracts the underlying hardware so that each virtual machine can have its own operating system, processor, memory, network interface, etc., as if they were running on separate physical machines. Virtualization layer improves resource utilization, simplifies management complexity, provides isolation and flexibility. It allows multiple different operating systems and applications to run simultaneously on the same physical server without interfering with each other, which is very useful for testing, development, deployment and optimization of IT resource allocation.

[0080] It should be noted that the user end and the computing task are in a one-to-many relationship, i.e. one user end can create multiple computing tasks, and there can be multiple user ends. After preparing the required operating environment (i.e. task image) for the computing task according to the foregoing steps, the computing resources are allocated to the task image to meet the task requirements. The edge node returns the computing result to the intermediate agent platform, and the intermediate agent platform returns the computing result to the user end after summarizing the computing result.

[0081] The method provided by the above embodiment realizes efficient distribution and processing of computing task images by issuing deployment instructions to the edge node and using the intermediate agent platform. The edge node uses container or virtual machine technology to create an independent and isolated operating environment for each task, which not only ensures exclusive allocation and utilization efficiency of resources, but also enhances security, significantly reduces network latency, and improves response speed, thereby providing users with low-latency, efficient customized computing services. Once the processing is complete, the result is immediately fed back to the user end, ensuring smooth process and optimized user experience.

[0082] The entire implementation process is shown in FIG. 1. Figure 6 As shown in FIG. 1, the location relationship of the user end, edge node, private cloud center, and intermediate agent platform is clear. The intermediate agent platform can be developed and embedded in the cloud center platform or separated and connected to both sides. The intelligent agent collects real-time computing resource information of the edge node and the private cloud center and registers in the intermediate agent platform. The intermediate agent platform receives the computing task of the user end, matches and schedules resources based on the task characteristics, real-time computing resource information, and predicted computing resource information of the edge node and the private cloud center, and determines the object for processing the computing task, which can be the cloud platform or the edge node or the private cloud center. Finally, the object processing result is returned to the user end.

[0083] Referring to FIG. 1, Figure 7As shown in Fig. 7, a main module schematic diagram of a task processing device 700 provided by an embodiment of the present application is shown, which comprises: a receiving module 701 configured to receive a computing task transmitted by a user end, obtain data processing requirements from the computing task, and determine data characteristics according to the data processing requirements; an obtaining module 702 configured to call an intelligent agent pre-deployed at an edge node, and obtain real-time computing resource information of the edge node; a predicting module 703 configured to call a preset machine learning model, and predict computing resource requirement information of the edge node in a future preset time period respectively; a determining module 704 configured to determine an object for processing the computing task according to the data processing requirements, the data characteristics, the real-time computing resource information and the computing resource requirement information; wherein the object is an edge node or a cloud center platform; and a processing module 705 configured to send the computing task to the object for processing, receive a processing result returned by the object and return the processing result to the user end.

[0084] The device further comprises a training module configured to: receive historical computing resource information of the edge node collected by the intelligent agent in a historical preset time period and a collection time point; register and pre-process the historical computing resource information to obtain computing resource index data; and train a machine learning model by using the collection time point and the computing resource index data to obtain a trained machine learning model; wherein the machine learning model is iteratively updated based on the real-time computing resource information and the collection time point.

[0085] In the device, the determining module 704 is configured to: determine the object for processing the computing task from the edge node and the private cloud center according to the data processing requirements, the data characteristics and the real-time computing resource information; in a case where the real-time computing resource information is less than the computing resource requirement information, calculate a difference value between the two, and recycle the computing resource information of the difference value to a preset personalized private cloud resource pool; wherein the preset personalized private cloud resource pool is formed based on idle computing resources; in a case where the real-time computing resource information is greater than the computing resource requirement information, allocate the computing resource of the difference value from the preset personalized private cloud resource pool to the object; or if the object is the edge node, switch the computing task to the cloud center platform for processing.

[0086] The device further comprises a resource recycling module configured to: in response to the object ending processing the computing task, recycle the computing resource allocated to the object to the preset personalized private cloud resource pool.

[0087] The application implements a device, which is applied to an intermediate agent platform, and in a case where the object is an edge node, the processing module 705 is configured to: send a deployment instruction to the edge node, so that the edge node pulls an image of the computing task from the intermediate agent platform for processing; wherein the edge node allocates an independent computing environment and resources for the computing task by using a container or virtual machine technology; and receive a processing result returned by the edge node, and return the processing result to the user terminal.

[0088] In the device, the computing task is generated according to a data processing requirement configured by the user through a development kit, and the development kit is arranged in the user terminal.

[0089] In addition, the specific implementation of the device in the embodiment of the application has been described in detail in the above method, and therefore, the repeated content will not be described here.

[0090] Figure 8 An exemplary system architecture 800 to which embodiments of the application can be applied is shown, including terminal devices 801, 802, 803, a network 804, and a server 805 (only examples).

[0091] The terminal devices 801, 802, 803 can be various electronic devices with a display screen and supporting web browsing, and various communication user terminal applications are installed, and the user can use the terminal devices 801, 802, 803 to interact with the server 805 through the network 804 to receive or send messages, etc.

[0092] The network 804 is a medium for providing a communication link between the terminal devices 801, 802, 803 and the server 805. The network 804 can include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0093] The server 805 can be a server providing various services, such as a background management server (only an example) providing support for a shopping website browsed by the user using the terminal devices 501, 502, 503. The background management server can analyze and process received product information query requests, etc., and feed back the processing result (such as target push information, product information--only examples) to the terminal device. It should be noted that the method provided by the embodiment of the application is generally executed by the server 805, and correspondingly, the device is generally arranged in the server 805.

[0094] It should be understood that, Figure 8 The number of terminal devices, networks, and servers in the above-mentioned system architecture is only illustrative. According to the needs of implementation, there can be any number of terminal devices, networks, and servers.

[0095] The above will be described in detail below with reference to Figure 9It shows a schematic diagram of the structure of a computer system 900 suitable for implementing a terminal device of the present invention. Figure 9 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0096] like Figure 9 As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 902 or programs loaded from storage section 908 into random access memory (RAM) 903. The RAM 903 also stores various programs and data required for the operation of the system 900. The CPU 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0097] The following components are connected to I / O interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to I / O interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 910 as needed so that computer programs read from it can be installed into storage section 908 as needed.

[0098] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 909, and / or installed from removable medium 911. When the computer program is executed by central processing unit (CPU) 901, it performs the functions defined above in the system of this invention.

[0099] It should be noted that the computer-readable medium shown in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or component. In the present application, the computer-readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or component. The program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0100] The flowcharts and block diagrams in the drawings illustrate the possible implementation architectures, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment or a part of code containing one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different order than that shown in the drawings. For example, two blocks that are shown in succession can actually be executed substantially in parallel, and sometimes in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams or flowcharts, and the combination of blocks in the block diagrams or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0101] The modules described in the embodiments of the present application can be implemented in the form of software, or can be implemented in the form of hardware. The modules described can also be arranged in a processor, for example, a processor can be described as including a receiving module, an obtaining module, a predicting module, a determining module, and a processing module. In some cases, the names of the modules do not constitute a limitation on the modules themselves, for example, the determining module can also be described as a "resource scheduling module".

[0102] As another aspect, the present application also provides a computer readable medium, which can be included in the device described in the above embodiments, or can exist independently without being assembled into the device. The computer readable medium carries one or more programs, which, when executed by the device, cause the device to perform any of the task processing methods described above.

[0103] The computer program product of the present application includes a computer program, which, when executed by a processor, implements the task processing method in the embodiments of the present application.

[0104] The specific embodiments described above do not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A task processing method characterized by, The method comprises: receiving a computing task transmitted by a user terminal, obtaining data processing requirements from the computing task, and determining data characteristics according to the data processing requirements; calling an intelligent agent pre-deployed at an edge node and a private cloud center to obtain real-time computing resource information of the edge node and the private cloud center, respectively; calling a preset machine learning model to predict computing resource demand information of the edge node and the private cloud center in a future preset time period, respectively; determining an object for processing the computing task according to the data processing requirements, the data characteristics, the real-time computing resource information, and the computing resource demand information, wherein the object is one of the edge node, the private cloud center, and a cloud center platform; the process comprises: determining the object for processing the computing task according to the data processing requirements, the data characteristics, and the real-time computing resource information; in the case that the real-time computing resource information is less than the computing resource demand information, calculating a difference value between the two, and recycling the computing resource information of the difference value to a preset personalized private cloud resource pool; wherein the preset personalized private cloud resource pool is formed based on idle computing resources; in the case that the real-time computing resource information is greater than the computing resource demand information, allocating the computing resource of the difference value from the preset personalized private cloud resource pool to the object; or if the object is the edge node or the private cloud center, transferring the computing task to the cloud center platform for processing; sending the computing task to the object for processing, receiving a processing result returned by the object, and returning the processing result to the user terminal.

2. The method of claim 1, wherein, The method further comprises: receiving historical computing resource information of the edge node and the private cloud center collected by the intelligent agent in a historical preset time period and collection time points; registering and preprocessing the historical computing resource information to obtain computing resource index data; training a machine learning model using the collection time points and the computing resource index data to obtain a trained machine learning model; wherein the machine learning model is iteratively updated based on the real-time computing resource information and the collection time points.

3. The method of claim 1, wherein, After receiving the processing result returned by the object and returning the processing result to the user terminal, the method further comprises: in response to the object completing processing of the computing task, recycling the computing resource allocated to the object to the preset personalized private cloud resource pool.

4. The method of claim 1, wherein, Applied to an intermediate agent platform, in the case that the object is the edge node, the sending the computing task to the object for processing, receiving the processing result returned by the object, and returning the processing result to the user terminal comprises: sending a deployment instruction to the edge node to enable the edge node to pull an image of the computing task from the intermediate agent platform for processing; wherein the edge node uses container or virtual machine technology to allocate an independent computing environment and resources for the computing task; receiving the processing result returned by the edge node and returning the processing result to the user terminal.

5. The method of claim 1, wherein, The computing task is generated according to the data processing requirements configured by the user through a development kit, and the development kit is arranged in the user terminal.

6. A task processing apparatus characterized by comprising: The method comprises: a receiving module configured to receive a computing task transmitted by a user terminal, obtain data processing requirements from the computing task, and determine data characteristics according to the data processing requirements; An acquisition module is configured to call an intelligent agent pre-deployed at an edge node and a private cloud center to acquire real-time computing resource information of the edge node and the private cloud center, respectively. A prediction module is configured to call a preset machine learning model to predict computing resource requirement information of the edge node and the private cloud center in a future preset time period, respectively. A determination module is configured to determine an object for processing the computing task according to the data processing requirement, the data characteristic, the real-time computing resource information and the computing resource requirement information, wherein the object is one of the edge node, the private cloud center and a cloud center platform, and the process includes: determining the object for processing the computing task according to the data processing requirement, the data characteristic and the real-time computing resource information; in a case where the real-time computing resource information is less than the computing resource requirement information, calculating a difference between the two, and recycling the difference of the computing resource information to a preset personalized private cloud resource pool, wherein the preset personalized private cloud resource pool is formed based on idle computing resources; in a case where the real-time computing resource information is greater than the computing resource requirement information, allocating the difference of the computing resource to the object from the preset personalized private cloud resource pool; or if the object is the edge node or the private cloud center, transferring the computing task to the cloud center platform for processing. A processing module is configured to send the computing task to the object for processing, receive a processing result returned by the object and return the processing result to a user terminal.

7. The apparatus of claim 6, wherein, When the object is the edge node, the processing module is configured to: send a deployment instruction to the edge node, so that the edge node pulls an image of the computing task from the intermediate agent platform for processing; wherein the edge node uses container or virtual machine technology to allocate an independent computing environment and resources for the computing task; receive a processing result returned by the edge node and return the processing result to the user terminal.

8. An electronic device, comprising: The method comprises: one or more processors; a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1-5.

9. A computer readable medium having stored thereon a computer program, characterized in that The program is executed by the processor to implement the method of any one of claims 1-5.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1-5.

Images