Task scheduling method and system, electronic equipment and storage medium

Abstract

The invention relates to a task scheduling method and system, electronic equipment and a storage medium. The method comprises the following steps: respectively acquiring task attribute information corresponding to a plurality of to-be-processed tasks, and respectively acquiring resource attribute information corresponding to a plurality of edge nodes; based on the task attribute information and the resource attribute information, distributing the plurality of to-be-processed tasks to corresponding edge nodes for processing; in the task processing process, if it is monitored that the resource utilization rate of any edge node is overloaded, other edge nodes or center nodes are called for cooperative processing. According to the scheme, the task execution efficiency, the resource utilization rate and the task scheduling efficiency can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a task scheduling method and system, an electronic device and a storage medium. BACKGROUND

[0002] Task scheduling refers to a process of managing and controlling the execution order, resource allocation and timing of multiple tasks in a computer system according to a specific strategy or rule.

[0003] In terms of task scheduling, the related art mainly adopts static scheduling algorithms such as FCFS (First-Come, First-Served), SJF (Shortest Job First) and the like. However, these methods are mainly suitable for centralized, relatively stable resource scenarios, and are difficult to adapt to complex and changeable application scenarios such as wide resource distribution, strong dynamics and diversified task types in an edge computing environment, resulting in low task execution efficiency, low resource utilization and low task scheduling efficiency. SUMMARY

[0004] To solve or partially solve the problems in the related art, the present application provides a task scheduling method and system, an electronic device and a storage medium, which can improve task execution efficiency, resource utilization and task scheduling efficiency.

[0005] The first aspect of the present application provides a task scheduling method, comprising: obtaining task attribute information corresponding to a plurality of to-be-processed tasks respectively, and obtaining resource attribute information corresponding to a plurality of edge nodes respectively; based on the task attribute information and the resource attribute information, distributing the plurality of to-be-processed tasks to the corresponding edge nodes for processing respectively; In the task processing process, if it is monitored that the resource utilization of any edge node is overloaded, other edge nodes or a central node are called for cooperative processing.

[0006] In an embodiment, the task attribute information includes at least one of priority, deadline and resource requirement, and the resource attribute information includes at least one of CPU usage, memory occupancy, storage capacity, network bandwidth, network delay and network throughput; the distributing the plurality of to-be-processed tasks to the corresponding edge nodes for processing respectively based on the task attribute information and the resource attribute information comprises: determining a to-be-processed task belonging to a high priority or having a deadline less than or equal to a preset time threshold or a resource requirement greater than or equal to a preset resource threshold as a first target task; and, determining, as the second target task, a to-be-processed task that belongs to a low priority or has a deadline greater than a preset time threshold or has a resource requirement less than a preset resource threshold; and determining, from the plurality of edge nodes, a target edge node that has a CPU usage rate less than or equal to a first preset threshold, and / or a memory occupancy rate less than or equal to a second preset threshold, and / or a storage capacity greater than or equal to a third preset threshold, and / or a network bandwidth greater than or equal to a fourth preset threshold, and / or a network delay less than or equal to a fifth preset threshold, and / or a network throughput greater than or equal to a sixth preset threshold; allocating the first target task to the target edge node for processing; and predicting a resource idle period of each of the edge nodes, so as to allocate the second target task to the corresponding edge node for processing when the resource idle period is reached.

[0007] In an embodiment, in the task processing process, if it is monitored that the resource utilization rate of any of the edge nodes is overloaded, other edge nodes or a central node are called for cooperative processing, including: In the task processing process, if it is monitored that the resource utilization rate of any of the edge nodes is greater than or equal to a preset load threshold, the complexity of the task processed by the edge node is determined; if the complexity indicates that the task is a simple task, other edge nodes are called for cooperative processing; or if the complexity indicates that the task is a complex task, a central node is called for cooperative processing.

[0008] In an embodiment, the calling of the central node for cooperative processing includes: obtaining edge processing data generated by the edge node for the complex task; compressing the edge processing data to obtain lightweight processing data; transmitting the lightweight processing data to the central node through a message queue, so that the central node mines potential rules and change trends of the complex task based on the lightweight processing data; receiving cloud end processing data returned by the central node for the lightweight processing data through the message queue; the cloud end processing data includes operation instructions generated by the central node for the complex task; sending the operation instructions to the edge node for execution.

[0009] In an embodiment, when the to-be-processed task is an application recommendation task, the method further includes: performing semantic analysis on each application in the application market to obtain semantic feature information of each application; when monitoring that a user accesses the application market, analyzing preference feature information of the user according to historical behavior data of the user in the application market; the historical behavior data includes at least one of historical download records, historical use time length, and historical scores; matching the semantic feature information of each application with the preference feature information of the user to obtain a target application list; the target application list includes a plurality of target applications to be recommended, and each target application has a corresponding recommendation priority; According to the recommendation priority, a plurality of target applications are displayed to the user in order.

[0010] In an embodiment, each target application has a plurality of corresponding versions; and the displaying of the plurality of target applications to the user in order according to the recommendation priority comprises: For each target application, a target version adapted to a terminal device of the user is determined from the plurality of corresponding versions by a target edge node based on a device type and / or a network state of the terminal device; wherein the target edge node is an edge node used for processing the application recommendation task; According to the recommendation priority and the target version, a plurality of target applications are displayed to the user in order.

[0011] In an embodiment, the method further comprises: monitoring a task state of each to-be-processed task in a whole life cycle; the whole life cycle includes a task creation stage, a task scheduling stage, a task execution stage, and a result feedback stage; When monitoring that a task state of any to-be-processed task in any stage is abnormal, performing fault diagnosis and fault repair on the to-be-processed task.

[0012] The second aspect of the present application provides a task scheduling system, comprising: An attribute information acquisition module is configured to acquire task attribute information corresponding to a plurality of to-be-processed tasks respectively, and acquire resource attribute information corresponding to a plurality of edge nodes respectively; A task allocation module is configured to allocate the plurality of to-be-processed tasks to the corresponding edge nodes for processing based on the task attribute information and the resource attribute information; A resource scheduling module is configured to, in a task processing process, if monitoring that a resource utilization rate of any edge node is overloaded, call other edge nodes or a central node for collaborative processing.

[0013] The third aspect of the present application provides an electronic device, comprising: a processor; and a memory having stored executable codes, which when executed by the processor, cause the processor to perform the method as described above.

[0014] The fourth aspect of the present application provides a computer readable storage medium having stored executable codes, which when executed by the processor of an electronic device, cause the processor to perform the method as described above.

[0015] The technical solutions provided by the present application can include the following beneficial results: The solutions provided by the present application respectively acquire task attribute information corresponding to a plurality of to-be-processed tasks, and respectively acquire resource attribute information corresponding to a plurality of edge nodes; based on the task attribute information and the resource attribute information, the plurality of to-be-processed tasks are respectively distributed to the corresponding edge nodes for processing; in the task processing process, if it is monitored that the resource utilization of any edge node is overloaded, other edge nodes or a central node are called for cooperative processing. The present application disperses a plurality of to-be-processed tasks to a plurality of edge nodes with sufficient resources for processing by fully utilizing the resources of each edge node, so as to avoid centralized processing by the cloud, thereby improving the task execution efficiency and the resource utilization rate, and by monitoring the resource utilization rate of each edge node in the task execution process in real time, when it is monitored that the resource utilization of any edge node is overloaded, other edge nodes or a central node can be called for cooperative processing in time, so as to avoid task accumulation, thereby improving the task scheduling efficiency and the resource utilization rate.

[0016] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0017] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings, in which like reference characters refer to like parts throughout the several views, and in which:

[0018] Figure 1 is a flowchart of a task scheduling method shown in an embodiment of the present application; Figure 2 is another flowchart of a task scheduling method shown in an embodiment of the present application; Figure 3 is a structural diagram of a task scheduling system shown in an embodiment of the present application; Figure 4 is a structural diagram of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0019] Embodiments of the present application will be described in more detail with reference to the drawings. Although the embodiments of the present application are shown in the drawings, it is understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0020] The terminology used in the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used in the description of the application and the appended claims, the singular forms "a", "an" and "the" are intended to include plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0021] It should be understood that although the terms "first", "second", "third", etc. can be employed in this application to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one piece of information from another piece of information of the same type. For example, the first information can also be called the second information without departing from the scope of the present application, and similarly, the second information can also be called the first information. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise specifically limited.

[0022] In terms of task scheduling, static scheduling algorithms (such as FCFS, SJF, etc.) are mostly used in related technologies. Among them, FCFS (First Come First Served) is a simple scheduling algorithm, which serves in the order of arrival of jobs or processes, i.e. the first arriving task is executed first, and the later arriving task is queued. However, this will result in a long average waiting time, and long tasks will block the execution of subsequent short tasks. SJF (Shortest Job First) is an optimized scheduling algorithm, which prioritizes the execution of jobs or processes with the shortest estimated execution time, i.e. short tasks are executed first to reduce the average waiting time. However, it is difficult to estimate the execution time of tasks, and it is easy to cause long tasks to be executed for a long time.

[0023] In addition, the above two methods are mainly suitable for centralized, relatively stable scenarios, and are difficult to adapt to complex and variable application scenarios such as wide resource distribution, strong dynamics, and diverse task types in edge computing environments, resulting in low task execution efficiency, low resource utilization, and low task scheduling efficiency.

[0024] To solve the above problems, the embodiment of the present application provides a task scheduling method, which disperses multiple to-be-processed tasks to multiple edge nodes with sufficient resources for processing by fully utilizing the resources of each edge node, so as to avoid centralized processing by the cloud, thereby improving the task execution efficiency and resource utilization rate, and by monitoring the resource utilization rate of each edge node in the process of executing the task in real time, when the resource utilization rate of any edge node is overloaded, other edge nodes or the central node can be called in time for collaborative processing, so as to avoid task accumulation, thereby improving the task scheduling efficiency and resource utilization rate.

[0025] In order to facilitate understanding of the scheme provided by the embodiment of the present application, the terms related to the embodiment of the present application are explained first, as follows. Edge fusion: Edge fusion is a technical concept and implementation method that relies on edge computing architecture, combines cloud computing, network communication, security protection, and multi-dimensional capabilities such as data storage, and provides integrated intelligent services on edge nodes close to users. It breaks through the limitations of traditional cloud network separation by deeply integrating multiple resources at the physical or logical level, achieving efficient support for terminal devices, application scenarios, and business processes. This mode not only reduces transmission delay and improves response speed, but also enhances data security and privacy protection capabilities based on local processing, thereby providing more flexible, intelligent, and reliable service support for smart cities, industrial internet, home digital ecosystems, and other scenarios. The core value of edge fusion is to promote the evolution of computing power and services from "centralized" to "distributed + edge", building a new information service infrastructure for the future by resource collaboration and capability sinking.

[0026] Adaptive dynamic scheduling: Adaptive dynamic scheduling is an intelligent management mechanism based on real-time environment perception and intelligent analysis capabilities, which can automatically adjust resource allocation strategies and task execution order in complex and variable network and computing environments. This mechanism can be applied in cloud computing, edge computing, Internet of Things, and other scenarios, aiming to cope with challenges brought by changing user demands, system load, network bandwidth, and device status. Its core is to collect and analyze key indicators (such as CPU utilization, memory occupancy, network delay, task priority, etc.) during system operation, combined with machine learning, reinforcement learning, and other algorithm models, to dynamically optimize resource scheduling strategies to maximize service performance and optimize resource usage. Adaptive dynamic scheduling not only improves the flexibility and stability of the system, but also effectively reduces operation and maintenance costs and energy consumption, which is an important technical support for intelligent and automated operation and maintenance.

[0027] The technical scheme of the embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0028] Figure 1is a flowchart of a task scheduling method shown in embodiments of the present application.

[0029] Referring to Figure 1 The task scheduling method of the present application can include: S110, respectively acquiring task attribute information corresponding to a plurality of to-be-processed tasks, and respectively acquiring resource attribute information corresponding to a plurality of edge nodes.

[0030] The embodiments of the present application fuse edge computing, artificial intelligence, cloud computing and other technical means to build a task scheduling system with the ability of a closed loop of "perception-decision-execution-feedback", wherein the task scheduling system covers seven core technical modules: task intelligent arrangement based on deep reinforcement learning, edge node resource dynamic perception and adaptive allocation, edge and cloud collaborative computing, application market intelligent recommendation and precise distribution, task full life cycle state monitoring and intelligent feedback, edge data efficient preprocessing and compression transmission, and multi-modal task triggering and collaborative execution. Each technical module forms an organic linkage with edge node resource state (i.e. resource attribute information) and task feature data (i.e. task attribute information) as the link: edge node resource dynamic perception and adaptive allocation provides real-time resource data (i.e. resource attribute information) for task intelligent arrangement, edge and cloud collaborative computing provides computing power support for data preprocessing and model training, task full life cycle state monitoring provides abnormal diagnosis basis for intelligent feedback mechanism, multi-modal task triggering drives other technical modules to execute in order according to business logic, intelligent recommendation and precise distribution rely on task arrangement and resource allocation results to optimize application scheduling, edge data efficient preprocessing and compression transmission provides efficient data flow basis for the whole process and its lightweight processing data output supports edge and cloud collaborative computing and task intelligent arrangement. Each technical module forms a complete closed loop of "perception input-intelligent decision-collaborative execution-feedback optimization" through data flow and strategy linkage, ensuring low delay, high reliability and efficient use of resources, thereby improving task processing efficiency, resource utilization and service quality, and being applicable to various complex business scenarios such as industrial Internet of Things, smart city, smart home, etc.

[0031] Among them, the technical module of edge node resource dynamic perception and adaptive allocation aims to realize efficient management and precise allocation of edge node resources, and improve the overall performance and stability of the system. This technical module takes Prometheus (an open source alarm system and monitoring toolkit) and Grafana (an open source data visualization and monitoring tool) as the core to build a perfect real-time resource monitoring system, which can comprehensively and real-timely collect and monitor the use of hardware resources and network resources of edge nodes.

[0032] In a specific implementation, the task scheduling system can deploy an Exporter (a data collection component) in each edge node in advance through Prometheus, so that the resource attribute information of the edge node where each Exporter component is located can be periodically collected. The resource attribute information can include hardware resource information and network resource information. Each Exporter component can store the collected resource attribute information in a time series database, so that when the task scheduling system receives a plurality of to-be-processed tasks and their task attribute information, it can directly obtain the latest resource attribute information of each edge node from the time series database.

[0033] In addition, the task scheduling system can visualize the resource attribute information through Grafana. In a specific implementation, Grafana reads the resource attribute information of each edge node from the time series database, and then visualizes the resource attribute information in the Grafana interface, so that the operation and maintenance personnel can intuitively view the resource usage of each edge node, and also provide a data basis for subsequent analysis.

[0034] S120, based on the task attribute information and the resource attribute information, distributing the plurality of to-be-processed tasks to corresponding edge nodes for processing.

[0035] The technical module of task intelligent arrangement based on deep reinforcement learning aims to achieve optimal matching between tasks and resources, and improve task execution efficiency and system resource utilization. The technical module constructs a task scheduling agent based on DQN (Deep Q-Network) and its improved algorithm, so as to fully exert its advantages in dynamic decision-making, and at the same time, combines CNN (Convolutional Neural Network) and RNN (Recurrent Neural Network) to efficiently extract features from multi-dimensional data.

[0036] Specifically, the task scheduling system is built-in with a feature extraction model and a task scheduling agent, wherein the feature extraction model is a model fusing CNN and RNN. After obtaining the task attribute information of each to-be-processed task and the resource attribute information of each edge node, the task scheduling system can first encode the task attribute information and the resource attribute information into a state vector, and then input the state vector into the feature extraction model, so as to extract spatial features from the state vector by the CNN in the feature extraction model, and extract time series features from the state vector by the RNN in the feature extraction model, thereby providing a comprehensive and accurate data basis for the subsequent decision-making of the task scheduling agent. The task scheduling system can input the spatial features and the time series features into the task scheduling agent, so as to generate a scheduling action by the task scheduling agent according to a policy network, wherein the scheduling action is to allocate the plurality of to-be-processed tasks to the corresponding edge nodes for processing.

[0037] In the task processing process, if it is monitored that the resource utilization rate of any edge node is overloaded, S130, the other edge nodes or the central node are called for collaborative processing.

[0038] In the resource allocation decision link, the task scheduling system introduces a game theory model, the resource competition and cooperation relationship between the plurality of edge nodes can be abstracted as a game process, and a scientific and reasonable resource allocation strategy is determined by solving the game equilibrium solution. At the same time, the task scheduling system deeply fuses the Kubernetes (container orchestration tool) scheduling concept, models and optimizes the resource allocation problem by means of linear programming algorithm, and combines the edge prediction model to make an early prediction on the future resource change trend, so as to dynamically adjust the scheduling strategy and realize the forward-looking allocation of resources.

[0039] The edge prediction model is trained by using a time series analysis algorithm such as LSTM (Long Short-Term Memory) on historical resource usage data, so as to learn the change rule and trend of the data. Therefore, in the process of processing the corresponding task by each edge node, the task scheduling system can use the edge prediction model to predict the resource usage of each edge node in the future time period, and obtain the resource demand prediction data of each edge node. If the resource demand prediction data of a certain edge node indicates that its own resource utilization rate is insufficient, it means that the edge node is still resource sufficient; if the resource demand prediction data of a certain edge node indicates that its own resource utilization rate is overloaded, it means that the edge node is in a resource emergency. In this case, the task scheduling system can allocate resources from the resource sufficient edge nodes to the edge node in a resource emergency for collaborative processing.

[0040] In a specific implementation, the task scheduling system can use a linear programming algorithm to convert the resource allocation problem into a mathematical model, taking maximizing resource utilization and minimizing edge node load difference as the objective function, combining resource total quantity restrictions, task resource requirements, and other constraint conditions, to solve the optimal resource allocation scheme. The game theory model considers the mutual influence between multiple edge nodes, regards each edge node as a game participant, and regards the resource allocation strategy of the edge node as a game action. By calculating the Nash equilibrium and other game equilibrium solutions, the best resource allocation strategy of each edge node under the resource competition and cooperation relationship is determined. The task scheduling system combines the linear programming solution and the game theory model decision result, dynamically adjusts the resource allocation between edge nodes, so as to automatically allocate resources from other edge nodes (i.e., edge nodes with insufficient resource utilization) when the resource utilization of a certain edge node is overloaded, to achieve dynamic balance and efficient use of resources, improve resource utilization, and effectively avoid task accumulation, and improve task scheduling efficiency.

[0041] In addition to being able to call other edge nodes for collaborative processing (referred to as "edge-edge collaboration"), the embodiments of the present application can also call the center node (i.e., the cloud server) for collaborative processing (referred to as "cloud-edge collaboration"). Through efficient collaborative computing between the edge and the cloud, both the real-time low-latency requirements of business scenarios and the overall intelligent level of the task scheduling system can be improved through the deep analysis capability of the cloud server.

[0042] In one example, assuming that the resource utilization of edge node A is overloaded (e.g., high CPU usage), and edge node A contains 5 tasks to be processed, the task scheduling system can call edge node B with insufficient resource utilization (e.g., low CPU usage) for collaborative processing (suitable for simple tasks), or can call the center node for collaborative processing (suitable for complex tasks) to avoid the accumulation of the 5 tasks to be processed in edge node A, thereby improving task scheduling efficiency and resource utilization.

[0043] As can be seen from the example, the scheme provided by the application respectively acquires task attribute information corresponding to a plurality of to-be-processed tasks, and respectively acquires resource attribute information corresponding to a plurality of edge nodes; based on the task attribute information and the resource attribute information, the plurality of to-be-processed tasks are respectively distributed to the corresponding edge nodes for processing; in the task processing process, if it is monitored that the resource utilization rate of any edge node is overloaded, other edge nodes or a central node are called for cooperative processing. The application disperses the plurality of to-be-processed tasks to a plurality of edge nodes with sufficient resources for processing by fully utilizing the resources of each edge node, so as to avoid centralized processing by the cloud, thereby improving the task execution efficiency and the resource utilization rate, and by monitoring the resource utilization rate of each edge node in the process of executing the task in real time, when it is monitored that the resource utilization rate of any edge node is overloaded, other edge nodes or a central node can be called for cooperative processing in time, so as to avoid task accumulation, thereby improving the task scheduling efficiency and the resource utilization rate.

[0044] Figure 2 is another flowchart of the task scheduling method shown in the application.

[0045] Referring to Figure 2 The task scheduling method of the application can include: S210, respectively acquiring task attribute information corresponding to a plurality of to-be-processed tasks, and respectively acquiring resource attribute information corresponding to a plurality of edge nodes.

[0046] This step can refer to the description in S110, which will not be repeated here.

[0047] S220, based on the task attribute information and the resource attribute information, respectively distributing the plurality of to-be-processed tasks to the corresponding edge nodes for processing.

[0048] This step can refer to the description in S120, which will not be repeated here.

[0049] In an embodiment, the task attribute information includes at least one of a priority, a deadline, and a resource requirement, the resource attribute information includes at least one of a CPU usage rate, a memory occupancy rate, a storage capacity, a network bandwidth, a network delay, and a network throughput; based on the task attribute information and the resource attribute information, respectively distributing the plurality of to-be-processed tasks to the corresponding edge nodes for processing can include: determining, as first target tasks, the pending tasks belonging to a high priority or having a deadline less than or equal to a preset time threshold or having a resource requirement greater than or equal to a preset resource threshold; and determining, as second target tasks, the pending tasks belonging to a low priority or having a deadline greater than the preset time threshold or having a resource requirement less than the preset resource threshold; and determining, from the plurality of edge nodes, a target edge node having a CPU usage less than or equal to a first preset threshold, and / or a memory occupancy less than or equal to a second preset threshold, and / or a storage capacity greater than or equal to a third preset threshold, and / or a network bandwidth greater than or equal to a fourth preset threshold, and / or a network delay less than or equal to a fifth preset threshold, and / or a network throughput greater than or equal to a sixth preset threshold; assigning the first target tasks to the target edge node for processing; and predicting a resource idle period of each edge node so as to assign the second target tasks to the corresponding edge node for processing when the resource idle period is reached.

[0050] In the embodiments of the present application, the to-be-processed tasks cover multiple priority tasks, multi-modal trigger tasks, and resource requirement diversified tasks, etc. The multiple priority tasks can include high priority tasks and low priority tasks, wherein the high priority tasks have extremely high requirements for real-time performance and reliability, such as device fault early warning tasks in industrial Internet of Things, emergency response tasks (such as dynamic regulation of traffic lights, fire alarm linkage) in smart city, security protection trigger tasks (such as abnormal intrusion alarm processing) in smart home, etc. Such tasks need to be quickly responded (the target response time is shortened to milliseconds or seconds) and preferentially occupy resources. The low priority tasks have lower requirements for real-time performance, but need to efficiently utilize idle resources, such as device state periodic reporting tasks, historical data statistical analysis tasks, and non-urgent update push tasks in application market, etc. Such tasks need to improve resource utilization without affecting high priority tasks. The multi-modal trigger task is a task triggered based on multiple event sources, including timing tasks, sensor event tasks, database change tasks, and user interaction tasks, wherein the timing tasks can include device periodic inspection, data backup, and other periodic tasks; the sensor event tasks can include adjustment tasks triggered by temperature / pressure exceeding of industrial sensors, and abnormal data processing tasks of smart city environment monitoring sensors; the database change tasks can include recommendation model optimization tasks triggered by user download quantity update in application market, and scheduling strategy adjustment tasks triggered by resource threshold (such as first preset threshold, second preset threshold, third preset threshold, fourth preset threshold, fifth preset threshold, and sixth preset threshold) change of edge node; the user interaction tasks can include download request and function calling instruction of user in application market, etc.

[0051] Based on this, the task attribute information can include but is not limited to: priority, deadline, resource requirement, etc. If a to-be-processed task belongs to a high-priority task (such as dynamic regulation of traffic signal lights), since the high-priority task has extremely high requirements on real-time performance and reliability, the task scheduling agent can determine the to-be-processed task as a first target task that needs to be responded quickly; if a to-be-processed task belongs to a low-priority task (such as non-urgent update push of an application market), since the low-priority task has relatively low requirements on real-time performance, the task scheduling agent can determine the to-be-processed task as a second target task that does not need to be responded quickly; if the deadline of a to-be-processed task is less than a preset time threshold (such as abnormal data of a smart city environment monitoring sensor needs to be processed immediately), it means that the time is urgent, and the task scheduling agent determines the to-be-processed task as a first target task that needs to be responded quickly; if the deadline of a to-be-processed task is greater than a preset time threshold (such as a device regular inspection task has a week before the deadline), it means that the time is generous, and the task scheduling agent can determine the to-be-processed task as a second target task that does not need to be responded quickly; if the resource requirement of a to-be-processed task is greater than a preset resource threshold (such as traffic camera picture recognition in a smart city), it means that more resources need to be provided for support, and the task scheduling agent can determine the to-be-processed task as a first target task that needs to occupy resources preferentially; if the resource requirement of a to-be-processed task is less than a preset resource threshold (such as a light task), it means that only a small amount of resources need to be provided for support, and the task scheduling agent can determine the to-be-processed task as a second target task that needs to utilize idle resources efficiently.

[0052] The resource attribute information can include hardware resource information and network resource information. The hardware resource information can include, but is not limited to, CPU usage, memory occupancy, and storage capacity, etc. The network resource information can include, but is not limited to, network bandwidth, network delay, and network throughput, etc. The CPU (Central Processing Unit) usage represents the proportion of time that the CPU executes non-idle tasks in a certain time period. High CPU usage can mean that the edge node is processing a large number of tasks, while low CPU usage can indicate that the edge node is in an idle state. The memory occupancy is the proportion of the currently used RAM (Random Access Memory) to the total available memory, which reflects the consumption of memory resources when the edge node processes tasks. Excessive memory occupancy can slow down the edge node. The storage capacity is the total space size of the computer storage device (such as HDD (Hard Disk Drive), SSD (Solid State Drive), etc.) that can save data, which determines how much data the edge node can store. The network bandwidth is the maximum amount of data that a network link can transmit in a unit of time. The higher the network bandwidth, the faster the network transmission speed in theory. The network delay is the time required for data to travel from the sending end to the receiving end, which determines the transmission response time. The network throughput is the amount of effective data successfully transmitted in a unit of time in the actual network environment, which is jointly constrained by network bandwidth and network delay, and reflects the real transmission efficiency.

[0053] Therefore, the task scheduling agent can determine a target edge node with CPU usage ≤ a first preset threshold, and / or memory occupancy ≤ a second preset threshold, and / or storage capacity ≥ a third preset threshold, and / or network bandwidth ≥ a fourth preset threshold, and / or network delay ≤ a fifth preset threshold, and / or network throughput ≥ a sixth preset threshold, from multiple edge nodes, so as to assign a first target task (such as a high-priority task) requiring fast response and preferential resource occupation to the target edge node for processing, thereby preferentially guaranteeing resource supply and ensuring fast response.

[0054] For the second target task (e.g., a low-priority task), the task scheduling agent considers the resource occupation of the first target task (e.g., a high-priority task) and uses the DQN network and its improved algorithm to allocate the low-priority task when the resource of the edge node is idle, so as to realize efficient execution of the low-priority task. In specific implementation, the task scheduling agent can input the task attribute information and the resource attribute information into the edge prediction model, so as to predict the resource idle period of each edge node through the edge prediction model. Then, when the resource idle period of any edge node is reached, the task scheduling agent can allocate the second target task (e.g., a low-priority task) to the edge node for processing, so as to maximize the resource utilization rate and realize the dual optimization of task execution efficiency and resource utilization rate without affecting the core business.

[0055] In addition, the embodiments of the present application also carefully design the reward mechanism to continuously optimize the scheduling strategy, so as to continuously improve the rationality and efficiency of task allocation. In specific implementation, during the task processing, the task scheduling system monitors the completion of the first target task of each edge node in real time. If a certain edge node completes the data processing and result feedback of the first target task on time, the task scheduling system can give positive reward to the edge node, for example, increase the number of allocated first target tasks / increase more resource allocation for the edge node; if a certain edge node does not complete the data processing and result feedback of the first target task on time, the task scheduling system can give negative reward to the edge node, for example, reduce the number of allocated first target tasks for the edge node, so as to promote the optimization of the scheduling strategy and ensure the fast response of the first target task (e.g., a high-priority task).

[0056] It can be seen that the task scheduling system continuously repeats the closed-loop process of state perception, feature extraction, strategy decision and reward feedback, and dynamically adjusts the task allocation strategy according to the business demand, so as to meet the demand of complex and changeable business scenarios and realize intelligent allocation and efficient execution of all-scene tasks.

[0057] S230, during the task processing, if the resource utilization rate of any edge node is greater than or equal to the preset load threshold, the complexity of the task processed by the edge node is determined.

[0058] In the process of processing the corresponding task at each edge node, the task scheduling system can use the edge prediction model to respectively predict the resource usage of each edge node in the future time period, obtaining resource demand prediction data of each edge node, including resource utilization. The task scheduling system can compare the resource utilization of each edge node with the preset load threshold respectively. If the resource utilization of a certain edge node is greater than or equal to the preset load threshold, the task scheduling system monitors that the resource utilization of the edge node is overloaded; if the resource utilization of a certain edge node is less than the preset load threshold, the task scheduling system monitors that the resource utilization of the edge node is insufficient.

[0059] The task scheduling system can determine the complexity of the task corresponding to the edge node with overloaded resource utilization. If the complexity represents a simple task, since the edge node has certain analysis capability and meets the low delay requirement of real-time, for a simple task, it can enter step S240; if the complexity represents a complex task, since the center node has deep analysis capability, for a complex task, it can enter step S250.

[0060] S240, if the complexity represents a simple task, other edge nodes are called for collaborative processing.

[0061] For a simple task, the task scheduling system can call other edge nodes for collaborative processing. In specific implementation, the task scheduling system can use a linear programming algorithm to convert the resource allocation problem into a mathematical model, taking maximizing resource utilization and minimizing edge node load difference as the objective function, combining resource total quantity restriction, task resource demand and other constraint conditions, to solve the optimal scheme of resource allocation. The game theory model considers the mutual influence among multiple edge nodes, regards each edge node as a game participant, and regards the resource allocation strategy of the edge node as a game action. By calculating the game equilibrium solution such as Nash equilibrium, the best resource allocation strategy of each edge node under the resource competition and cooperation relationship is determined. The task scheduling system combines the linear programming solution result and the game theory model decision result, dynamically adjusts the resource allocation among edge nodes, so as to automatically allocate resources from other edge nodes (i.e. edge nodes with insufficient resource utilization) when monitoring that the resource utilization of a certain edge node is overloaded, realizes dynamic balance and efficient utilization of resources, improves resource utilization, and can effectively avoid task accumulation, and improves task scheduling efficiency.

[0062] S250, if the complexity represents a complex task, a center node is called for collaborative processing.

[0063] For complex tasks, the task scheduling system can call the center node for collaborative processing. Specifically, the edge and cloud collaborative computing technology module aims to build an efficient and flexible cloud-edge collaborative computing system, taking into account the low latency and deep analysis capabilities of computing. This technology module is based on the Service Mesh (Service Network) architecture and designs a collaborative service governance mechanism for the edge and cloud. Its core relies on components such as Istio (an open-source service mesh system) and Apache Kafka (an open-source messaging system project) to achieve service management and data interaction.

[0064] At the service governance level, Istio serves as a key tool for implementing service routing and traffic control. By defining virtual services and destination rules within the service mesh, Istio can precisely control the flow of service requests between the edge and the cloud. Based on factors such as service load and version information, Istio dynamically allocates traffic to ensure the stability and efficiency of service calls. Apache Kafka is used to implement asynchronous communication and data caching between the edge and the cloud. Edge processing data generated by edge nodes can be sent to Kafka (a message queue) in real time, and cloud servers (i.e., center nodes) can retrieve edge processing data from Kafka as needed for processing. This approach effectively decouples the computing processes of the edge and the cloud, and the message persistence feature of Kafka allows temporary caching of data to prevent data loss. In an embodiment, calling the center node for collaborative processing can include: obtaining edge processing data generated by an edge node for a complex task; compressing the edge processing data to obtain lightweight processing data; transmitting the lightweight processing data to the center node through a message queue, so that the center node mines potential patterns and trends of the complex task based on the lightweight processing data; receiving cloud processing data returned by the center node for the lightweight processing data through the message queue; the cloud processing data includes operation instructions generated by the center node for the complex task; and sending the operation instructions to the edge node for execution.

[0065] The task scheduling system can obtain edge processing data generated by an edge node with overloaded resource utilization for a complex task. Before transmitting the edge processing data to a cloud server (i.e., a central node) through Kafka (a message queue), the task scheduling system can preprocess and compress the edge processing data to improve data transmission efficiency. Specifically, the technical module of efficient preprocessing and compressed transmission of edge data aims to improve the data processing capability of the edge node, reduce network pressure, and realize efficient transmission and utilization of edge processing data. The technical module constructs a data preprocessing pipeline at the edge based on Apache Beam (an open-source data processing framework). Apache Beam is a unified programming model that supports multiple data processing engines and can efficiently execute data processing tasks in different computing environments. Through Apache Beam, the edge processing data can be conveniently filtered, aggregated, normalized, and other preprocessing operations can be performed to remove noise and redundant information in the edge processing data and extract valuable data features. In specific implementation, the data preprocessing pipeline of Apache Beam is deployed at the edge node to input the edge processing data into the pipeline, so that the edge processing data is sequentially filtered, aggregated, normalized, and other preprocessing operations are performed. The filtering operation can remove data that does not meet the requirements in the edge processing data, the aggregation operation can merge and count similar data, and the normalization operation can convert the edge processing data into a unified format and range.

[0066] In addition, the embodiments of the present application can also deploy a lightweight Flink (a unified computing framework combining batch processing and stream processing) cluster on the edge side for preprocessing of edge processing data. In specific implementation, the Flink cluster is connected to various edge computing devices through a connector to continuously obtain edge processing data. The Flink cluster uses the stream processing capability of Flink to perform real-time preprocessing operations such as cleaning, filtering, and aggregation on the edge processing data to remove abnormal data, aggregate and count scattered similar data, reduce data redundancy and invalid information, and reduce data transmission pressure.

[0067] The task scheduling system can compress the preprocessed edge processing data to obtain lightweight processing data. Specifically, in terms of data compression, the task scheduling system combines efficient compression algorithms such as Snappy and Zstandard to realize lossless compression of data. Snappy is a fast compression algorithm with high compression speed, suitable for compressing data with high real-time requirements. Zstandard is a general-purpose compression algorithm that balances compression ratio and compression speed, and can efficiently compress various types of data.

[0068] Based on this, the task scheduling system can select a suitable compression algorithm according to the characteristics and application scenarios of the data to compress the preprocessed edge processing data, so as to reduce the volume of the data. For example, for data with high real-time requirements, the task scheduling system can use the Snappy algorithm to compress the preprocessed edge processing data to obtain lightweight processing data; for data that requires a higher compression ratio, the task scheduling system can use the Zstandard algorithm to compress the preprocessed edge processing data to obtain lightweight processing data.

[0069] After that, the task scheduling system can send the lightweight processing data to Kafka (message queue) so that the cloud server (i.e. the center node) can obtain the lightweight processing data from Kafka as needed for processing. Since the amount of data transmitted is greatly reduced, the network bandwidth occupation can be effectively reduced. For example, in the power grid sensor data scenario, for 50MB of edge processing data, after preprocessing and compression processing, the data volume can be reduced to 4MB, and the transmission time is shortened from 400 seconds to 32 seconds, which improves the effective data transmission efficiency in the NB-IoT (Narrowband Internet of Things) scenario by 92%, and significantly enhances the feasibility of supporting massive device access.

[0070] The cloud server is built with a Spark (open source cluster computing environment) platform to undertake the lightweight processing data transmitted from Kafka for deep mining of complex tasks. Specifically, Spark can perform tasks such as machine learning model training and large-scale data correlation analysis due to its powerful distributed computing and in-memory computing capabilities. After Spark obtains the lightweight processing data from Kafka, it performs deep analysis on the lightweight processing data according to the pre-set algorithm and business logic. In specific implementation, the cloud prediction model can be pre-trained based on historical data, so that in actual application, the cloud server can perform deep analysis on the lightweight processing data through the cloud prediction model to mine the potential rules and trends of complex tasks, and obtain operation instructions for complex tasks, including decision results or model parameters and other key information. The cloud server can send the operation instructions to Kafka as cloud processing data.

[0071] The task scheduling system obtains the cloud processing data from Kafka, and then extracts the operation instructions from the cloud processing data to send the operation instructions to the edge node with overloaded resource utilization, which executes the operation instructions.

[0072] It can be seen that, through efficient collaborative processing between the edge and the cloud, the embodiments of the present application can not only meet the low delay requirement of real-time business scenarios, but also improve the overall intelligent level of the task scheduling system through the deep analysis capability of the cloud server.

[0073] Taking the traffic signal dynamic regulation task as an example, the collaborative processing of the edge node and the cloud server can realize efficient traffic signal dynamic regulation through the following process (including steps S1-S4), thereby meeting the low delay requirement of real-time traffic response and optimizing the overall scheduling strategy through the deep analysis capability of the cloud server: S1, local real-time data collection and processing of the edge node (low delay execution): (1) Deployment of the edge node: Deploy edge computing devices (such as edge servers or intelligent camera gateways) at each intersection in the city, and access terminal devices such as traffic cameras, vehicle flow sensors, and pedestrian detectors at the intersection.

[0074] (2) Collection and processing of local real-time data: The edge node collects multi-dimensional data at the intersection every second and processes the multi-dimensional data, for example: the edge node identifies vehicle type, vehicle quantity, and vehicle speed based on real-time video streams collected by traffic cameras; the edge node counts the number of vehicles passing through each lane in a unit of time based on vehicle flow collected by vehicle flow sensors; and the edge node determines a pedestrian red light violation event based on trigger signals collected by a pedestrian detector.

[0075] S2, preprocessing and compression processing of edge processing data: (1) Preprocessing of edge processing data: The task scheduling system can perform preprocessing operations on edge processing data through a lightweight Flink cluster, for example: performing lightweight target detection on video streams, filtering invalid pictures (such as stationary objects and non-traffic participants), extracting key features (such as straight / left turn lane congestion level and vehicle queue length), and then performing aggregation calculation on edge processing data to generate real-time traffic indicators (such as vehicle flow density, average speed, and pedestrian waiting time in each direction at the current intersection).

[0076] (2) If a sudden situation (such as a traffic accident causing lane congestion or a dense pedestrian crossing) is detected, it is immediately marked as a “high priority event”.

[0077] (3) Compression processing of edge processing data: Since the traffic signal dynamic regulation task has high real-time requirements, the task scheduling system can use the Snappy algorithm to compress the preprocessed edge processing data (such as 200KB) to obtain lightweight processing data (such as 50KB), which can include traffic indicators and high priority event markers.

[0078] S3, Edge and Cloud Data Interaction and Collaborative Decision-making (Efficient Linkage): (1) Data transmission from edge to cloud: The task scheduling system transmits lightweight processed data (such as traffic indicators, high-priority event markers) to the cloud server through Apache Kafka message queue asynchronous transmission, as follows: Normal period: send traffic indicators to the cloud server every 30 seconds; Emergency situation: immediately push high-priority event details (such as "an accident occurred in the east-west lane, with a queue length exceeding 50 meters"), triggering the cloud's rapid response.

[0079] (2) Cloud deep analysis and instruction generation: After receiving lightweight processed data, the cloud server's Spark platform conducts deep mining combined with historical data and global traffic models, as follows: The cloud prediction model is based on historical data (such as weekday morning peak traffic patterns) to predict the traffic flow trend of the intersection and associated road segments (such as upstream and downstream intersections) in the next 10 minutes; The cloud server calls the global traffic optimization model (such as a signal timing algorithm based on reinforcement learning), combined with real-time data from surrounding intersections, to generate dynamic timing schemes (such as extending the green light duration in the congested direction and shortening the red light duration in the empty direction); For sudden accidents, generate emergency dispatch strategies (such as temporarily adjusting surrounding intersection signals to guide traffic diversion).

[0080] S4, Edge node executes instructions and real-time feedback (closed-loop optimization): (1) Cloud-to-edge instruction issuance: The cloud server sends the optimized signal timing scheme (such as "extend the east-west green light by 15 seconds and shorten the south-north red light by 5 seconds") to Kafka as an operation instruction, which is received by the task scheduling system from Kafka and pushed to the edge node.

[0081] (2) Edge node local execution: The edge node responds to the operation instruction and controls the intersection signal controller to execute the adjustment. In addition, the edge node can collect real-time road condition data (such as the change in lane queue length and the improvement in traffic efficiency) after the execution.

[0082] (3) Feedback and strategy iteration: The edge node feeds back the instruction execution effect data (such as "after extending the green light, the east-west lane traffic efficiency has improved by 30%") to the cloud server, which updates the model parameters based on the new data to continuously optimize subsequent dispatch strategies.

[0083] As can be seen from the above examples, the edge and cloud collaborative processing has the advantages of low delay response, global intelligent optimization, and efficient resource utilization. For example, the local processing and instruction execution time of the edge node can be controlled within 100 milliseconds, avoiding the delay of traditional "terminal-cloud-terminal" full-link transmission (usually hundreds of milliseconds to seconds), ensuring that the sudden traffic conditions can be quickly handled. Global intelligent optimization can be achieved by integrating multi-intersection data and historical experience to generate a timing scheme that not only adapts to a single intersection, but also optimizes the regional traffic network (such as reducing the overall congestion time of the main road), improving the overall intelligence level of the task scheduling system. Efficient resource utilization can be achieved by filtering redundant data (such as raw video stream of several MB per second, only several hundred KB after preprocessing), which can greatly reduce the network transmission pressure, and the cloud server focuses on complex analysis rather than real-time response, with clear division of labor, which can greatly improve the utilization rate of computing power. Therefore, the "real-time response" of the edge node and the "deep analysis" of the cloud server form a closed loop, perfectly adapting to the dual demands of intelligent transportation for "low delay" and "high intelligence".

[0084] In an embodiment, when the to-be-processed task is an application recommendation task, the method can further include: performing semantic analysis on each application in the application market to obtain semantic feature information of each application; when it is monitored that a user accesses the application market, analyzing preference feature information of the user according to historical behavior data of the user in the application market; the historical behavior data includes at least one of historical download records, historical use time length, and historical scores; matching the semantic feature information of each application with the preference feature information of the user to obtain a target application list; the target application list includes a plurality of target applications to be recommended, and each target application has a corresponding recommendation priority; and displaying the plurality of target applications to the user in order according to the recommendation priority.

[0085] In addition, in the related art, the whole life cycle management of a task from creation to completion is lacking in the context of integrating an application market, so that collaborative optimization in the task scheduling and execution process cannot be achieved, affecting the stability and service quality of the overall system, resulting in system response lag and unstable service quality, especially in scenarios such as industrial Internet of Things and smart city that require high real-time and reliability, which is particularly prominent.

[0086] To this end, the embodiments of the present application provide two core technical modules of application market intelligent recommendation and accurate distribution, and task full life cycle state monitoring and intelligent feedback. Among them, the technical module of application market intelligent recommendation and accurate distribution aims to solve the problem of "information overload" in the application market in related technologies, so as to improve the efficiency of users obtaining high-quality applications and enhance the activity and stickiness of users. The technical module takes the BERT (Bidirectional Encoder Representation from Transformers, pre-trained language representation model) of the Transformer structure as the core for application semantic understanding. The BERT model can deeply analyze the text information such as the description, function and label of the application, accurately extract the semantic feature information of the application, and lay the foundation for subsequent personalized recommendation, thanks to its powerful pre-training ability and the ability to capture context information. At the same time, a user-application matching model is constructed in combination with LightGBM (Light Gradient Boosting Machine, efficient gradient boosting framework). The LightGBM can quickly and accurately analyze the historical behavior data of the user (such as historical download records, historical use time, historical ratings, etc.), and mine the preference feature information (such as interest, demand, etc.) of the user. By matching the semantic feature information extracted by the BERT model with the preference feature information analyzed by the LightGBM, personalized application recommendation can be realized, and the target application meeting the interest and demand of each user can be accurately pushed.

[0087] In a specific implementation, for the application recommendation task, the task scheduling system can perform semantic analysis on all applications in the application market by using the BERT model to extract the key semantic feature information of each application, and store the semantic feature information in the database. At the same time, the historical behavior data (such as historical download records, historical use time, historical ratings, etc.) of each user in the application market is collected, and the historical behavior data is trained by using LightGBM to build a user preference model (or user-application matching model), so that when any user accesses the application market, the task scheduling system can quickly and accurately analyze the preference feature information of the user based on the historical behavior data (such as historical download records, historical use time, historical ratings, etc.) of the user in the application market through the user preference model, and the task scheduling system can extract the semantic feature information of each application from the database to match the semantic feature information of each application with the preference feature information of the user, and obtain a target application list containing multiple target applications to be recommended, wherein each target application has a corresponding recommendation priority. The higher the recommendation priority, the more the corresponding target application meets the interests and needs of the user, so the task scheduling system can sort and display these target applications to the user according to the recommendation priority from high to low.

[0088] In an embodiment, each target application has a plurality of versions corresponding thereto; and sorting and displaying the plurality of target applications to the user according to the recommendation priority can comprise: For each target application, determining a target version adapted to the terminal device from the plurality of versions corresponding thereto by the target edge node based on the device type and / or network state of the terminal device of the user; wherein the target edge node is an edge node used for processing the application recommendation task; and sorting and displaying the plurality of target applications to the user according to the recommendation priority and the target version.

[0089] Each target application contains a plurality of versions (such as a lightweight version, a complete version, etc.), and in the target application distribution stage, the task scheduling system can realize intelligent distribution based on Docker (application container engine) image packaging and CDN (Content Delivery Network, content distribution network) network. The Docker image can package the target application and its dependencies to ensure the consistency and portability of the target application in different environments; and the CDN network dynamically selects the optimal version of the target application for pushing according to the terminal device information (such as device type, network state, etc.) of the user by using its widely distributed edge nodes.

[0090] In a specific implementation, when processing the application recommendation task, the target edge node can monitor the device type of the terminal device used by the user when accessing the application market and the network state thereof in real time due to its proximity to the terminal device of the user, so as to determine the target version of each target application corresponding to the current terminal device from a plurality of versions of each target application based on the device type and / or the network state. After the task scheduling system obtains the target version of each target application corresponding to the target edge node, the target applications can be ranked and displayed to the user in descending order of the recommendation priority and the corresponding target version.

[0091] In an example, it is assumed that the user accesses the application market using a terminal device X, and the target application list generated for the user includes target applications A-E, wherein the size relationship of the recommendation priority of the target applications is: target application A > target application C > target application E > target application D > target application B, and the target applications all include a plurality of versions (such as a lightweight version, a complete version, etc.). If the device type of the terminal device X is a high-configuration device, the task scheduling system can push the complete version with richer functions for the target applications A-E, and push in descending order of the recommendation priority. If the network state of the terminal device X is poor (such as low network bandwidth), the task scheduling system can push the optimized lightweight version for the target applications A-E, and push in descending order of the recommendation priority.

[0092] In an embodiment, the method can further include: monitoring the task state of each to-be-processed task in the whole life cycle; the whole life cycle includes a task creation stage, a task scheduling stage, a task execution stage, and a result feedback stage; when the task state of any to-be-processed task in any stage is monitored to be abnormal, fault diagnosis and fault repair are performed on the to-be-processed task.

[0093] The technical module of task whole life cycle state monitoring and intelligent feedback aims to improve the efficiency and reliability of task management, greatly shorten the task exception processing time, and improve the operation and maintenance efficiency. The technical module constructs Jaeger (a distributed tracking system) based on the OpenTelemetry (an open source observability framework) standard. OpenTelemetry provides a unified standard and tool for the observability of Jaeger, and can realize link tracking and visual monitoring of the whole process from the creation to the execution of a task. By inserting tracking code at each key link of each to-be-processed task, Jaeger can record the execution path, time consumption of each stage, and data flow of each to-be-processed task, facilitating operation and maintenance personnel to comprehensively understand the execution of each to-be-processed task.

[0094] The task scheduling system can combine Prometheus and Jaeger for task status monitoring and resource consumption recording. Prometheus is an open-source system monitoring and alerting tool that can collect real-time metrics data (such as CPU usage, memory occupancy, network bandwidth, etc.) for each pending task, providing data support for analyzing the resource consumption of each pending task. Jaeger is an open-source distributed tracing system that can be integrated with OpenTelemetry to trace and analyze the scheduling link of each pending task in detail, helping to locate performance bottlenecks and abnormal points in the task execution process. When any pending task encounters an exception in any stage (such as task creation stage / task scheduling stage / task execution stage / result feedback stage), the task scheduling system automatically triggers a fault diagnosis and response mechanism based on a rule engine (such as Drools). Drools is a powerful rule engine that can quickly diagnose the abnormal situation of a pending task according to pre-set rules and execute corresponding fault repair according to the diagnosis result. For example, if it is monitored that any pending task has high CPU usage in the task execution stage, the task scheduling system can automatically adjust the resource allocation of the pending task; if it is monitored that any pending task encounters a deadlock or other serious problems in the task execution stage, the task scheduling system can restart the pending task or perform manual intervention.

[0095] In specific implementation, in the task creation stage, the task scheduling system adds OpenTelemetry tracing context information for each pending task, ensuring that the execution process of each pending task in the whole life cycle of task creation stage→task scheduling stage→task execution stage→result feedback stage can be traced. Prometheus periodically collects various metrics data of each pending task and stores it in a time series database. Jaeger receives the tracing data sent by OpenTelemetry and generates a scheduling link diagram for each pending task. When Prometheus monitors that the metrics data of any pending task in any stage has abnormal fluctuations, it triggers the rule engine (Drools) to perform fault diagnosis. The rule engine analyzes and judges the abnormal situation of the pending task according to the pre-set rules and executes the corresponding fault repair operation, and at the same time feeds back the repair result and abnormal information to the operation and maintenance personnel.

[0096] In addition, the embodiments of the present application also provide a technical module for multi-modal task triggering and cooperative execution, which aims to realize the automatic triggering and orderly execution of the to-be-processed tasks, thereby improving the system response speed and user experience. The technical module designs a task triggering model based on the event storm method, which is a technique for identifying and analyzing business events and can help the task scheduling system to comprehensively sort out various possible event sources and task triggering conditions. By modeling and analyzing the business process, the association between different event sources and to-be-processed tasks is determined, providing a basis for the automatic triggering of the to-be-processed tasks.

[0097] Specifically, the task scheduling system realizes task state management and cooperative logic control in combination with FSM (Finite State Machine). FSM is a mathematical model that can describe the conversion rules and conditions between different states of to-be-processed tasks. In the embodiments of the present application, each to-be-processed task is abstracted as an FSM, and the execution process of each to-be-processed task can be represented as a state conversion process. By defining state conversion rules and event triggering conditions, the task scheduling system can accurately control the execution order and state changes of each to-be-processed task, ensuring the cooperative execution between these to-be-processed tasks.

[0098] Among them, the task scheduling system supports multiple event sources (such as timing events, sensor events, database change events, etc.), and accesses and processes them uniformly through Spring Cloud Stream. Spring Cloud Stream is a framework for building message-driven microservices, which provides unified message channels and binding mechanisms, and can easily integrate various event sources and message middleware. By using Spring Cloud Stream, the task scheduling system can uniformly access different types of events, and automatically trigger the corresponding to-be-processed tasks according to the type and triggering condition of the event.

[0099] In specific implementation, the task scheduling system analyzes the business process using the event storm method, determines all possible event sources and task triggering conditions, and maps them to the state conversion rules of FSM. In the running process of the task scheduling system, Spring Cloud Stream listens to the changes of various event sources in real time, and when it detects an event that meets the triggering condition, it automatically triggers the corresponding to-be-processed task. During the execution of the to-be-processed task, state updating and cooperative execution are performed according to the state conversion rules of FSM. For example, if a to-be-processed task is executed, the next related to-be-processed task execution process will be triggered according to the state conversion rules, thereby realizing the automatic triggering and orderly execution of the to-be-processed tasks.

[0100] To intuitively understand the technical advantages of the embodiments of the present application in task scheduling, the embodiments of the present application provide test results of each technical module, as follows: 1. Test of the technical module for task intelligent arrangement based on deep reinforcement learning: In a simulated Internet of Things scenario, facing 1000 tasks to be processed and 50 edge nodes, the response time of a first target task (such as a high-priority task) is shortened by 62%, for example, to 1.2 seconds, the resource utilization rate of a second target task (such as a low-priority task) is increased to 88%, and when the resource utilization rate of the edge node is overloaded, the completion rate of the task to be processed still reaches 98.7%. 2. Test of the technical module for dynamic sensing and adaptive allocation of edge node resources: Under the pressure test of the edge computing cluster, the task scheduling system can reduce the load difference between different edge nodes by 68% within 5 minutes, and the standard deviation of the resource utilization rate is reduced from 0.35 to 0.12. The task scheduling system can automatically adjust the task scheduling strategy more than 1200 times within 24 hours, so that the task execution is zero failure. 3. Test of the technical module for edge and cloud collaborative computing: In the smart city scenario, the edge side completes data preprocessing within 100 milliseconds, and the compression rate is 70%. The cloud side processes complex tasks for 3.5 seconds, and the overall task processing time is shortened by 58%. When the network fluctuates, the data transmission success rate can still reach more than 99.5%.

[0101] 4. Test of the technical module for intelligent recommendation and accurate distribution of application market: In the application market, the personalized recommendation function increases the application click rate by 42% and the user time length by 35 minutes. Intelligent distribution improves the application download speed by 60%, and the download success rate of users in a poor network state (such as low network bandwidth) can still reach 95%. 5. Test of the technical module for task full life cycle state monitoring and intelligent feedback: In the distributed system test, the abnormal task positioning time is shortened by 80%, for example, to 3 seconds, the fault response can be completed within 10 seconds, the task exception processing time is shortened from 15 minutes to 1.5 minutes, and the operation and maintenance efficiency is improved by 10 times.

[0102] 6. Test of the technical module for efficient preprocessing and compressed transmission of edge data: 10MB of edge processing data per second can be shortened to 3MB after preprocessing and further shortened to 0.8MB after compression, with a compression ratio of 12.5:1. Under the network bandwidth of 1Mbps, the transmission time from the edge to the cloud is shortened from 80 seconds to 6.4 seconds, and the network bandwidth occupancy rate can be reduced to 92%. 7. Test of the technical module for multi-modal task triggering and collaborative execution: in the test environment, the task is identified and executed within 200 milliseconds under multi-event triggering, more than 100,000 events are processed per week, the task execution accuracy is 99.8%, and there is no conflict in collaboration.

[0103] From the above test results, it can be seen that in the context of application market integration, the embodiments of the application break through the service limitations of the application market in related technologies and build an efficient and intelligent new service system. Through deep integration of edge computing and intelligent scheduling technology, the embodiments of the application can effectively cope with the increasing complex demands of the application market, realize the optimal allocation of application resources and significantly improve the quality of service. With the characteristics of edge nodes close to the terminal devices of users, the embodiments of the application sink key services such as application recommendation and application distribution to the edge side, so as to realize real-time perception of the device type and network state of the terminal device through the edge node, obtain the semantic feature information of each application and the preference feature information of each user through the BERT model and LightGBM, and dynamically adjust the application scheduling strategy with millisecond-level response speed in combination with the resource attribute information of the edge node. In the application recommendation link, the embodiments of the application use a reinforcement learning algorithm to continuously optimize the recommendation model, accurately match user demand and application function, and improve the accuracy of recommendation and user satisfaction. At the same time, an intelligent management and control mechanism is established, Prometheus, Grafana and other tools are used to monitor the edge node and application service in the whole link, and when problems such as service exceptions and resource bottlenecks occur, the rule engine and automatic operation and maintenance strategy are combined to automatically trigger fault diagnosis and fault repair processes, thereby guaranteeing the stability and reliability of the application market service.

[0104] The embodiments of the application aim to solve the problems of lack of adaptive ability in task scheduling and imperfect task life cycle management in related technologies, and provide more efficient and intelligent task scheduling strategies by integrating edge computing, artificial intelligence and cloud computing technologies.

[0105] As can be seen from the above examples, the scheme provided by the application has the following technical advantages: 1. Task intelligent arrangement based on deep reinforcement learning: a deep Q network (DQN) and its improved algorithm are used as a task scheduling agent, a convolutional neural network (CNN) and a recurrent neural network (RNN) are used to extract features of task attribute information and resource attribute information, the task scheduling agent generates scheduling actions according to a policy network, and the task scheduling strategy is optimized through a reward mechanism to realize optimal matching of tasks and resources.

[0106] 2、Edge node resource dynamic perception and adaptive allocation: build a real-time resource monitoring system with Prometheus and Grafana to collect hardware resource information and network resource information of edge nodes; introduce a game theory model for resource allocation decision-making, combine the Kubernetes scheduling concept, and combine linear programming and edge prediction model to predict resource change trend to dynamically adjust task scheduling strategy.

[0107] 3、Edge and cloud collaborative computing: design a collaborative service governance mechanism based on Service Mesh architecture, use Istio to realize service routing and traffic control, and use Apache Kafka to realize asynchronous communication and data caching; deploy a lightweight Flink cluster on the edge side for data preprocessing, and build a Spark platform on the cloud to complete complex analysis tasks.

[0108] 4、Intelligent application market recommendation and accurate distribution: use the BERT model with Transformer structure for application semantic understanding, combine LightGBM to build a user-application matching model; based on Docker image packaging and CDN network, dynamically select the optimal version for application push according to the device type, network status and other information of the user's terminal device.

[0109] 5、Task full life cycle state monitoring and intelligent feedback: build Jaeger based on OpenTelemetry standard, combine Prometheus and Jaeger to record task state and resource consumption; when the task state is abnormal, automatically trigger fault diagnosis and response mechanism based on rule engine (such as Drools).

[0110] 6、Efficient preprocessing and compressed transmission of edge data: build a data preprocessing pipeline based on Apache Beam to filter, aggregate, normalize and preprocess edge processing data; combine high-efficiency compression algorithms such as Snappy and Zstandard to realize lossless data compression transmission.

[0111] 7、Multi-modal task triggering and collaborative execution: design a task triggering model based on event storm method, combine finite state machine (FSM) to realize task state management and collaborative logic control; use Spring Cloud Stream to uniformly access and process multiple event sources.

[0112] As can be seen, the scheme provided by the application covers the following beneficial effects: 1. Full closed-loop intelligent decision-making system: A dynamic closed loop of "perception-decision-execution-feedback" is constructed, and through deep reinforcement learning (DQN+CNN / RNN), game theory model, LSTM prediction and other technologies, the limitations of static scheduling algorithms in related technologies are broken through, and the paradigm upgrade from "manual experience driven" to "data intelligent driven" is realized. The decision accuracy of the embodiment of the present application can reach 98.7%, which is more than 25% higher than that in the related art, and the matching efficiency of task-resource is improved by 300% (for example, the response speed of high-priority tasks in industrial Internet of Things is 1.2 seconds, which is 62% shorter than that in the related art), forming a significant technical barrier.

[0113] 2. Resource management paradigm innovation: Edge node resource management realizes the upgrade from "passive response" to "active prediction": through Prometheus+Grafana real-time monitoring and game theory model, the standard deviation of load difference between different edge nodes is reduced from 0.35 to 0.12, and the resource utilization rate is increased to 88%; combined with LSTM prediction algorithm, manual operation intervention is reduced by 70%, and the management cost of large-scale deployment is significantly reduced.

[0114] 3. Cloud-edge collaborative computing efficiency leap: Based on the Service Mesh architecture, the data processing delay on the edge side is <100ms, and the processing time of complex tasks on the cloud side is <5s, and the overall task link efficiency is improved by 58% compared with the related art, for example, in the Internet of Vehicles scene, the end-to-end processing delay of real-time road condition data can be controlled within 500ms, meeting the ultra-low latency demand of automatic driving and the like.

[0115] As can be seen, the scheme provided by the present application covers the following core benefits of large-scale application: 1. Industrial Internet of Things scene: cost reduction and efficiency improvement and capacity improvement, through intelligent resource scheduling, the hardware procurement cost of edge nodes is greatly reduced, realizing the intensification of hardware investment.

[0116] 2. Smart city scene: emergency response and service efficiency optimization, the task scheduling system can support multi-service integration (such as smart traffic, environmental monitoring), can effectively reduce the unit computing power cost and improve the task processing energy efficiency ratio, and therefore can adapt to large-scale city-level edge deployment.

[0117] 3. Application market commercial value: user growth and revenue improvement, based on the intelligent distribution technology of Docker image, the download success rate in the area with low network bandwidth can still be improved, the coverage user scale is expanded, and the market penetration rate and user stickiness are significantly improved.

[0118] 4. Efficient task processing: intelligent orchestration technology based on deep reinforcement learning can accurately match tasks and resources, so that both the first target task (such as high-priority tasks) and the second target task (such as low-priority tasks) can be efficiently processed, significantly improving task execution efficiency, ensuring that the first target task (such as high-priority tasks) can be quickly responded to, and the second target task (such as low-priority tasks) can reasonably utilize resources, eliminating the inefficiency and lag of task scheduling in related technologies.

[0119] 5. Intelligent resource management: through real-time sensing of edge node resource status, combined with game theory and intelligent algorithms for dynamic allocation, effectively avoiding resource waste and uneven distribution, achieving balanced utilization and efficient scheduling of resources, ensuring stable operation of the system while improving overall resource utilization.

[0120] 6. Collaborative computing upgrade: edge and cloud collaborative computing mechanism fully utilizes the advantages of both, edge side quickly completes data preprocessing, cloud side conducts deep analysis, forming an efficient collaborative system that meets business low-latency requirements and realizes deep processing of complex tasks, and comprehensively improves computing efficiency.

[0121] 7. Precise user service: intelligent recommendation and precise distribution technology deeply mines user preferences and application characteristics, realizes personalized application recommendation, and simultaneously flexibly pushes appropriate versions according to the device type and network status of the user's terminal device, thereby improving the efficiency of users obtaining high-quality applications and enhancing user experience and stickiness.

[0122] 8. Reliable task monitoring: whole life cycle state monitoring and intelligent feedback technology tracks and monitors tasks in real time, and once an exception occurs, it can quickly locate and trigger intelligent diagnosis and repair, significantly shortening fault handling time, reducing manual operation and maintenance cost, and ensuring stable task execution.

[0123] 9. Optimized data transmission: data preprocessing and compression transmission technology effectively reduces data redundancy, improves data transmission efficiency, reduces network transmission pressure, and ensures that edge processing data can be efficiently transmitted to the cloud server, providing high-quality data basis for subsequent analysis and processing of the cloud server.

[0124] 10. Sequential task execution: multi-modal task triggering and collaborative execution technology, based on multiple event sources to realize automatic triggering of tasks, and through scientific state management and collaborative logic, ensures sequential execution of tasks, avoids task conflicts, and improves the coherence and efficiency of the overall system operation.

[0125] Corresponding to the foregoing application function implementation method embodiments, the present application also provides a task scheduling system, an electronic device and corresponding embodiments.

[0126] Figure 3is a structural schematic diagram of a task scheduling system shown by an embodiment of the present application.

[0127] Referring to Figure 3 The task scheduling system of the present application can include: The attribute information acquisition module 310 is configured to acquire task attribute information corresponding to the plurality of to-be-processed tasks respectively, and acquire resource attribute information corresponding to the plurality of edge nodes respectively; The task allocation module 320 is configured to allocate the plurality of to-be-processed tasks to the corresponding edge nodes for processing based on the task attribute information and the resource attribute information. The resource scheduling module 330 is configured to, in the process of task processing, if it is monitored that the resource utilization rate of any edge node is overloaded, call other edge nodes or the central node for collaborative processing.

[0128] In an embodiment, the task attribute information includes at least one of priority, deadline, and resource requirement, and the resource attribute information includes at least one of CPU usage rate, memory occupancy rate, storage capacity, network bandwidth, network delay, and network throughput; the task allocation module 320 can include: A first target task determination submodule is configured to determine a to-be-processed task belonging to a high priority or a deadline less than or equal to a preset time threshold or a resource requirement greater than or equal to a preset resource threshold as a first target task; and A second target task determination submodule is configured to determine a to-be-processed task belonging to a low priority or a deadline greater than a preset time threshold or a resource requirement less than a preset resource threshold as a second target task; and A target edge node determination submodule is configured to determine, from the plurality of edge nodes, a target edge node with a CPU usage rate less than or equal to a first preset threshold, and / or a memory occupancy rate less than or equal to a second preset threshold, and / or a storage capacity greater than or equal to a third preset threshold, and / or a network bandwidth greater than or equal to a fourth preset threshold, and / or a network delay less than or equal to a fifth preset threshold, and / or a network throughput greater than or equal to a sixth preset threshold; A first target task allocation submodule is configured to allocate the first target task to the target edge node for processing; and A second target task allocation submodule is configured to predict a resource idle period of each edge node, so as to allocate the second target task to the corresponding edge node for processing when the resource idle period is reached.

[0129] In an embodiment, the resource scheduling module 330 can include: The complexity determination submodule is configured to determine the complexity of the task handled by the edge node if the resource utilization of any edge node is greater than or equal to the preset load threshold during the task processing. The edge-edge cooperative processing submodule is configured to call other edge nodes for cooperative processing if the complexity indicates that the task is a simple task. The cloud-edge cooperative processing submodule is configured to call the central node for cooperative processing if the complexity indicates that the task is a complex task.

[0130] In an embodiment, the cloud-edge cooperative processing submodule can include: The edge processing data acquisition unit is configured to acquire edge processing data generated by the edge node for the complex task. The compression unit is configured to compress the edge processing data to obtain lightweight processing data. The transmission unit is configured to transmit the lightweight processing data to the central node through a message queue, so that the central node mines potential rules and trends of the complex task based on the lightweight processing data. The cloud-end processing data receiving unit is configured to receive cloud-end processing data returned by the central node for the lightweight processing data through the message queue. The cloud-end processing data includes operation instructions generated by the central node for the complex task. The operation instruction execution unit is configured to send the operation instructions to the edge node for execution.

[0131] In an embodiment, when the task to be processed is an application recommendation task, the system can further include: The semantic analysis module is configured to perform semantic analysis on each application in the application market to obtain semantic feature information of each application. The preference analysis module is configured to analyze the preference feature information of the user according to historical behavior data of the user in the application market when it is monitored that the user accesses the application market. The historical behavior data includes at least one of historical download records, historical use time, and historical ratings. The matching module is configured to match the semantic feature information of each application with the preference feature information of the user to obtain a target application list. The target application list includes a plurality of target applications to be recommended, and each target application has a corresponding recommendation priority. The recommendation module is configured to sort and display the plurality of target applications to the user according to the recommendation priority.

[0132] In an embodiment, each target application has a plurality of versions. The recommendation module can include: The target version determination submodule is used to determine the target version suitable for the terminal device from multiple versions based on the device type and / or network status of the user's terminal device through the target edge node for each target application; wherein, the target edge node is the edge node used to process application recommendation tasks; The recommendation submodule is used to sort and display multiple target applications to users according to recommendation priority and target version.

[0133] In one embodiment, the system may further include: The task status monitoring module is used to monitor the status of each task throughout its entire lifecycle, which includes the task creation phase, task scheduling phase, task execution phase, and result feedback phase. The task fault handling module is used to diagnose and repair faults in any task to be processed when the task status of any task at any stage is detected to be abnormal.

[0134] As can be seen from this example, the solution provided in this application obtains task attribute information corresponding to multiple tasks to be processed, and resource attribute information corresponding to multiple edge nodes. Based on the task attribute information and resource attribute information, the multiple tasks to be processed are allocated to the corresponding edge nodes for processing. During task processing, if the resource utilization of any edge node is detected to be overloaded, other edge nodes or central nodes are invoked for collaborative processing. This application fully utilizes the resources of each edge node, distributing multiple tasks to multiple edge nodes with sufficient resources for processing, thereby avoiding centralized processing in the cloud, thus improving task execution efficiency and resource utilization. Furthermore, by monitoring the resource utilization of each edge node in real time during task execution, when the resource utilization of any edge node is detected to be overloaded, other edge nodes or central nodes can be invoked in a timely manner for collaborative processing to avoid task backlog, thereby improving task scheduling efficiency and resource utilization.

[0135] Regarding the system in the above embodiments, the specific ways in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated further here.

[0136] Figure 4 This is a schematic diagram of the structure of an electronic device shown in an embodiment of this application.

[0137] See Figure 4 The electronic device 400 includes a memory 410 and a processor 420.

[0138] The processor 420 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gates or transistor logic, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can be any conventional processor. The memory 410 can include various types of storage units, such as a system memory, a read-only memory (ROM), and a permanent storage device. Among them, the ROM can store static data or instructions required by the processor 420 or other modules of the computer. The permanent storage device can be a read-and-write storage device. The permanent storage device can be a non-volatile storage device that does not lose stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a mass storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device can be a removable storage device (such as a floppy disk, an optical drive). The system memory can be a read-and-write memory device or a volatile read-and-write memory device, such as a dynamic random access memory. The system memory can store some or all of the instructions and data required by the processor during runtime. In addition, the memory 410 can include a combination of any computer readable storage media, including various types of semiconductor storage chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), magnetic disks and / or optical disks. In some embodiments, the memory 410 can include a read-and / or write removable storage device, such as a compact disc (CD), a read-only digital versatile disc (such as DVD-ROM, double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a min SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc. The computer readable storage medium does not include a carrier wave and a transient electronic signal transmitted through a wireless or wired transmission.

[0139] The memory 410 stores executable code, which, when processed by the processor 420, can cause the processor 420 to perform part or all of the above-mentioned methods.

[0140] In addition, the method according to the present application can also be implemented as a computer program or a computer program product, which includes computer program code instructions for performing part or all of the steps of the above-mentioned methods of the present application.

[0141] Alternatively, the present application can also be implemented as a computer readable storage medium (or non-transitory machine readable storage medium or machine readable storage medium) having stored thereon executable codes (or computer program or computer instruction codes) which, when executed by a processor of an electronic device (or a server, etc.), cause the processor to perform some or all of the steps of the above-described methods according to the present application.

[0142] The present application also provides a computer program product comprising computer instructions which, when executed by a processor, implement the above-described methods.

[0143] The above has described embodiments of the present application, the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes are obvious to those skilled in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical application or improvement to the technology in the market, or to enable other ordinary skilled in the art to understand the embodiments disclosed herein.

Claims

1. A task scheduling method, characterized by, The application comprises the following steps: Respectively acquiring task attribute information corresponding to a plurality of to-be-processed tasks, and respectively acquiring resource attribute information corresponding to a plurality of edge nodes; Based on the task attribute information and the resource attribute information, the plurality of to-be-processed tasks are respectively distributed to the corresponding edge nodes for processing; In the task processing process, if the resource utilization of any edge node is overloaded, other edge nodes or a central node are called for cooperative processing.

2. The method of claim 1, wherein, The task attribute information includes at least one of priority, deadline, and resource requirement, and the resource attribute information includes at least one of CPU usage, memory occupancy, storage capacity, network bandwidth, network delay, and network throughput; based on the task attribute information and the resource attribute information, the plurality of to-be-processed tasks are respectively distributed to the corresponding edge nodes for processing, which comprises the following steps: Determine the to-be-processed tasks belonging to high priority or having a deadline less than or equal to a preset time threshold or a resource requirement greater than or equal to a preset resource threshold as first target tasks; and Determine the to-be-processed tasks belonging to low priority or having a deadline greater than a preset time threshold or a resource requirement less than a preset resource threshold as second target tasks; From the plurality of edge nodes, determine a target edge node having a CPU usage less than or equal to a first preset threshold, and / or a memory occupancy less than or equal to a second preset threshold, and / or a storage capacity greater than or equal to a third preset threshold, and / or a network bandwidth greater than or equal to a fourth preset threshold, and / or a network delay less than or equal to a fifth preset threshold, and / or a network throughput greater than or equal to a sixth preset threshold; Distribute the first target tasks to the target edge node for processing; and Predict a resource idle period of each edge node, so as to distribute the second target tasks to the corresponding edge node for processing when the resource idle period is reached.

3. The method of claim 1, wherein, In the task processing process, if the resource utilization of any edge node is overloaded, other edge nodes or a central node are called for cooperative processing, which comprises the following steps: In the task processing process, if the resource utilization of any edge node is greater than or equal to a preset load threshold, the complexity of the task processed by the edge node is determined; If the complexity represents that the task is a simple task, other edge nodes are called for cooperative processing; or If the complexity represents that the task is a complex task, a central node is called for cooperative processing.

4. The method of claim 3, wherein, The calling of the central node for cooperative processing comprises the following steps: Acquiring edge processing data generated by the edge node for the complex task; Compressing the edge processing data to obtain lightweight processing data; Transmitting the lightweight processing data to the central node through a message queue, so that the central node mines potential rules and change trends of the complex task based on the lightweight processing data; receive, through the message queue, cloud processing data returned by the center node for the lightweight processing data; the cloud processing data comprises operation instructions generated by the center node for the complex task; send the operation instructions to the edge nodes for execution.

5. The method of claim 1, wherein, When the to-be-processed task is an application recommendation task, the method further comprises: performing semantic analysis on each application in an application market to obtain semantic feature information of each application; when monitoring that a user accesses the application market, analyzing preference feature information of the user according to historical behavior data of the user in the application market; the historical behavior data comprises at least one of historical download records, historical use time length, and historical scores; matching the semantic feature information of each application with the preference feature information of the user to obtain a target application list; the target application list comprises a plurality of target applications to be recommended, and each target application has a corresponding recommendation priority; ranking and displaying the plurality of target applications to the user according to the recommendation priority.

6. The method of claim 5, wherein, Each target application has a plurality of corresponding versions; ranking and displaying the plurality of target applications to the user according to the recommendation priority comprises: for each target application, determining a target version adapted to a terminal device of the user from the plurality of corresponding versions based on a device type and / or network state of the terminal device by a target edge node; wherein the target edge node is an edge node used for processing the application recommendation task; ranking and displaying the plurality of target applications to the user according to the recommendation priority and the target version.

7. The method of claim 1, wherein, The method further comprises: performing task state monitoring of each to-be-processed task in a whole life cycle; the whole life cycle comprises a task creation stage, a task scheduling stage, a task execution stage, and a result feedback stage; when monitoring that the task state of any to-be-processed task in any stage is abnormal, performing fault diagnosis and fault repair on the to-be-processed task.

8. A task scheduling system characterized by, comprise: an attribute information acquisition module, configured to respectively acquire task attribute information corresponding to a plurality of to-be-processed tasks, and respectively acquire resource attribute information corresponding to a plurality of edge nodes; a task allocation module, configured to allocate the plurality of to-be-processed tasks to corresponding edge nodes for processing based on the task attribute information and the resource attribute information; a resource scheduling module, configured to, in a task processing process, if monitoring that the resource utilization rate of any edge node is overloaded, call other edge nodes or a center node for collaborative processing.

9. An electronic device, comprising: comprise: a processor; and a memory having executable code stored thereon, when the executable code is executed by the processor, causing the processor to execute the method of any one of claims 1-7.

10. A computer-readable storage medium having executable code stored thereon, when the executable code is executed by a processor of an electronic device, causing the processor to execute the method of any one of claims 1-7.

Images