Multi-modal task allocation and evaluation method combining genetic algorithm and agent optimization

Abstract

The invention relates to the technical field of multi-modal task processing, in particular to a multi-modal task allocation and evaluation method combining a genetic algorithm and agent optimization. According to the method, behavior events on a multi-modal platform serve as modal subtasks to be packaged into a standardized triple, and the standardized triple is mapped into a task graph structure; then defining a gene coding rule of a task scheduling scheme, iterating a population by adopting a genetic algorithm, and obtaining an optimized scheduling scheme of each task; and the agents arranged on the execution nodes carry out task refined distribution. The invention provides a multi-modal task allocation method combining a genetic algorithm and intelligent agent optimization. The system can accurately identify an implicit logic relationship between tasks by constructing a task graph structure and introducing a time / semantic / modal three-dimensional dependency modeling rule; a global scheduling and modal agent execution mechanism driven by a genetic algorithm is adopted, and cooperation of generation of an upper-layer scheduling scheme and refined execution of local tasks of a lower-layer node is achieved. The method shows excellent resource load balancing capability in a heterogeneous computing environment, and has good expansibility and convergence performance.

Description

Technical Field

[0001] The present invention relates to the technical field of multimodal task processing, and in particular to a multimodal task allocation and evaluation method combining genetic algorithm and intelligent agent optimization. Background Art

[0002] With the rapid development of artificial intelligence and multimodal fusion technologies, modern intelligent systems in key areas such as smart healthcare, intelligent transportation, industrial quality inspection, e-commerce recommendations, and financial risk control increasingly rely on the simultaneous processing of heterogeneous data from multiple sources, including text, images, audio, and video. These tasks are often characterized by high concurrency, diversity, and strong resource sensitivity, posing significant challenges to the system's scheduling mechanisms, resource management, and real-time responsiveness.

[0003] Current mainstream resource scheduling frameworks are mostly based on centralized optimization or single-layer scheduling strategies, which have the following technical limitations:

[0004] 1) Inadequate handling of task heterogeneity. Multimodal tasks have significantly different requirements for computing resources. For example, video analysis is highly dependent on GPUs and bandwidth, while text processing focuses more on low latency and CPU throughput. However, existing systems struggle to uniformly model and rationally allocate tasks.

[0005] 2) Flat scheduling structure and lack of collaborative scheduling mechanism: In a multi-node, multi-resource environment, the traditional single-layer scheduling architecture struggles to effectively coordinate local task execution with global resource optimization. This is particularly true for cross-modal task collaboration scenarios (e.g., image and text recommendations, audio and video alignment).

[0006] 3) Lack of ability to model dependencies between tasks: Many practical tasks have implicit dependencies such as time order, semantic relevance, and modality pairing, such as medical image and report analysis, and video and voice synchronization. However, traditional scheduling systems cannot explicitly express these dependencies, resulting in chaotic scheduling order and increased transmission redundancy. Summary of the Invention

[0007] In order to overcome the defects of multimodal task scheduling in the above-mentioned prior art, the present invention proposes a multimodal task allocation method that combines genetic algorithm and agent optimization. By converting user behavior into a multimodal task graph, combining genetic algorithm with multi-agent collaborative mechanism, optimal resource scheduling of modal tasks such as text, image and video is achieved, and dynamic optimization of scheduling strategies is achieved based on user feedback. The present invention can uniformly model and express multimodal tasks, adapt to heterogeneous resources, and realize efficient multimodal task allocation and collaborative resource utilization.

[0008] The present invention proposes a multimodal task allocation method combining genetic algorithm and agent optimization, comprising:

[0009] Behavior events on the multi-modal platform are encapsulated as standardized triplets {task type, required resource, data size} as modal sub-tasks and mapped into a task graph structure G task =(V, E), where V is a set of nodes representing individual modal sub-tasks, and E is a set of directed edges representing the dependency relationship between tasks;

[0010] A gene encoding rule for defining a task scheduling scheme is defined, chromosomes corresponding to all nodes on the task graph structure are obtained to form an individual, and a genetic algorithm is used to iterate the population to obtain an optimized scheduling scheme for each task; the scheduling scheme includes: execution node, resource type, and scheduling priority;

[0011] According to the optimized scheduling scheme, the tasks are assigned to the corresponding execution nodes, and the agents arranged on the execution nodes perform task refinement and allocation.

[0012] Preferably, the dependency relationship between tasks includes: time sequence dependency, semantic dependency, and modal pairing dependency;

[0013] The determination condition of the time sequence dependency is that the time interval of the behavior event is less than a set time difference threshold;

[0014] The determination condition of the semantic dependency is that the similarity of the semantic embedding vectors extracted using the same pre-trained language model is greater than a set semantic similarity threshold;

[0015] The determination condition of the modal pairing dependency is that the modal types of the two tasks form a set modal combination, and the time stamp associated with the modal combination satisfies the time stamp proximity condition; the time stamp proximity condition is: synchronous start or synchronous end;

[0016] The directed edges all point from an early task to a late task.

[0017] Preferably, an agent corresponding to each task modality is arranged on each execution node, and the agents of the same modality on adjacent execution nodes are used to determine the task similarity; when similar tasks are found and the migration condition is met, the tasks are migrated between adjacent execution nodes according to a set migration rule;

[0018] The migration condition is that the migration probability of the two adjacent execution nodes on the modal task is greater than a modal migration threshold τ, and at least one of the execution nodes has surplus computing resources;

[0019] The migration rule is: if both execution nodes have surplus computing resources, the task located on the high-load execution node among the two tasks is migrated to the low-load execution node; if only one execution node has surplus computing resources, the task located on the execution node without surplus computing resources among the two tasks is migrated to the execution node with surplus computing resources.

[0020] Preferably, the calculation formula of the migration probability is:

[0021]

[0022] wherein, η represents a set migration sensitivity factor; S represents a similarity matrix of the same modal task on two adjacent execution nodes; τ is a modal migration threshold.

[0023] Preferably, the modal migration threshold τ is a set value, or is calculated in combination with the quantile of the modal similarity data recorded in the historical task migration.

[0024] Preferably, the agent dynamically determines the task execution priority through a scoring function, and the scoring function is:

[0025] Score match (Agent, Task) = λ · Sim(C agent , R task ) - (1 - λ) · Load cur

[0026] wherein, Score match (Agent, Task) represents the scoring function of the agent, Sim(C agent , R task ) represents the similarity score of the capability vector C agent of the agent and the demand vector R task , Load cur represents the current resource load level of the execution node where the agent is located; λ represents a trade-off factor between overall matching and load balancing.

[0027] The capability vector C agent includes the modal type M that the agent can handle, the historical average response time RT avg , the historical throughput TP history and Load cur of the task;

[0028] The demand vector R task includes the modal type M req required by the task, the type and scale of the computing resources Res req required by the task and the execution priority Pri urg of the task;

[0029] Sim(C agent , R task ) is calculated based on the matching degree of M and M req , the matching degree of RT avg and Pri urg , and the adaptability score of TP history and Res req .

[0030] Preferably, RT avg and Pri urg The matching degree function is as follows:

[0031]

[0032] Wherein, is the priority Pri urg defined license response time, and λ is a factor for adjusting the matching sensitivity, and the value range is [1, 5].

[0033] Preferably, TP history and Res req The adaptability score function is as follows:

[0034]

[0035] Wherein, β is a hyperparameter for adjusting the steepness of the curve, and the value range is [2, 10].

[0036] Preferably, the agent scores the execution priority of the task based on the weighted sum of the click rate, the page dwell time and the video completion rate and the difference between the weighted calculation of the system load, and then dynamically adjusts the execution priority of the task according to the score.

[0037] Preferably, the method for obtaining the optimized scheduling scheme of each task by iterating the population by using the genetic algorithm comprises the following steps:

[0038] S31, define the gene coding rule of the scheduling scheme of the modal subtask, initialize the population, and the single individual in the population contains the scheduling scheme corresponding to all nodes on the task graph structure;

[0039] S32, iterate and mutate the population;

[0040] S33, calculate the fitness F of the individual in the current population, and take the individual corresponding to the maximum value of the fitness F as the target individual of the current population; when the population reaches the iteration termination condition, analyze the optimized scheduling scheme of each node from the target individual;

[0041] The fitness function is: F = μ·U res -(1-μ)·T delay ;

[0042] Wherein, U res is the system resource utilization rate; T delay is the average response delay of task execution; and μ is a weight factor of scheduling efficiency and response performance.

[0043] The advantages of the present application are:

[0044] (1) The multi-modal task allocation method combining genetic algorithm and agent optimization is proposed, the implicit logical relationship between tasks can be accurately identified by constructing a task graph structure and introducing time / semantic / modal three-dimensional dependency modeling rules; the global scheduling scheme generation and local task fine execution coordination are realized by adopting the global scheduling mechanism driven by genetic algorithm (GA) and the local execution mechanism of modal agent (Agent).

[0045] (2) The execution priority score function of the task is constructed combined with the user behavior index, the real-time value judgment and scheduling response update of the task can be realized, so as to dynamically perceive the task urgency and resource state, realize the execution priority ordering and resource dynamic allocation of the task, and effectively cope with the sudden high concurrency scene.

[0046] (3) The modal similarity matrix and migration probability model are introduced, the intelligent migration, sequence rearrangement and calculation merging of tasks between adjacent nodes are realized, the communication redundancy and cache invalidation risk are significantly reduced, the cross-node communication cost is reduced, and the cache hit rate is improved.

[0047] (4) Based on the "perception-scheduling-feedback" closed-loop optimization mechanism driven by user feedback, the user behavior change can be perceived in real time, the scheduling parameters such as weight coefficient and migration threshold can be dynamically adjusted, and the scheduling model can be continuously optimized through reinforcement learning strategy, so that the resource allocation strategy has self-learning and evolution ability, and the system robustness and long-term performance are significantly improved.

[0048] (5) The local scheduling ability of the agent is optimized combined with the task capability-demand matching score model, and the task subdivision efficiency is further improved.

[0049] (6) The hierarchical agent collaborative optimization system of "global scheduling + local execution + feedback driven" is constructed, the resource scheduling automation, task execution efficiency and system optimization self-adaptation in the multi-modal task scene are realized, and an extensible and evolvable intelligent resource management solution is provided for large-scale graphic, audio and video mixed content platform.

[0050] (7) The hierarchical architecture design is adopted, the global GA scheduling + local agent execution collaborative mechanism is adopted, the global resource optimization and local execution efficiency are considered; the multi-modal dependency modeling is adopted, the task graph is constructed according to the time, semantic and modal three-dimensional rules, and the problem of insufficient dependency modeling of traditional system is solved; the dynamic closed-loop optimization is adopted, the scheduling strategy is continuously self-adapted and evolved through the combination of user feedback and reinforcement learning; the modal perception migration mechanism is adopted, the migration strategy based on similarity is adopted, the cross-node communication cost is reduced, and the cache hit rate is improved.

[0051] (8)The application has high concurrent bearing capacity, heterogeneous resource compatibility and multi-modal processing adaptability, wide application range, good industry universality, and can be widely applied to the following typical multi-modal high-density task scheduling scenes; For example: intelligent medical care of MRI image segmentation and electronic medical record joint analysis, intelligent traffic of road condition video and traffic atlas fusion judgment, financial risk control of image-text recognition and synthesis report processing and multi-modal review, image-text content retrieval, video shopping guide and voice interaction system collaborative recommendation of e-commerce platform recommendation, etc. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 A multi-modal task allocation method combining genetic algorithm and agent optimization is proposed for the application.

[0053] Figure 2 A task graph structure construction method flowchart is proposed for the application.

[0054] Figure 3 A scheduling scheme optimization method flowchart is proposed for the application. DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the application will be described in detail below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0056] A multi-modal task allocation method combining genetic algorithm and agent optimization is proposed in the embodiment.

[0057] S1, collect behavior events on the multi-modal platform, encapsulate each behavior event as a modal sub-task into a standardized triple {task type, required resource, data size}, ensure the structure of task expression is uniform, and provide a general interface for scheduling and migration mechanism.

[0058] Table 1: Triple explanation

[0059]

[0060]

[0061] The multi-modal platform is a platform with multiple behavior events, such as e-commerce platforms, medical platforms and education platforms, etc. All behavior events automatically record timestamps, user IDs, device information, context environment and other metadata, providing auxiliary information for subsequent task modeling.

[0062] The behavior events are classified according to the task modalities: text behavior, image behavior, audio behavior and video behavior.

[0063] The text behavior includes keyword input, search word submission, comment publishing and the like.

[0064] The image behavior includes browsing a main image of a commodity, uploading a medical image, screenshot comparison and the like.

[0065] The audio behavior includes voice input, video audio track analysis, medical inquiry recording and the like.

[0066] The video behavior includes watching a live broadcast segment, monitoring video analysis, video report reading and the like.

[0067] The task modalities can be obtained through lightweight rule matching and deep model inference, so as to automatically determine the modalities of the tasks. The rule matching can quickly determine the modalities according to file formats (such as.mp3,.jpg), path names, interface fields and the like; when the rules are not clear or the data is ambiguous, the modalities are determined through model inference, so as to improve flexibility and fault tolerance.

[0068] The model inference is deep calculation based on a neural network model. For example, the image behavior can be identified through a residual network such as a ResNet network, the text behavior can be identified through a natural language large model such as BERT, and the audio behavior can be identified through a DeepSpeaker embedded neural network structure.

[0069] In specific implementation, a ResNet50 classification network can be introduced to classify the image behavior based on image feature rule matching, so as to identify the task type, required resources and data size of the image behavior, thereby constructing a standardized triple of the image behavior, such as a certain image segmentation task represented as ("image", "GPU", "30MB");

[0070] A BERT fine-tuned text classifier can be used in combination with regular filtering rules to analyze the text behavior, so as to construct a standardized triple, such as a certain keyword extraction task represented as ("text", "CPU", "5KB");

[0071] A DeepSpeaker embedded model can be used in combination with file header identification to analyze the audio behavior, such as a triple of a medical inquiry voice analysis task represented as ("audio", "CPU", "3MB");

[0072] Video frame sampling in combination with image and audio track joint identification can be used to analyze the video behavior, such as a triple of a certain video recognition task represented as ("video", "GPU", "120MB").

[0073] The triple structure has good expansibility and can support more fields. In actual implementation, the triple can also be expanded according to needs, and specific examples are shown in Table 2.

[0074] Table 2: Triple expansion

[0075]

[0076] S2, mapping the triple of the modal subtask {task type, required resource, data size} to a task graph structure G task =(V, E), where V is a set of nodes representing each modal subtask, and E is a set of directed edges representing the dependency relationship between tasks;

[0077] The dependency relationship between tasks includes:

[0078] Time sequence dependency: the time interval between events is less than a preset threshold (such as 5 seconds);

[0079] Semantic dependency: two text behaviors are extracted by BERT to obtain semantic embeddings and calculate the cosine similarity. If the cosine similarity is greater than 0.8, it is determined that the two text behaviors constitute semantic dependency;

[0080] Modal pairing dependency: such as a video and audio synchronization task, which is determined according to a timestamp and a modal combination template. For example, the modal combination template is set as "the metadata of the audio behavior and the video behavior remain consistent", and the metadata includes: timestamp, user ID, device information, context environment, etc. According to the template, if the audio behavior and the video behavior meet the modal combination template, it is determined that the audio behavior and the video behavior constitute modal pairing dependency.

[0081] In the task graph structure G task , each triple corresponds to a node, and each node is attached with attributes including modal type, data volume, timestamp, scheduling priority label, etc. according to the metadata associated with the behavior event, so as to facilitate accurate determination of subsequent scheduling strategies.

[0082] In this way, the set of modal subtasks is converted into a task graph structure, providing a structured expression basis for global scheduling.

[0083] Referring to Figure 2 , specifically, the step S2 can be further divided into the following sub-steps:

[0084] S21, mapping the triple of each modal subtask to a node in the task graph structure to construct a node set V;

[0085] S22, calculating the edge connection describing the dependency relationship between nodes according to the attributes attached to the nodes to construct an edge set E. The dependency relationship includes: time sequence dependency, semantic dependency, modal pairing dependency, and other self-defined dependency templates, etc.

[0086] Various dependency relationships can be determined by a set of rule templates; for example, a time sequence dependency rule can be set as: when the difference between the log timestamps of tasks A and B is less than a set time difference threshold θ t , it is considered that tasks A and B have a time dependency; if the log timestamp of task A is earlier than B, a one-way edge A→B is added in the task graph structure; θ t can be set to 5 seconds and can be adaptively set or dynamically updated according to the system interaction frequency;

[0087] The semantic dependency rule can be set as: for text tasks A and B, a pre-trained language model such as BERT is used to extract task content embedding vectors e A and e B , and the semantic similarity between e A and e B is calculated; if the semantic similarity between the two is greater than a set semantic similarity threshold θ s , it is determined that the text tasks A and B have semantic association; if the log timestamp of task A is earlier than B, a one-way edge A→B is added in the task graph structure; the semantic similarity can be calculated using a cosine function, and θ s can be set to 0.8 and can be fine-tuned according to the task modality difference and historical semantic matching;

[0088] The modality pairing rule can be set as: if the modality types of two tasks A and B form a set modality combination and satisfy the time stamp proximity condition associated with the modality combination, it is determined that tasks A and B have modality pairing; if the log timestamp of task A is earlier than B, a one-way edge A→B is added in the task graph structure. The modality combination is self-defined, for example, an audio behavior and a video behavior form an audio-video modality combination, and the corresponding time stamp proximity condition can be set as: tasks A and B start synchronously or end synchronously.

[0089] Through the above steps S21 and S22, a simple task graph structure has been formed; to support more fine-grained scheduling, a weight and a dependency type label can also be added to each edge; the weight represents the dependency strength, which is automatically mapped to a score between 0 and 1 according to the time interval or the semantic similarity; the type label identifies the dependency source, such as time sequence dependency, semantic dependency, or modality pairing.

[0090] The task graph structure in the embodiment supports incremental construction and real-time updating mechanism, so that when a new task arrives, the dependency relationship with the existing task is determined by the above rules, and the node and edge are dynamically added to realize the update of the new task; when the dependency relationship is invalid, such as exceeding the time window, task cancellation, etc., the corresponding edge connection is cancelled, so as to guarantee the real-time and effectiveness through the automatic update of the task graph structure.

[0091] S3, a gene coding rule of the scheduling scheme of the modality subtask is defined, a chromosome gene corresponding to the scheduling scheme of each node in the task graph structure is obtained, and chromosome genes corresponding to all nodes on the task graph structure constitute an individual gene chain (referred to as an individual); that is, the scheduling scheme corresponding to a single node is coded as a chromosome gene, the chromosome genes of all nodes constitute an individual, and multiple individuals constitute a population; an optimized scheduling scheme S of each node is obtained through population optimization opt .

[0092] The gene is a scheduling scheme, and contains multi-dimensional information such as an execution node of a task, a resource type (CPU / GPU), and a scheduling priority. Tasks with high scheduling priority are preferentially scheduled.

[0093] With reference to Figure 3 , the step S3 specifically includes the following sub-steps:

[0094] S31, a gene coding rule of the scheduling scheme of the modality subtask is defined, and a population is initialized. A single individual in the population contains scheduling schemes corresponding to all nodes on the task graph structure.

[0095] The population initialization adopts a hybrid strategy of random generation and heuristic load balancing. The initial population diversity can be guaranteed through random generation, and local convergence is avoided. The heuristic strategy is adopted to distribute the initial scheme according to historical load data, which significantly improves the initial quality of the population and accelerates the convergence of the algorithm.

[0096] S32, iteration and mutation are performed on the population.

[0097] In this step, first, high-adaptability individuals are selected by using a roulette method combined with an elite reservation strategy, to ensure that high-quality solutions are reserved and the algorithm converges faster. Then, diversified gene recombination operations are performed, such as double-point crossover, resource exchange , and task node exchange operations, to enhance the diversity of genetic operations and effectively avoid falling into local optimization. Dynamic mutation operations are further introduced, such as small-probability resource switching (for example, ) and scheduling priority fine-tuning operations of tasks, and the initial mutation probability is set to 0.05. The initial mutation probability is dynamically adjusted according to the convergence of the fitness, to ensure that the system has stability and dynamic adjustment capability.

[0098] The fitness function is: F = μ·U res -(1-μ)·T delay

[0099] wherein U res is the system resource utilization rate, defined as the weighted average of the resource (CPU / GPU) utilization rate of each node; and T delayThe average response delay of the tasks corresponding to each node is uniformly set as seconds; μ is a weight factor of scheduling efficiency and response performance, and the initial value can be set as 0.5, and then dynamically adjusted according to real-time system requirements. Specifically, when the system load is high, μ is appropriately increased to preferentially improve resource utilization; when the system load is low and the response speed is critical, μ is appropriately reduced to reduce the response delay.

[0100] S33, calculate the fitness F of the individuals in the current population, and take the individual corresponding to the maximum fitness F as the target individual of the current population, and determine whether the population iteration termination condition is reached;

[0101] No, return to step S32;

[0102] Yes, output the target individual as the optimized scheduling scheme S opt of each node in the task graph structure.

[0103] The population iteration termination condition is that the preset maximum iteration number is reached, or the improvement amplitude of the maximum fitness F in a plurality of consecutive iteration periods is less than a set change rate, for example, 1%, or the resource load and response delay of each node meet the predetermined performance threshold.

[0104] S4, according to the optimized scheduling scheme S opt of each modality subtask, assign the task to the assigned execution node. The execution node is a computing unit or service container with execution capability, such as a server, a virtual machine, an embedded platform, or a processing module in a certain microservice system.

[0105] In specific implementation, after step S4, the execution nodes of the modality subtasks are further optimized through task migration, and specific reference is made to the following steps:

[0106] S5, find similar tasks with the same modality on adjacent execution nodes, calculate the migration probability, and when the migration probability is greater than the historical experience quantile threshold τ, migrate the similar tasks on the high-load execution node to the low-load execution node.

[0107] The judgment condition of the similar tasks is that the modality is the same, and the value of the modality similarity degree calculation function based on the shared feature extractor is greater than the modality migration threshold τ;

[0108] τ=Quantile(S history ,q)

[0109] Wherein, Quantile is a quantile function, S history represents the modality similarity degree data recorded in the historical task migration, and q represents the quantile of the historical data, which can be regularly updated according to the task performance and resource load to realize adaptive optimization of the threshold; in specific implementation, q can be set as the 90% quantile.

[0110] The calculation formula of the migration probability is:

[0111]

[0112] wherein η represents a migration sensitivity factor for adjusting the steepness of the task migration probability curve, and the initial value can be set to 0.5, and then dynamically optimized according to the migration performance; S represents a similarity matrix of the same modal tasks on two adjacent execution nodes.

[0113] In specific implementation, the migration sensitivity factor η can be adjusted according to the migration success rate and resource load balancing. If the task execution effect is significantly improved after migration, η is appropriately increased to increase the migration probability. If the migration effect is poor or increases the system burden, η is reduced to reduce unnecessary migration.

[0114] The adjustment of the migration sensitivity factor η realizes the closed-loop optimization of the task migration mechanism, and effectively realizes the efficient distribution of tasks between nodes and the rational use of resources.

[0115] In specific implementation, modal perception agents can be dynamically deployed on each execution node, and each modal perception agent is used to finely distribute the tasks of the corresponding modal received by the server node.

[0116] That is, the execution node deploys multi-modal perception agents, such as one or more of an image agent, a text agent, an audio agent, and a video agent, to support multi-modal processing. When the execution node receives an image task, the image agent determines whether it has local processing capability based on the matching of its capability vector and the task demand vector. If the task size is large or the resources are limited, the agent will further refine the task and distribute it to the terminal execution module inside the execution node or the subordinate edge node for processing, realizing modal perception driven task fine-grained scheduling and collaborative execution.

[0117] In the embodiment, each modal agent of each execution node continuously maintains and updates the similarity matrix S of the same modal agent of the adjacent execution node, denoted as:

[0118] S ij =f sim (F i ,F j )

[0119] S={S ij} i,j

[0120] wherein F i represents the modal feature vector of the i-th task in the agent, and F jf sim is the modality feature vector of the jth task of the same modality agent in the adjacent execution node, f sim is the modality similarity calculation function based on the shared feature extractor.

[0121] In specific implementation, different modalities can adopt different f sim functions; for example, the image modality can adopt a deep visual feature inference model such as a color histogram or ResNet feature embedding as the f sim function, and the text modality can adopt a pre-trained language model embedding such as TF-IDF vector or BERT as the f sim function.

[0122] There is a cross-modality collaboration requirement or dependency relationship between tasks (such as video task correlating audio analysis, text comment correlating image recognition), so it is necessary to process the similarity estimation, resource collaboration and task migration decision of multi-modality tasks from a global perspective. Specifically, the multi-modality scene can perform weighted fusion on the similarity of each modality.

[0123] In this way, the similar tasks in step S5 can be directly obtained from the similarity matrix S. In this way, assuming that there is a channel between the execution nodes PQ, i.e., PQ is adjacent; the task i in the image perception agent a1 in the execution node P and the task j in the image perception agent b1 in the execution node Q, S ij (T) represents the similarity function value of the task i and the task j in the similarity matrix S, P transfer；a1,b1 represents the migration probability P transfer calculated based on the similarity matrix S between the agents a1 and b1.

[0124] At this time, if S ij (T)≥τ and P transfer；a1,b1 ≥τ, it is judged that the task i and the task j can be migrated; at this time, it is further judged that the load of the execution nodes P and Q, if the P and Q computing resources are sufficient, and the P load is higher than the Q load, the task i is migrated to the agent b1; if the P computing resources are insufficient, the Q computing resources are sufficient, the task i is migrated to the agent b1; if the P and Q are both insufficient in computing resources, no task migration is performed.

[0125] In specific implementation, a scoring function is also set, and the agent dynamically determines the execution priority of the task through the following scoring function. The scoring function is:

[0126] Score match (Agent, Task)=λ·Sim(C agent ,R task )-(1-λ)·Load cur

[0127] Among them, Score match (Agent,Task) represents the scoring function of the agent, Sim(C agent ,R task ) represents the agent's capability vector C agent and the demand vector R task Similarity score of Load cur Indicates the current resource load level of the execution node where the agent is located; λ represents the trade-off factor between overall matching and load balance, which can take values ​​between 0 and 1 and can be dynamically adjusted according to the real-time load. For example, when the system load is high, reducing λ can reduce the load; when the system load is low, increasing λ can improve the task matching quality.

[0128] C agent =[M,RT avg ,TP history ,Load cur ]

[0129] R task =[M req ,Res req ,Pri urg ]

[0130] Sim(C agent ,R task )=

[0131] w1·δ(M,M req )+w2·g(RT avg ,Pri urg )+w3·h(TP history ,Res req )

[0132] Where M represents the modality type that the agent can process, such as text, image, audio, video, etc.; RT avg Represents the historical average response time of the task; TP history Indicates historical throughput, specifically the number of tasks completed per unit time; Load cur Indicates the real-time resource load level of the current node; M req Indicates the modality type required for the task; Res req Indicates the type and scale of computing resources required for the task; Pri urg Indicates the urgency or execution priority of the task. For example, real-time push tasks have the highest execution priority.

[0133] δ(M,M req ) represents the modal matching indicator function, which is 1 when the modes are consistent and 0 otherwise;

[0134] g(RT avg,Pri urg ) represents a matching degree function based on response time and task urgency;

[0135] In implementation, different priority levels can be set to correspond to response time permission thresholds. When the average response time RT avg of the intelligent agent is less than the permitted response time defined by the corresponding priority Pri urg , the matching degree function value is higher. To enhance fault tolerance and distinguishability, the matching degree function can adopt the following flexible function form:

[0136]

[0137] where λ is a factor for adjusting the matching sensitivity, and the default value range is [1, 5], which can be automatically adjusted according to the task scenario.

[0138] h(TP history ,Res req ) represents a historical throughput capacity and task resource demand adaptability scoring function; TP history represents the historical average throughput capacity of the execution node, such as the number of tasks or data volume processed per unit time; Res req represents the resource demand of the current task, such as the required CPU / GPU core number, memory / video memory capacity, etc.

[0139] To unify the dimensions, different types of resource requests are standardized into a unified "processing demand value" or "unit task resource consumption". In this embodiment, the historical throughput capacity and task resource demand adaptability scoring function h(TP history ,Res req ) is defined as follows:

[0140]

[0141] where, is used to measure whether the resource demand exceeds the historical average processing capacity; β is a hyperparameter for adjusting the steepness of the curve, and its value range is [2, 10]; if Res req ≤ TP history , the function output is close to 1, indicating that the system has sufficient capacity; if Res req > TP history , the function output tends to 0, indicating mismatch. This is a Sigmoid type of adaptability function, which can express the matching degree, has fault tolerance, and has good gradient characteristics.

[0142] w1, w2, and w3 are internal weights of the similarity function, which are periodically optimized according to historical data statistics.

[0143] ​In this embodiment, an online feedback closed-loop mechanism can also be introduced. According to the performance after task execution (such as response delay, task success rate, resource utilization, etc.), the weight parameters w1, w2, w3 and the trade-off factor λ in the scoring function are periodically adjusted to achieve the optimal balance state of long-term task allocation.

[0144] Through the above-optimized multi-agent task execution mechanism, dynamic and accurate matching of task allocation and effective balancing of resource load are realized, and the response efficiency and stability of the overall system are improved.

[0145] As can be seen from the above, the execution priority of the task is an important indicator when the agent distributes the task. In this embodiment, a task execution priority scoring function is also proposed. The agent dynamically fine-tunes the execution priority of the task through the priority scoring mechanism to ensure that high-value tasks are responded to first:

[0146] Priority=α·CTR+β·DwellTime+γ·VCR-δ·Load

[0147] Where Priority is the execution priority score of the task, CTR is the click rate, which measures the user's interest in the recommended content; Dwell Time is the page dwell time, which reflects the value of user interaction and exploration; VCR is the video completion rate, which reflects the completion quality of the user for the recommended task content; Load is the system load, which is used to monitor the current system resource occupation level in real time;

[0148] α is the user interest weight, which can be set to a high initial value, for example, the initial value is set to 0.99, and it is dynamically adjusted according to the change of CTR; for example, when it is detected that CTR is continuously decreasing, α is automatically increased;

[0149] β is the interaction value weight, which is set to a moderate initial value, for example, the initial value is set to 0.5, and it is increased in time according to the change of dwell time; for example, when the average page dwell time is shortened, β is automatically increased;

[0150] γ is the task completion quality weight, which can be set to a moderate initial value, for example, the initial value is set to 0.5, and it is increased according to the decrease of VCR; for example, when the VCR index continuously falls below the preset threshold, γ is automatically increased;

[0151] δ is the system load sensitivity weight, which is set to a low initial value, for example, the initial value is set to 0.01, and it is dynamically increased to reduce resource pressure when the system load is high; for example, when the system resource utilization rate exceeds the safety threshold (such as CPU / GPU usage rate is higher than 80%), δ is automatically increased.

[0152] In specific implementation, the weights α, β, γ and δ can be automatically adjusted at an interval setting period, and the period can be set to about half an hour.

[0153] In this embodiment, a reinforcement learning mechanism (such as Q-learning or UCB multi-arm tiger machine) can be introduced to automatically optimize the weights a, b, g and d based on real-time feedback to maximize the overall recommendation revenue in the long term; after each weight update, the system performance change is recorded in real time, the historical tuning experience is accumulated, and the recommendation effect is continuously improved.

[0154] In specific implementation, the adjusted weights a, b, g and d can also be deployed and verified on a small scale based on local nodes such as execution nodes P and Q; the effect is evaluated based on the trend of the system performance indicators, and the execution priority scoring function of the task is updated and optimized in time to form a stable weight combination strategy, ensuring that the system is continuously optimized and remains in an efficient running state.

[0155] The system performance indicators can specifically use: CPU / GPU processor utilization, bandwidth load, scheduling delay, etc.; the recommendation consistency, user behavior improvement amplitude, task success rate, etc. can also be evaluated in combination with user click rate indicators, so as to form a comprehensive evaluation of system performance and effect, and the effect of the method is intuitively displayed in combination with the evaluation results, and the implementability and effectiveness are displayed. In specific implementation, the evaluation period can be set to 5-30 minutes to realize real-time tracking of system performance, which is beneficial to judging the stability of system performance.

[0156] The evaluation results can also be used to guide the fitness function optimization process of the genetic algorithm, the weight of the agent, the modal recognition model and the scoring function optimization, support resource expansion and contraction decision and intelligent deployment strategy generation on the operation and maintenance side.

[0157] To verify the effectiveness of the multi-modal task allocation and evaluation method combining genetic algorithm and agent optimization proposed in the present application, comparative experiments are designed and carried out. The experimental environment is based on a multi-modal task scheduling platform, and the task data sources include typical multi-modal businesses such as image analysis, text processing, audio recognition and video understanding, and the total amount of simulated tasks is about 10,000, covering four types of modal behavior events. The system is deployed in a cluster platform containing 8 heterogeneous execution nodes (CPU / GPU hybrid configuration).

[0158] The baseline of the experiment is: the traditional static priority scheduling scheme, that is, scheduling based on the static priority order specified in the task initialization stage, the tasks are evenly distributed among the nodes, and dynamic migration and local load sensing optimization are not supported.

[0159] The method of the present application introduces:

[0160] Multi-modal task triple standardization and task graph structure modeling;

[0161] Global scheduling optimization strategy based on genetic algorithm;

[0162] Fine-grained task distribution and dynamic scoring mechanism of modal-aware intelligent agent

[0163] Resource migration mechanism between nodes and dynamic adjustment strategy of scoring function.

[0164] The experiment mainly evaluates the following three core indicators:

[0165] Resource utilization (CPU / GPU usage);

[0166] Task average response time (average time from task reception to completion);

[0167] Task accuracy / hit rate (in multi-modal recommendation tasks, the accuracy of hitting user intent)

[0168] The experimental results are shown in the following table:

[0169] Indicator name Conventional static priority scheduling scheme Method of the present invention Relative promotion range CPU / GPU resource utilization 65.3％ 78.4％ 20.0％ Average response latency (seconds) 2.57 1.80 -29.9％ Multimodal task accuracy 72.5％ 80.1％ 10.05％

[0170] The results show that, compared with the traditional scheduling strategy, the method has significant optimization effect in resource utilization, response speed and recommendation accuracy, and can realize more stable task distribution and execution performance in high concurrency scenarios.

[0171] Of course, for those skilled in the art, the present application is not limited to the details of the above exemplary embodiments, but also includes the same or similar structures that can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description, and therefore all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any reference signs in the claims should not be considered as limiting the claims involved.

[0172] In addition, it should be understood that although the present specification is described in terms of embodiments, not every embodiment contains only one independent technical solution, and the description of the specification is only for the sake of clarity, and those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can be combined appropriately to form other embodiments that those skilled in the art can understand.

[0173] The technologies, shapes, and structural parts not described in detail in the present application are well-known technologies.

Claims

1. A multimodal task allocation method combining genetic algorithm and agent optimization, characterized in that: The method comprises the following steps of: Behavior events on the multi-modal platform are encapsulated as standardized triples {task type, required resource, data size} for each modality sub-task and mapped into a task graph structure G task =(V, E), where V is a set of nodes representing individual modality sub-tasks, and E is a set of directed edges representing dependencies between tasks; A gene coding rule of a task scheduling scheme is defined, chromosomes corresponding to all nodes on a task graph structure are obtained, a genetic algorithm is used to iterate a population, and an optimized scheduling scheme of each task is obtained; The scheduling scheme comprises an execution node, a resource type, and a scheduling priority; According to the optimized scheduling scheme, the tasks are distributed to the corresponding execution nodes, and intelligent agents arranged on the execution nodes perform task refinement and distribution.

2. The multi-modal task allocation method combining genetic algorithm and agent optimization as claimed in claim 1, wherein, The dependency relationship between tasks comprises a time sequence dependency, a semantic dependency, and a modal pairing dependency; The judgment condition of the time sequence dependency is that the time interval of a behavior event is less than a set time difference threshold; The judgment condition of the semantic dependency is that the similarity of semantic embedding vectors extracted by using the same pre-trained language model is greater than a set semantic similarity threshold; The judgment condition of the modal pairing dependency is that the modal types of two tasks constitute a set modal combination, and the time stamp associated with the modal combination satisfies a time stamp proximity condition; the time stamp proximity condition is that the start time is synchronous or the end time is synchronous; Each directed edge points from an early task to a late task.

3. The multi-modal task allocation method combining genetic algorithm and agent optimization as claimed in claim 1, wherein, Intelligent agents corresponding to the modalities of the tasks are arranged on each execution node, and the intelligent agents of the same modality on adjacent execution nodes are used to judge the similarity of the tasks; when similar tasks are found and a migration condition is met, the tasks are migrated between the adjacent execution nodes according to a set migration rule; The migration condition is that the migration probability of the two adjacent execution nodes on the tasks of the same modality is greater than a modal migration threshold τ, and at least one of the execution nodes has a surplus of computing resources; The migration rule is that, when the computing resources of the two execution nodes are both in surplus, the tasks located on the high-load execution node among the two tasks are migrated to the low-load execution node; or when the computing resources of only one execution node are in surplus, the task located on the execution node without a surplus of computing resources among the two tasks is migrated to the execution node with a surplus of computing resources.

4. The multi-modal task allocation method combining genetic algorithm and agent optimization as claimed in claim 3, wherein, The calculation formula of the migration probability is: wherein η represents a set migration sensitivity factor; S represents a similarity matrix of the tasks of the same modality on the two adjacent execution nodes; and τ is the modal migration threshold.

5. The multi-modal task allocation method combining genetic algorithm and agent optimization as claimed in claim 4, wherein, The modal migration threshold τ is a set value, or is calculated in combination with the quantile of the modal similarity data recorded in the historical task migration.

6. The multi-modal task allocation method combining genetic algorithm and agent optimization as claimed in claim 3, wherein, The intelligent agents dynamically determine the execution priority of the tasks by using a scoring function, and the scoring function is: Score match (Agent,Task) = λ · Sim(C agent ,R task ) - (1 - λ) · Load cur where Score match (Agent, Task) denotes the score function of the agent, Sim(C agent , R task ) denotes the similarity score of the capability vector C agent and the requirement vector R task , Load cur denotes the current resource load level of the execution node where the agent is located; λ denotes the trade-off factor of the overall matching and load balancing. Capability vector C agent including modalities M that the agent can handle, task history average response time RT avg , history throughput TP history and Load cur ; a demand vector R task including a modal type M required for the task req a type and scale of computing resources Res required for the task req and a priority Pri of execution of the task urg ; Sim(C agent ,R task ) is calculated based on the matching degree of M and M req , the matching degree of RT avg and Pri urg , and the adaptability score of TP history and Res req .

7. The multi-modal task allocation method combining genetic algorithm and agent optimization as claimed in claim 6, wherein, RT avg and Pri urg The matching function for Pri is as follows: wherein, is the priority Pri urg defined permission response time, λ is a factor to adjust the matching sensitivity, and λ ∈ [1, 5].

8. The multi-modal task allocation method combining genetic algorithm and agent optimization as claimed in claim 6, wherein, TP history and Res req The fitness score function for and Res is as follows: wherein β is a hyperparameter for adjusting the steepness of the curve, and the value range of β is [2, 10].

9. The multi-modal task allocation method combining genetic algorithm and agent optimization as claimed in claim 6, wherein, The intelligent agents score the execution priority of the tasks based on the difference between the weighted sum of the click rate, the page dwell time, and the video completion rate and the system load, and then dynamically adjust the execution priority of the tasks according to the score.

10. The multi-modal task allocation method combining genetic algorithm and agent optimization of any one of claims 1-9, wherein, The method for obtaining the optimized scheduling scheme of each task by using the genetic algorithm to iterate the population comprises the following steps: S31, a gene coding rule of a scheduling scheme of a modal subtask is defined, a population is initialized, and each individual in the population comprises a scheduling scheme corresponding to all nodes on a task graph structure; S32, the population is iterated and mutated; S33, the fitness F of each individual in the current population is calculated, and the individual corresponding to the maximum fitness F is taken as a target individual of the current population; When the population reaches an iteration termination condition, the optimized scheduling scheme of each node is analyzed from the target individual. The fitness function is: F = μ · U res - (1 - μ) · T delay ; where U res is the system resource utilization; T delay is the average response latency of task execution; and μ is the weight factor of scheduling efficiency and response performance.

Images