Intelligent multi-model cooperative scheduling method

Abstract

The invention discloses an intelligent multi-model cooperative scheduling method which is executed on independent hardware scheduling equipment deployed in a local environment, the independent hardware scheduling equipment is connected with a front-end intelligent agent and a plurality of rear-end large language models through physical interfaces, and relates to the technical field of artificial intelligence system architecture. Scheduling logic is deployed in independent hardware scheduling equipment in a local environment, and the independent hardware scheduling equipment is directly connected with a front-end agent, namely a task initiating end, and a plurality of rear-end large language models, namely task executing ends, through physical interfaces to serve as a task scheduling carrier; it is ensured that user data is not leaked in the scheduling process from the physical level, enterprise-level data security compliance requirements are met, and the data security problem which cannot be ensured by a software scheme is relieved; and real-time scheduling decision and load monitoring are realized through the dynamic scheduling model, so that the efficiency and precision of multi-model cooperative processing are greatly improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence system architecture technology, and in particular to an intelligent multi-model collaborative scheduling method. Background Technology

[0002] Large language models refer to deep learning models trained on massive amounts of text data, enabling them to generate natural language text or understand the meaning of language text. These models can provide in-depth knowledge and language production on a wide range of topics by being trained on huge datasets.

[0003] With the rapid development of large language model technology, a single model can no longer meet the complex and diverse application requirements. Existing multi-model scheduling schemes mainly suffer from the following technical shortcomings: Existing scheduling solutions are mostly pure software middleware deployed on general computing platforms. In software scheduling solutions, user raw data needs to be frequently transmitted between different system components, lacking physical security isolation, posing a risk of data leakage, and failing to meet enterprise-level data security requirements. Summary of the Invention

[0004] To address the technical problems existing in the background art, this invention proposes an intelligent multi-model collaborative scheduling method.

[0005] This invention proposes an intelligent multi-model collaborative scheduling method, which is executed on an independent hardware scheduling device deployed in a local environment. The independent hardware scheduling device is connected to the front-end intelligent agent and multiple large language models in the back-end through physical interfaces, and includes the following steps: S1. Receive task requests sent by the front-end intelligent agent through an independent hardware scheduling device. The task requests include task content data and resource requirement information. Parse the task content data and extract its task feature vector. Based on the task feature vector and resource requirement information, a task scheduling instruction is generated according to a preset mapping relationship; the task scheduling instruction is used to indicate the large language model selection and resource allocation constraints in the subsequent scheduling process. S2. Input the task scheduling instructions into the dynamic scheduling model, which is a pre-trained and deployed neural network scheduling model; the dynamic scheduling model is used to generate cross-model scheduling schemes. The cross-model scheduling scheme includes: the combination of target large language models, the sub-task content allocated to each target large language model, the corresponding data distribution strategy, and the expected execution time window for each sub-task; For illustrative purposes, the target large language model is the large language model matched by the task request among multiple large language models, and is used as the target large language model. S3. Within an independent hardware scheduling device, task requests are preprocessed to obtain preprocessed data; Based on the data distribution strategy in the cross-model scheduling scheme, the part of the preprocessed data corresponding to the content of each sub-task is provided to the corresponding target large language model to form sub-task input data for each target large language model; S4. Based on the cross-model scheduling scheme, generate corresponding execution instructions for each target large language model and send the execution instructions to each target large language model in parallel, so that each target large language model can perform inference processing based on the corresponding subtask input data within its expected execution time window and output intermediate output results and / or final output results.

[0006] Preferably, in S1, the task content data is parsed, and its task feature vector is extracted, as follows: An embedding model is used to convert the natural language text in the task content data into numerical vectors, which serve as task feature vectors.

[0007] Preferably, in S2, the dynamic scheduling model generates a cross-model scheduling scheme based on the following inputs combined with task scheduling instructions: The current system status includes the length of the pending task queue, the number of tasks currently being executed, and the average task waiting time. Historical capability information of large language models, including the historical task success rate and average response latency of each candidate large language model; Current hardware resource status, including processor load, memory usage, and network bandwidth usage.

[0008] Preferably, in step S3, the task request is preprocessed to obtain preprocessed data, as follows: Identify structured fields and unstructured text in task content data; Convert structured fields to a preset format; Noise filtering is applied to unstructured text, removing illegal characters and meaningless symbols; Based on the preset list of sensitive fields, the sensitive fields in the task content data are desensitized and replaced with placeholders that have no impact on the large language model. Obtain the preprocessed data; The preprocessed data is associated with resource requirement information to form preprocessed data for subsequent distribution.

[0009] Preferably, S4 further includes: real-time monitoring of the execution status of each target large language model, including the current inference progress, response latency, and error codes; During real-time monitoring, the intermediate and final output results of each target large language model are collected simultaneously.

[0010] When an abnormal state is detected in the target large language model, the subtasks that have not yet been assigned to the target large language model are transferred to other models; if there are still unfinished subtasks in the target large language model, the waiting is stopped immediately and the current model is judged to have failed to execute the corresponding subtask.

[0011] Preferably, in S4, the abnormal state includes: Abnormal state a: No intermediate or final output result of the target large language model is received within the preset maximum response time; Abnormal state b: The error code returned by the target large language model indicates a network anomaly, authorization failure, or internal model error; Abnormal state c: During the current task execution, the target large language model continuously exceeds the corresponding resource threshold for a preset duration, depending on whether it is processor load, memory usage, or network bandwidth usage.

[0012] The intelligent multi-model cooperative scheduling method proposed in this invention has the following beneficial technical effects: 1. This application deploys the scheduling logic on a standalone hardware scheduling device in a local environment. This standalone hardware scheduling device is directly connected to the front-end intelligent agent (i.e., the task initiator) and multiple large language models (i.e., the task execution end) in the back-end through a physical interface, serving as the carrier for task scheduling. This ensures that user data is not leaked during the scheduling process from a physical perspective, meeting enterprise-level data security compliance requirements and alleviating data security issues that software solutions cannot guarantee. Furthermore, it achieves real-time scheduling decisions and load monitoring through a dynamic scheduling model, thereby significantly improving the efficiency and accuracy of multi-model collaborative processing.

[0013] 2. With the design of independent hardware scheduling equipment, system expansion only requires adding scheduling equipment or connecting more large language models, without reconstructing the entire system architecture, supporting elastic growth in business scale. Attached Figure Description

[0014] Figure 1 This is a flowchart of the method. Detailed Implementation

[0015] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar symbols denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0016] like Figure 1The method described herein is an intelligent multi-model collaborative scheduling method. This method is executed on an independent hardware scheduling device deployed in a local environment. The independent hardware scheduling device is connected to the front-end intelligent agent and multiple large language models at the back-end via physical interfaces. The method includes the following steps: S1. Receive task requests sent by the front-end intelligent agent through an independent hardware scheduling device. The task requests include task content data and resource requirement information. Parse the task content data and extract its task feature vector. In an optional embodiment, in S1, the task content data is parsed, and its task feature vector is extracted, as follows: An embedding model is used to convert the natural language text in the task content data into numerical vectors, which serve as task feature vectors. Based on the task feature vector and resource requirement information, a task scheduling instruction is generated according to a preset mapping relationship; the task scheduling instruction is used to indicate the model selection and resource allocation constraints in the subsequent scheduling process. S2. Input the task scheduling instructions into the dynamic scheduling model, which is a pre-trained and deployed neural network scheduling model; the dynamic scheduling model is used to generate cross-model scheduling schemes. In an optional embodiment, in S2, the dynamic scheduling model generates a cross-model scheduling scheme based on the following inputs combined with task scheduling instructions: The current system status includes the length of the pending task queue, the number of tasks currently being executed, and the average task waiting time. Historical capability information of large language models, including the historical task success rate and average response latency of each candidate large language model; Current hardware resource status, including processor load, memory usage, and network bandwidth usage; The cross-model scheduling scheme includes: the combination of target large language models, the sub-task content allocated to each target large language model, the corresponding data distribution strategy, and the expected execution time window for each sub-task; For illustrative purposes, the target large language model is the large language model matched by the task request among multiple large language models, and is used as the target large language model. In an optional embodiment, S3, within an independent hardware scheduling device, the task request is preprocessed to obtain preprocessed data; In S3, the task request is preprocessed to obtain preprocessed data, as follows: Identify structured fields and unstructured text in task content data; Convert structured fields to a preset format; Noise filtering is applied to unstructured text, removing illegal characters and meaningless symbols; Based on the preset list of sensitive fields, the sensitive fields in the task content data are desensitized and replaced with placeholders that have no impact on the large language model. Obtain the preprocessed data; The preprocessed data is associated with resource requirement information to form preprocessed data for subsequent distribution.

[0017] Based on the data distribution strategy in the cross-model scheduling scheme, the part of the preprocessed data corresponding to the content of each sub-task is provided to the corresponding target large language model to form sub-task input data for each target large language model; S4. According to the cross-model scheduling scheme, generate corresponding execution instructions for each target large language model and send the execution instructions to each target large language model in parallel, so that each target large language model can perform inference processing based on the corresponding subtask input data within its expected execution time window and output intermediate output results and / or final output results. In an optional embodiment, S4 further includes: real-time monitoring of the execution status of each target large language model, the execution status including current inference progress, response latency and error code; During real-time monitoring, the intermediate and final output results of each target large language model are collected simultaneously.

[0018] When an abnormal state is detected in the target large language model, the subtasks that have not yet been assigned to the target large language model are transferred to other models; if there are still unfinished subtasks in the target large language model, the waiting is stopped immediately and the current model is judged to have failed to execute the corresponding subtask.

[0019] In an optional embodiment, in S4, the abnormal state includes: Abnormal state a: No intermediate or final output result of the target large language model is received within the preset maximum response time; Abnormal state b: The error code returned by the target large language model indicates a network anomaly, authorization failure, or internal model error; Abnormal state c: During the current task execution, the target large language model continuously exceeds the corresponding resource threshold for a preset duration, depending on whether it is processor load, memory usage, or network bandwidth usage.

[0020] In an optional embodiment, in S2, the pre-training process of the dynamic scheduling model includes the following steps: S21. Construct the scheduling training dataset: Collect multiple rounds of task scheduling execution logs and corresponding front-end intelligent agent feedback information from the system trial operation and online operation phases; Each scheduling execution log entry should include at least: the task feature vector at that time, resource requirement information, current system status, real-time capability assessment information of candidate large language models, current hardware resource status, the actual cross-model scheduling scheme used, and result quality assessment indicators after the task is completed. The scheduling execution logs are associated with the corresponding quality scores and execution satisfaction evaluations to form a scheduling training dataset.

[0021] S22. Construct training samples: Each record in the scheduling training dataset is used to construct a training sample. The training samples include: The input feature vector is obtained by concatenating and normalizing the task feature vector, resource requirement information, current system state vector, candidate large language model capability evaluation vector, and hardware resource state vector. The supervision label includes the actual combination of target large language models used in the corresponding record, the subtask division of each model, the data distribution strategy and execution time window, and / or the scheduling performance score calculated by combining the result quality evaluation index and the execution latency index.

[0022] S23. Training the neural network scheduling model: A neural network with a multi-layer fully connected structure and an attention mechanism is selected as the network structure of the dynamic scheduling model. The input feature vector is input into the neural network, and the output is the predicted cross-model scheduling scheme or scheduling strategy parameters. Define a loss function that includes at least one or more of the following: cross-entropy loss to constrain the correctness of model combination and subtask partitioning, and mean squared error loss to constrain the scheduling performance score to approximate the true score. An optimization algorithm based on gradient descent is used to iteratively update the parameters of the neural network, and the network weights are adjusted according to the gradient of the loss function in each iteration. During training, the training set and validation set are divided. When the loss value on the validation set does not decrease or reaches the preset threshold in several consecutive rounds of training, training is stopped, and the neural network parameters at this time are used as the model parameters after training convergence.

[0023] S24. Model Solidification and Deployment: The neural network scheduling model after training convergence is quantized and compressed to convert floating-point parameters into fixed-point representations suitable for neural network processors to reduce storage consumption and inference latency. The quantized and compressed model parameter files and inference configuration files are written to the non-volatile storage space of the local independent hardware scheduling device; When the independent hardware scheduling device is powered on or restarted, the model parameter file is loaded into the local storage area of ​​the neural network processor, and the loaded neural network is registered as a dynamic scheduling model, so that it is in a working state that can be called by step S2. In subsequent operation, the dynamic scheduling model, as a fixed inference model, receives task scheduling instructions and replaces the locally stored model parameters periodically or as needed, thereby enabling continuous optimization of model performance without leaking scheduling experience data.

[0024] This application deploys the scheduling logic on a standalone hardware scheduling device in a local environment. This standalone hardware scheduling device is directly connected to the front-end intelligent agent (i.e., the task initiator) and multiple large language models (i.e., the task execution end) in the back-end through a physical interface, serving as the carrier for task scheduling. This ensures that user data is not leaked during the scheduling process from a physical perspective, meeting enterprise-level data security compliance requirements and alleviating data security issues that software solutions cannot guarantee. Furthermore, it achieves real-time scheduling decisions and load monitoring through a dynamic scheduling model, thereby significantly improving the efficiency and accuracy of multi-model collaborative processing.

[0025] With the design of independent hardware scheduling equipment, system expansion only requires adding scheduling equipment or connecting more large language models, without reconstructing the entire system architecture, supporting elastic growth in business scale.

[0026] For clarification, "acquisition" in this application refers to obtaining the required content or data using existing technical means.

[0027] Furthermore, any content not described in detail in this specification is existing technology known to those skilled in the art.

[0028] In the embodiments provided by this invention, it should be understood that the disclosed system or method can be implemented in other ways. For example, the embodiments of the invention described above are merely illustrative; for instance, the division of modules is only a logical functional division, and there may be other division methods in actual implementation.

[0029] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0030] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or in the form of hardware plus software functional modules.

[0031] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the basic characteristics of the present invention.

[0032] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. An intelligent multi-model cooperative scheduling method, characterized in that, The method is executed on a standalone hardware scheduling device deployed in a local environment. This standalone hardware scheduling device connects to the front-end intelligent agent and multiple large language models in the back-end via physical interfaces, and includes the following steps: S1. Receive task requests sent by the front-end intelligent agent through an independent hardware scheduling device. The task requests include task content data and resource requirement information. Parse the task content data and extract its task feature vector. Based on the task feature vector and resource requirement information, a task scheduling instruction is generated according to a preset mapping relationship; the task scheduling instruction is used to indicate the model selection and resource allocation constraints in the subsequent scheduling process. S2. Input the task scheduling instructions into the dynamic scheduling model, which is a pre-trained and deployed neural network scheduling model; the dynamic scheduling model is used to generate cross-model scheduling schemes. The cross-model scheduling scheme includes: the combination of target large language models, the sub-task content allocated to each target large language model, the corresponding data distribution strategy, and the expected execution time window for each sub-task; S3. Within an independent hardware scheduling device, task requests are preprocessed to obtain preprocessed data; Based on the data distribution strategy in the cross-model scheduling scheme, the part of the preprocessed data corresponding to the content of each sub-task is provided to the corresponding target large language model to form sub-task input data for each target large language model; S4. Based on the cross-model scheduling scheme, generate corresponding execution instructions for each target large language model and send the execution instructions to each target large language model in parallel, so that each target large language model can perform inference processing based on the corresponding subtask input data within its expected execution time window and output intermediate output results and / or final output results.

2. The intelligent multi-model cooperative scheduling method according to claim 1, characterized in that, In S1, the task content data is parsed, and its task feature vector is extracted, as follows: An embedding model is used to convert the natural language text in the task content data into numerical vectors, which serve as task feature vectors.

3. The intelligent multi-model cooperative scheduling method according to claim 1, characterized in that, In S2, the dynamic scheduling model generates a cross-model scheduling scheme based on the following inputs and task scheduling instructions: The current system status includes the length of the pending task queue, the number of tasks currently being executed, and the average task waiting time. Historical capability information of large language models, including the historical task success rate and average response latency of each candidate large language model; Current hardware resource status, including processor load, memory usage, and network bandwidth usage.

4. The intelligent multi-model cooperative scheduling method according to claim 1, characterized in that, In S3, the task request is preprocessed to obtain preprocessed data, as follows: Identify structured fields and unstructured text in task content data; Convert structured fields to a preset format; Noise filtering is applied to unstructured text, removing illegal characters and meaningless symbols; Based on the preset list of sensitive fields, the sensitive fields in the task content data are desensitized and replaced with placeholders that have no impact on the large language model. Obtain the preprocessed data; The preprocessed data is associated with resource requirement information to form preprocessed data for subsequent distribution.

5. The intelligent multi-model cooperative scheduling method according to claim 1, characterized in that, S4 also includes: real-time monitoring of the execution status of each target large language model, including the current inference progress, response latency and error codes; During real-time monitoring, the intermediate and final output results of each target large language model are collected simultaneously.

6. When an abnormal state is detected in the target large language model, the subtasks that have not yet been assigned to the target large language model are transferred to other models; if there are still unfinished subtasks in the target large language model, the waiting is stopped immediately and the current model is judged to have failed to execute the corresponding subtask.

7. The intelligent multi-model cooperative scheduling method according to claim 5, characterized in that, In S4, abnormal states include: Abnormal state a: No intermediate or final output result of the target large language model is received within the preset maximum response time; Abnormal state b: The error code returned by the target large language model indicates a network anomaly, authorization failure, or internal model error; Abnormal state c: During the current task execution, the target large language model continuously exceeds the corresponding resource threshold for a preset duration, depending on whether it is processor load, memory usage, or network bandwidth usage.

Images