Computing power management method of intelligent computing center and model training method and device

Abstract

The invention provides a computing power management method of an intelligent computing center and a model training method and device.The computing power management method comprises the steps that S1, multi-modal operation data of a multi-modal computing task operated on the intelligent computing center at the current moment are obtained, extracting context features of each modal operation data from the multi-modal operation data, and combining all the context features to obtain feature vectors; the multi-modal operation data comprises modal operation data of at least one of the following modals: texts, monitoring indexes and visual data; s2, inputting the feature vector into a multi-modal resource allocation large language model deployed in an intelligent computing center to obtain a multi-modal resource allocation strategy for allocating computing power resources to a multi-modal computing task; and S3, indicating the intelligent computing center to execute the resource allocation strategy. According to the method, the multi-modal resource allocation strategy can be efficiently formulated for the multi-modal computing task, and the management efficiency of the computing power of the intelligent computing center is improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the technical field of computing power infrastructure, and in particular, to a computing power management method, model training method and device for an intelligent computing center. Background Art

[0002] With the development of artificial intelligence technology and computing power technology, the concept of intelligent computing center has emerged. "Intelligent computing center" refers to the use of large-scale heterogeneous computing power resources, including general computing power and intelligent computing power, mainly for artificial intelligence applications (such as artificial intelligence deep learning model development, model training and model reasoning and other scenarios) to provide the required computing power, data and algorithms. Intelligent computing center covers facilities, hardware, software, and can provide full-stack capabilities from bottom-level computing power to top-level application enablement.

[0003] In the scenario of allocating computing power resources for multimodal computing tasks, the existing computing power management method requires manual allocation strategies for each multimodal computing task in advance, and the allocation strategies are called and executed accordingly when executing multimodal computing tasks. The efficiency of manually specifying allocation strategies is low, resulting in low efficiency in the management of computing power resources. Summary of the invention

[0004] The embodiments of the present invention provide a computing power management method, a model training method and a device for an intelligent computing center to solve the problem that in the scenario of allocating computing power resources for multimodal computing tasks, the existing computing power management method requires manual pre-setting of allocation strategies for each multimodal computing task, and the allocation strategies are correspondingly retrieved and executed when executing the multimodal computing tasks. The efficiency of manually specifying allocation strategies is low, resulting in low management efficiency of computing power resources.

[0005] In order to solve the above-mentioned technical problems, the present invention is achieved as follows:

[0006] In a first aspect, an embodiment of the present invention provides a computing power management method for an intelligent computing center, including:

[0007] Step S1: obtaining multimodal operation data of a multimodal computing task running on an intelligent computing center at a current moment, extracting context features of each modal operation data from the multimodal operation data, and combining all the context features to obtain a feature vector; the multimodal operation data includes the modal operation data of at least one of the following modalities: text, monitoring indicators, and visual data;

[0008] Step S2: inputting the feature vector into a multimodal resource allocation large language model deployed in the intelligent computing center to obtain a multimodal resource allocation strategy for allocating computing resources to the multimodal computing task;

[0009] Step S3: instructing the intelligent computing center to execute the resource allocation strategy.

[0010] Optionally, the step S3 includes:

[0011] Instructing the intelligent computing center to execute the resource allocation strategy under quota constraints;

[0012] The quota constraint condition is expressed as follows:

[0013]

[0014] Among them, C t R is the upper limit of the total amount of resources of the intelligent computing center at time t; i (t) is the computing power resource of computing task i at time t, and R i (t)≥0, which is the resource non-negativity constraint of each computing task i; N is the total number of computing tasks.

[0015] In a second aspect, an embodiment of the present invention provides a method for training a large language model for multimodal resource allocation, comprising:

[0016] Step S01: acquiring historical multimodal operation data of a multimodal computing task on an intelligent computing center, extracting historical context features of each modality operation data from the historical multimodal operation data, and combining all the historical context features to obtain a historical feature vector; the historical multimodal operation data includes the modality operation data of at least one of the following modalities: text, monitoring indicators, and visual data;

[0017] Step S02: obtaining a historical resource allocation strategy for the multimodal computing task executed by the intelligent computing center corresponding to the historical multimodal operation data; and labeling the historical feature vector with the historical resource allocation strategy to obtain a training feature vector;

[0018] Step S03: using the training feature vector to train a multimodal large language model to obtain a multimodal resource allocation large language model.

[0019] Optionally, the objective function of training the multimodal large language model is determined according to the priority weight of the multimodal computing task and the resource requirement of the multimodal computing task.

[0020] Optionally, the objective function is expressed as follows:

[0021]

[0022] Among them, u i (t) = MMLLM priority )X i(t)) is a multimodal large language model MMLLM based on the historical feature vector X i (t) The calculated priority weight of computing task i at time t; R i (t) = MMLLM resource (X i (t)) is a multimodal large language model MMLLM based on the historical feature vector X i (t) The predicted resource requirements of computing task i; N is the total number of computing tasks.

[0023] In a third aspect, an embodiment of the present invention provides a computing power management device for an intelligent computing center, including:

[0024] An acquisition module is used to acquire multimodal operation data of a multimodal computing task running on an intelligent computing center at a current moment, extract context features of each modal operation data from the multimodal operation data, and combine all the context features to obtain a feature vector; the multimodal operation data includes the modal operation data of at least one of the following modalities: text, monitoring indicators, and visual data;

[0025] An execution module, configured to input the feature vector into a multimodal resource allocation large language model deployed in the intelligent computing center, and obtain a multimodal resource allocation strategy for allocating computing power resources to the multimodal computing task;

[0026] The execution module is also used to instruct the intelligent computing center to execute the resource allocation strategy.

[0027] In a fourth aspect, an embodiment of the present invention provides a training device for a large language model for multimodal resource allocation, comprising:

[0028] A training data acquisition module, used to acquire historical multimodal operation data of a multimodal computing task on an intelligent computing center, extract historical context features of each modal operation data from the historical multimodal operation data, and combine all the historical context features to obtain a historical feature vector; the historical multimodal operation data includes the modal operation data of at least one of the following modalities: text, monitoring indicators, and visual data;

[0029] A labeling module, used for obtaining a historical resource allocation strategy for the multimodal computing task executed by the intelligent computing center corresponding to the historical multimodal operation data; and labeling the historical feature vector corresponding to the historical resource allocation strategy to obtain a training feature vector;

[0030] The training module is used to train the multimodal large language model using the training feature vector to obtain the multimodal resource allocation large language model.

[0031] In a fifth aspect, an embodiment of the present invention provides an electronic device, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method for managing computing power of an intelligent computing center as described in any one of the first aspects, or implements the steps of the method for training a large language model for multimodal resource allocation as described in any one of the second aspects.

[0032] In a sixth aspect, an embodiment of the present invention provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the computing power management method of the intelligent computing center as described in any one of the first aspects are implemented, or the steps of the training method of a large language model for multimodal resource allocation as described in any one of the second aspects are implemented.

[0033] In the seventh aspect, an embodiment of the present invention provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps of the computing power management method of the intelligent computing center as described in any one of the first aspects, or implement the steps of the training method of a large language model for multimodal resource allocation as described in any one of the second aspects.

[0034] In an embodiment of the present invention, through step S1: obtaining the multimodal operation data of the multimodal computing task running on the intelligent computing center at the current moment, extracting the context features of each modal operation data from the multimodal operation data, and combining all the context features to obtain a feature vector; the multimodal operation data includes the modal operation data of at least one of the following modalities: text, monitoring indicators, and visual data; step S2: inputting the feature vector into a multimodal resource allocation large language model deployed in the intelligent computing center to obtain a multimodal resource allocation strategy for allocating computing resources to the multimodal computing task; step S3: instructing the intelligent computing center to execute the resource allocation strategy. The embodiment of the present invention can efficiently formulate a multimodal resource allocation strategy for multimodal computing tasks, thereby improving the management efficiency of the computing power of the intelligent computing center. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0036] Figure 1 A flowchart of a computing power management method for an intelligent computing center according to an embodiment of the present invention;

[0037] Figure 2 A flowchart of a method for training a large language model for multimodal resource allocation according to an embodiment of the present invention;

[0038] Figure 3 This is a schematic diagram of the system architecture of intelligent computing power scheduling - dynamic isolation and quota management system for computing resources;

[0039] Figure 4 A schematic diagram of the execution results of the resource allocation strategy obtained for the intelligent computing power scheduling - dynamic isolation and quota management system of computing resources;

[0040] Figure 5 This is a principle block diagram of a computing power management device for an intelligent computing center according to an embodiment of the present invention;

[0041] Figure 6 A principle block diagram of a training device for a large language model for multimodal resource allocation according to an embodiment of the present invention;

[0042] Figure 7 The figure is a functional block diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] The terms "first", "second", etc. of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of one type, and the number of objects is not limited, for example, the first object can be one or more. In addition, "or" in the present application represents at least one of the connected objects. For example, "A or B" covers three schemes, namely, Scheme 1: including A but not including B; Scheme 2: including B but not including A; Scheme 3: including both A and B. The character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0045] In the technical solution of the present disclosure, words such as “connect”, “couple” or “connected” are not limited to physical or mechanical connections, but may include electrical connections.

[0046] In addition, the technical features involved in the different embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0047] First, the technical terms involved in the present invention are briefly explained below.

[0048] The "computing power" mentioned in the present invention is the ability of computer equipment or computing / data center to process information. It is the ability of computer hardware and software to work together to execute certain computing requirements. It is the computing power to achieve target result output by processing information data. It is a new type of productivity that integrates information computing power, network carrying capacity, and data storage capacity. It mainly provides services to the society through computing power infrastructure.

[0049] The "computing power" (Computational Power, CP) described in the present invention is the ability of a data center server to process data and output results. It is a comprehensive indicator to measure the computing power of a data center, including general computing power, super computing power and intelligent computing power. The commonly used unit of measurement is the number of floating point operations performed per second (FLOPS: Floating Point Operations Per Second, 1EFLOPS = 10^18FLOPS). The larger the value, the stronger the comprehensive computing power. According to calculations, 1EFLOPS is approximately the computing power output of 5 Tianhe-2A or 500,000 mainstream server CPUs or 2 million mainstream notebooks. The calculation formula is: CP = CP 通用 +CP 智能 +CP 超级 .

[0050] The "Network Power" (NP) described in the present invention is a manifestation of the data transmission capability of computing facilities, including comprehensive capabilities such as network architecture, network bandwidth, transmission latency, intelligent management and scheduling, etc. The carrying capacity involves network transmission within and between data centers, and is a comprehensive indicator for measuring network transmission scheduling capabilities. In the embodiment of the present invention, the carrying capacity uses the memory bandwidth.

[0051] The "Storage Power" (SP) described in the present invention is the comprehensive ability of a data center in terms of data storage capacity, performance, safety and reliability, and green and low-carbon. It is a comprehensive indicator for measuring the data storage capacity of a data center, including external storage devices such as storage arrays and server built-in storage devices. The commonly used unit of measurement for storage capacity is exabyte (EB, 1EB = 2^60bytes), and the commonly used unit of measurement for performance is the number of reads and writes per second per unit capacity (IOPS / TB, Input / Output Operations Per Second / TB). The disaster recovery ratio is an important manifestation of safety and reliability.

[0052] The "computing power infrastructure" described in the present invention is a new type of information infrastructure that integrates information computing power, network carrying capacity, and data storage capacity. It can realize centralized computing, storage, transmission and application of information, and presents characteristics such as diversity and ubiquity, intelligence and agility, security and reliability, and green and low-carbon.

[0053] The “computing power” mentioned in the present invention includes general computing power, intelligent computing power and super computing power.

[0054] The “general computing power” mentioned in the present invention refers to the computing power provided by servers based on CPU (Central Processing Unit) chips, which is used to support basic general computing such as cloud computing and edge computing.

[0055] The "intelligent computing power" described in the present invention is a computing platform for large-scale deployment of special chips such as GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), ASIC (Application Specific Integrated Circuit) for various innovative applications of artificial intelligence, such as natural language processing and machine vision.

[0056] The "super computing power" mentioned in the present invention is mainly the computing power provided by high-performance computing clusters such as supercomputers. It utilizes the centralized computing resources of multiple computer systems working in parallel and uses a dedicated operating system to handle extremely complex or data-intensive problems. It is mainly used for calculations in cutting-edge scientific fields, such as planetary simulation, drug molecule design, and gene analysis.

[0057] The "intelligent computing center" described in the present invention refers to a facility that provides the required computing power, data and algorithms for artificial intelligence applications (such as artificial intelligence deep learning model development, model training and model reasoning scenarios) by using large-scale heterogeneous computing power resources, including general computing power (CPU: Central Processing Unit) and intelligent computing power (GPU: Graphics Processing Unit, FPGA: Field Programmable Gate Array, ASIC: Application Specific Integrated Circuit, etc.). The intelligent computing center covers facilities, hardware, and software, and can provide full-stack capabilities from bottom-level computing power to top-level application enablement.

[0058] The "computing resources" mentioned in the present invention refer to the technologies and facilities with information calculation, transmission, storage and application capabilities required for the development of the digital society, including but not limited to computing resources such as CPU (Central Processing Unit), GPU (Graphics Processing Unit), network resources such as switches and routers, storage resources such as storage arrays and distributed storage, security resources such as firewalls and intrusion detection systems, as well as supporting and guarantee resources such as wind, fire, water and electricity.

[0059] The "Large Language Model (LLM)" described in the present invention is an artificial intelligence model based on deep learning, which is designed to understand and generate natural language. These models are usually trained with large amounts of text data to learn the structure, grammar, semantics and context of the language, and can perform a wide range of tasks including text summarization, translation, sentiment analysis, etc.

[0060] The "Multimodal Large Language Model (MMLLM)" described in the present invention is an artificial intelligence model that can process multiple modal data such as text, images, and monitoring data. MMLLM realizes the prediction and priority judgment of resource requirements through cross-modal feature extraction and semantic fusion to generate a resource allocation strategy that meets dynamic load requirements. It is the core technology of this system to realize intelligent resource management.

[0061] The embodiment of the present invention provides a computing power management method for an intelligent computing center, see Figure 1 As shown, Figure 1 The figure is a flow chart of a computing power management method of an intelligent computing center according to an embodiment of the present invention. The computing power management method of an intelligent computing center includes:

[0062] Step S1: obtaining multimodal operation data of a multimodal computing task running on an intelligent computing center at a current moment, extracting context features of each modal operation data from the multimodal operation data, and combining all the context features to obtain a feature vector; the multimodal operation data includes the modal operation data of at least one of the following modalities: text, monitoring indicators, and visual data;

[0063] Step S2: inputting the feature vector into a multimodal resource allocation large language model deployed in the intelligent computing center to obtain a multimodal resource allocation strategy for allocating computing resources to the multimodal computing task;

[0064] Step S3: instructing the intelligent computing center to execute the resource allocation strategy.

[0065] It should be noted that in the embodiments of the present invention, modalities are distinguished by the type of data processed by the computing task (also called modal data). For example, two computing tasks that process data of text and video correspond to two different modalities. Therefore, it can be understood that a multimodal computing task is a computing task that needs to process multiple modal data.

[0066] In the embodiment of the present invention, context features refer to context features extracted from multiple modal data (e.g., text, monitoring indicators, visual data) of modal operation data for describing information such as task status, system load, and device status. Each modal data has its own specific context features, for example:

[0067] For text, the context feature may include at least one of the following: task status, load peak, and abnormal event description in the text log.

[0068] For monitoring indicators, context features may include at least one of the following: time series changes of monitoring data (eg, CPU usage, memory usage, I / O rate, etc.), and dynamic conditions reflecting system load.

[0069] For visual data, the context feature may include at least one of the following: hardware anomalies identified in the device status image, and hardware status change information identified in the device status image.

[0070] These context features are cross-modally fused to generate a multimodal feature vector (that is, a feature vector obtained by combining all the context features in the embodiment of the present invention), which can provide comprehensive support for resource demand prediction and allocation strategy generation.

[0071] In some embodiments of the present invention, optionally, a method for training a large language model for multimodal resource allocation includes: obtaining historical multimodal operation data of a multimodal computing task on an intelligent computing center, extracting historical context features of each modal operation data from the historical multimodal operation data, and combining all the historical context features to obtain a historical feature vector; the historical multimodal operation data includes the modal operation data of at least one of the following modalities: text, monitoring indicators, and visual data; obtaining a historical resource allocation strategy executed by the intelligent computing center on the multimodal computing task corresponding to the historical multimodal operation data; and annotating the historical feature vector with the historical resource allocation strategy to obtain a training feature vector; and using the training feature vector to train a large multimodal language model to obtain a large multimodal resource allocation language model. Thus, the large multimodal resource allocation language model deployed in the intelligent computing center in step S2 can output a multimodal resource allocation measurement based on the input feature vector.

[0072] In an embodiment of the present invention, through step S1: obtaining the multimodal operation data of the multimodal computing task running on the intelligent computing center at the current moment, extracting the context features of each modal operation data from the multimodal operation data, and combining all the context features to obtain a feature vector; the multimodal operation data includes the modal operation data of at least one of the following modalities: text, monitoring indicators, and visual data; step S2: inputting the feature vector into a multimodal resource allocation large language model deployed in the intelligent computing center to obtain a multimodal resource allocation strategy for allocating computing resources to the multimodal computing task; step S3: instructing the intelligent computing center to execute the resource allocation strategy. The embodiment of the present invention can efficiently formulate a multimodal resource allocation strategy for multimodal computing tasks, thereby improving the management efficiency of the computing power of the intelligent computing center.

[0073] In some embodiments of the present invention, optionally, step S3 includes:

[0074] Instructing the intelligent computing center to execute the resource allocation strategy under quota constraints;

[0075] The quota constraint condition is expressed as follows:

[0076]

[0077] Among them, C t R is the upper limit of the total amount of resources of the intelligent computing center at time t; i (t) is the computing power resource of computing task i at time t, and R i (t)≥0, which is the resource non-negativity constraint of each computing task i; N is the total number of computing tasks.

[0078] Through the above quota constraints, the embodiment of the present invention realizes guidance for the intelligent computing center in the specific execution process, effectively avoids errors caused by exceeding the total resource limit, improves the stability and reliability of resource allocation in the intelligent computing center, and ensures safety.

[0079] The embodiment of the present invention provides a method for training a large language model for multimodal resource allocation, see Figure 2 As shown, Figure 2 This is a flow chart of a method for training a large language model for multimodal resource allocation according to an embodiment of the present invention. The method for training a large language model for multimodal resource allocation includes:

[0080] Step S01: acquiring historical multimodal operation data of a multimodal computing task on an intelligent computing center, extracting historical context features of each modality operation data from the historical multimodal operation data, and combining all the historical context features to obtain a historical feature vector; the historical multimodal operation data includes the modality operation data of at least one of the following modalities: text, monitoring indicators, and visual data;

[0081] Step S02: obtaining a historical resource allocation strategy for the multimodal computing task executed by the intelligent computing center corresponding to the historical multimodal operation data; and labeling the historical feature vector with the historical resource allocation strategy to obtain a training feature vector;

[0082] Step S03: using the training feature vector to train a multimodal large language model to obtain a multimodal resource allocation large language model.

[0083] It should be noted that in the embodiments of the present invention, modalities are distinguished by the type of data processed by the computing task (also called modal data). For example, two computing tasks that process data of text and video correspond to two different modalities. Therefore, it can be understood that a multimodal computing task is a computing task that needs to process multiple modal data.

[0084] In the embodiment of the present invention, context features refer to context features extracted from multiple modal data (e.g., text, monitoring indicators, visual data) of modal operation data for describing information such as task status, system load, and device status. Each modal data has its own specific context features, for example:

[0085] For text, the context feature may include at least one of the following: task status, load peak, and abnormal event description in the text log.

[0086] For monitoring indicators, context features may include at least one of the following: time series changes of monitoring data (eg, CPU usage, memory usage, I / O rate, etc.), and dynamic conditions reflecting system load.

[0087] For visual data, the context feature may include at least one of the following: hardware anomalies identified in the device status image, and hardware status change information identified in the device status image.

[0088] These contextual features are cross-modally fused to generate a multimodal feature vector (that is, a historical feature vector is obtained by combining all the historical contextual features in the embodiment of the present invention) for model training, ensuring that the trained multimodal large language model can accurately learn the historical operation characteristics of the computing task.

[0089] In step S02 of the embodiment of the present invention, the historical resource allocation strategy executed by the intelligent computing center on the multimodal computing task corresponding to the historical multimodal operation data is obtained; and the historical feature vector is labeled corresponding to the historical resource allocation strategy to obtain a training feature vector. The embodiment of the present invention labels the corresponding historical resource allocation strategy based on the rich computing task characteristics (historical feature vectors), so that the trained multimodal large language model can accurately learn the historical operation characteristics of the computing task, and accurately learn the correspondence between the historical operation characteristics and the resource allocation strategy, thereby improving the accuracy of learning cognition and ensuring that a multimodal resource allocation large language model that outputs a resource allocation strategy with high accuracy can be trained.

[0090] As a comprehensive model, the Multimodal Large Language Model (MMLLM) can achieve in-depth analysis of complex environments by understanding and reasoning about multi-modal data. MMLLM combines the capabilities of natural language processing, multimodal fusion, and adaptive reasoning, giving it the following significant advantages in the field of computing resource management:

[0091] Cross-modal information fusion: MMLLM has the ability to extract deep features from multiple modal data such as text, images, monitoring indicators, system logs, etc. For example, the system can extract task status and exception descriptions from text logs, capture fluctuation trends in resource usage from monitoring indicators, and identify hardware anomalies from visual data (e.g., device status images). MMLLM is able to integrate data from different modalities into a unified context feature vector, thereby providing comprehensive load information for the system.

[0092] Multi-level semantic understanding and reasoning: MMLLM's multimodal reasoning capability enables it to understand the relationships and contexts between different modal data. For example, when the system load suddenly increases, MMLLM can infer the cause by monitoring the numerical changes in the data and the abnormal records in the text log, and generate an adaptive resource allocation strategy. This multi-level reasoning capability can effectively handle dynamically changing load environments and ensure the system's response speed and scheduling accuracy in complex environments.

[0093] Real-time prediction and dynamic scheduling: MMLLM can predict resource requirements in real time by analyzing multimodal data, and generate dynamic allocation strategies to meet the needs of high concurrency, multi-tasking and high-load environments. Compared with traditional static rules or simple machine learning models, MMLLM can perform fine-grained analysis and dynamic prediction of the current load, adapting to frequent load fluctuations, making it particularly suitable for scenarios with high real-time requirements such as big data processing, streaming media, and edge computing.

[0094] Adaptive optimization and feedback loop: This paper uses the adaptive optimization characteristics of MMLLM to feed back the resource allocation effect to the model through a real-time feedback loop mechanism, so that it can perform self-optimization based on incremental learning of multimodal data. This feedback mechanism enables the system to quickly adapt to new load patterns and task requirements when they arise, reducing the need for frequent retraining of the model and improving the stability and efficiency of the system.

[0095] Based on the above characteristics and advantages of MMLLM, when the training feature vectors obtained in steps S01 and S02 are used to perform model training in step S03 of the embodiment of the present invention, a multimodal resource allocation large language model that outputs resource allocation strategies with high accuracy can be trained.

[0096] In the embodiment of the present invention, through step S01, step S02 and step S03, the embodiment of the present invention can train a multimodal resource allocation large language model that outputs resource allocation strategies with high accuracy.

[0097] In some embodiments of the present invention, optionally, the objective function for training the multimodal large language model is determined according to the priority weight of the multimodal computing task and the resource requirements of the multimodal computing task. In an embodiment of the present invention, the objective function is used to control model training. Further, the objective function is determined according to the priority weight of the multimodal computing task and the resource requirements of the multimodal computing task, that is, the priority weight of the multimodal computing task and the resource requirements of the multimodal computing task are used for the control of model training, which is conducive to improving training efficiency.

[0098] In some embodiments of the present invention, the objective function is expressed as follows:

[0099]

[0100] Among them, u i (t) = MMLLM priority (X i (t)) is a multimodal large language model MMLLM based on the historical feature vector X i (t) The calculated priority weight of computing task i at time t; R i (t) = MMLLM resource (X i (t)) is a multimodal large language model MMLLM based on the historical feature vector X i (t) The predicted resource requirements of computing task i; N is the total number of computing tasks.

[0101] That is, the objective function is used to control the trained multimodal resource allocation large language model to maximize the product of priority weight and resource demand. In other words, the multimodal resource allocation large language model can ensure that the resource requirements of high-priority tasks are met first, while taking into account the basic resource allocation of low-priority tasks.

[0102] The following is a description with reference to specific embodiments.

[0103] An intelligent computing power scheduling - computing resource dynamic isolation and quota management system based on a multimodal large language model (Multimodal Large Language Model, MMLLM) (hereinafter referred to as the system or the system, that is, the executive body of the embodiment of the present invention). Traditional resource management methods mainly rely on single-modal data (such as logs or monitoring indicators). When dealing with complex and highly dynamic load scenarios, they cannot fully capture the multi-dimensional demand relationships and diverse system states of different tasks, resulting in low efficiency in resource allocation and scheduling. This system introduces the multimodal understanding capabilities of MMLLM to integrate and analyze multimodal information from text logs, system monitoring data, visual information (such as device status images) and other unstructured data, and constructs comprehensive contextual features for real-time prediction and dynamic adjustment of computing resource allocation strategies.

[0104] See also Figure 3 As shown, Figure 3 This is a schematic diagram of the system architecture of the intelligent computing power scheduling-dynamic isolation and quota management system for computing resources. The system includes the following modules: Data Collection Module (DCM): Real-time collection of multimodal data, including text logs, system monitoring data, and visual data, to provide data support for subsequent feature extraction and prediction. Multimodal Feature Extraction Module (MFTM): Extracts and integrates features of data of different modalities to generate a high-dimensional feature vector X(t) as the input of MMLLM. Multimodal Large Language Model Reasoning Module (MMLLM-IM): Generates resource allocation strategy A(t) and dynamically predicts resource requirements through the multimodal context understanding and reasoning capabilities of MMLLM. Resource Management Module (RMM): Based on the strategy generated by MMLLM, adjusts the resource quotas of different tasks to ensure the flexibility and adaptability of resource allocation. Feedback and Adaptive Optimization Module (FOM): Monitors the execution effect of the strategy and uses multimodal feedback data for incremental learning and adaptive updating of MMLLM to improve the accuracy and responsiveness of the model.

[0105] In order to improve the system's comprehensive understanding of resource requirements, the system integrates data sources from multiple modalities, including but not limited to the following categories:

[0106] Text logs: System monitoring logs contain a large amount of text information describing task status and abnormal situations. Through MMLLM's natural language processing capabilities, the system can extract key contextual information from the logs, such as load peaks, abnormal events, and task status changes. This information provides descriptive features of the system status, which helps MMLLM generate more accurate resource demand forecasts.

[0107] System monitoring indicators: The system extracts key numerical indicators (such as CPU usage, memory usage, I / O rate, and network traffic) from structured monitoring data, which can reflect the resource load of the system in real time. MMLLM integrates these indicators with text information to better capture the time pattern and peak characteristics of load changes.

[0108] Visual information: In many computing resource management scenarios, visual information about device status (such as images of device working status) can provide direct visual data to the system to help identify hardware anomalies and system failures. MMLLM can process visual data, identify device status changes, and provide intuitive device status support for resource scheduling.

[0109] Through the multimodal feature extraction capability of MMLLM, the system can semantically fuse these different modal data and generate feature vectors containing contextual information to ensure the comprehensiveness and accuracy of resource demand forecasts.

[0110] Based on multimodal feature extraction, MMLLM uses its deep semantic reasoning capabilities to generate resource allocation strategies in real time to ensure that the system can achieve refined resource management in a load fluctuation environment. The specific process includes the following:

[0111] Feature vector generation: The system first integrates multimodal data to generate a feature vector X(t), which contains text, monitoring indicators, and visual data features that describe resource requirements. MMLLM builds a complete load demand picture through a multi-level understanding of feature vectors. Contextual information of different modalities refers to contextual features extracted from multimodal data (including text, monitoring indicators, and visual data) to describe information such as task status, system load, and device status.

[0112] Each modality of data has its own specific contextual characteristics, such as:

[0113] Text modal context features: descriptions of task status, load peaks, or abnormal events in text logs. Monitoring indicator modal context features: time series changes in monitoring data (such as CPU usage, memory usage, I / O rate, etc.) reflect the dynamic situation of system load. Visual modal context features: hardware anomalies or status change information identified in device status images. These context information are cross-modally fused to generate multimodal feature vectors, providing comprehensive background support for resource demand prediction and allocation strategy generation.

[0114] Multimodal reasoning and policy generation: MMLLM performs deep reasoning based on the input feature vector X(t) and generates a resource allocation policy A(t) by fusing contextual information from different modalities. For example, when text logs and monitoring data simultaneously show a sudden increase in load, MMLLM can verify the device status based on visual data, determine whether resource demand requires an immediate response, and generate an adaptive allocation policy. This cross-modal reasoning ensures that the system achieves high accuracy and adaptability in resource allocation.

[0115] Among them, cross-modality is mainly reflected in the following aspects:

[0116] A. Multimodal feature extraction and fusion: The system extracts features from text (such as task status, exception logs), monitoring indicators (such as CPU usage, memory usage), and visual data (such as device status images), and fuses these features to generate high-dimensional feature vectors. These vectors comprehensively describe the global status of the task and system.

[0117] B. Cross-modal reasoning capability: MMLLM performs reasoning by inputting fused multi-modal feature vectors and combining them with contextual information. For example, when text logs and monitoring indicators show a sudden increase in load, the system can confirm the device status through visual data, determine whether resource allocation is needed, and generate a dynamic resource allocation strategy.

[0118] C. Contextual association: Cross-modal fusion can discover potential relationships between different modalities. For example, the description of "load anomaly" in the text log may form a consistent cross-modal association with the "CPU utilization surge" in the monitoring data and the device failure in the visual image.

[0119] To achieve cross-modal reasoning, the following steps are required:

[0120] Data preparation:

[0121] Collect diverse datasets covering different modalities (text, monitoring metrics, visual data).

[0122] The data is preprocessed, including text cleaning, indicator normalization, and image enhancement, to ensure input consistency.

[0123] Model architecture design:

[0124] Use a model architecture that supports multimodal fusion (such as MMLLM), the core of which is the ability to process inputs from different modalities and generate a unified feature representation.

[0125] Design modular feature extractors (such as NLP models to process text, CNN to process visual data, and statistical models to process monitoring indicators), and combine them with cross-modal attention mechanisms to achieve multimodal fusion.

[0126] Joint training:

[0127] Construct a joint training objective so that the model can simultaneously optimize the extraction of different modal features and cross-modal semantic fusion. Use supervised learning methods to train the model using labeled data (such as task priority or resource requirement labels). Use a cross-modal consistency loss function to ensure semantic alignment between different modal information.

[0128] Feedback optimization:

[0129] Collect system operation feedback in real time, such as task completion time, resource utilization, etc., as additional training data. Use incremental learning or online fine-tuning methods to continuously improve the model's cross-modal reasoning capabilities.

[0130] D. Resource isolation and dynamic quota management: Based on the strategy generated by MMLLM, the system isolates resources for different tasks and allocates refined quotas to ensure resource stability and fairness under high load conditions. The system can allocate different CPU, memory, and I / O resources according to task priority and real-time requirements to ensure task performance.

[0131] To ensure the continuous adaptability of MMLLM in complex load environments, this system designs a feedback closed-loop mechanism, which monitors the execution effect of the allocation strategy in real time and uses multimodal feedback data for adaptive optimization of the model. The specific steps include:

[0132] 1. Feedback data collection: The system collects data after task execution in real time, including task completion time, resource utilization, system response time, etc. These data reflect the actual effect of the allocation strategy and serve as an important input for model optimization.

[0133] 2. Multimodal feedback and model optimization: MMLLM performs adaptive optimization based on feedback data and improves prediction accuracy through incremental learning or model updates. For example, if the system finds that some tasks are not allocated enough resources when under high load, the feedback module will record this situation and use it as a reference for adjusting model parameters. MMLLM continuously updates parameters through this feedback process, making future resource allocation strategies more in line with real-time needs.

[0134] 3. Adaptive learning: With the support of multimodal feedback data, MMLLM achieves precise resource allocation through adaptive learning. Unlike traditional methods, the system can dynamically update model parameters without frequent retraining, thereby reducing model maintenance costs and improving the long-term adaptability of resource allocation.

[0135] In a multi-tasking environment, differences in task priorities may lead to competition for resource allocation. This system uses the multimodal reasoning capabilities of MMLLM to automatically identify task priorities and resource requirements, achieve fair resource allocation, and ensure that the resource requirements of high-priority tasks are met while taking into account the reasonable allocation of low-priority tasks:

[0136] A1. Task priority identification: MMLLM analyzes task priorities through multimodal data. For example, the system can identify the urgency of tasks and resource consumption patterns based on log information and monitoring data, thereby determining the priorities of different tasks.

[0137] A2. Fairness guarantee: Based on task priority, the system adjusts resource quotas through the allocation strategy generated by MLLLM to ensure that low-priority tasks can still obtain basic resources while the needs of high-priority tasks are met. For example, in a cloud computing environment, MLLLM can allocate more CPU and memory resources to critical tasks based on priority, while limiting the resource upper limit of low-priority tasks to prevent excessive resource occupation.

[0138] A3. Resource scheduling in a multi-task environment: In a multi-task concurrent environment, this system uses MLLLM to analyze the load characteristics and resource requirements of different tasks to achieve balanced resource scheduling and dynamic adaptation. When the number of tasks increases suddenly, the system can automatically determine the allocation priority and isolation level of resources to avoid system performance degradation caused by resource competition.

[0139] The multimodal resource management system of this system adopts a modular architecture and can be flexibly expanded and adjusted in different application scenarios according to needs. The data acquisition, feature extraction, MMLLM reasoning, resource management and feedback optimization modules in the system are independent of each other, and through a unified data interface and standardized API connection, it provides a variety of resource management solutions for the following scenarios:

[0140] B1. Cloud computing and big data processing: In cloud computing and big data platforms, this system can dynamically expand or release computing resources according to real-time load changes to ensure that the system maintains stable performance during peak loads.

[0141] B2. Edge computing and IoT: In edge computing and IoT scenarios, the system can dynamically monitor the status and load demand of each node and allocate resources based on node priority, thereby supporting low-latency and high-frequency data transmission applications.

[0142] B3. Smart Manufacturing and Industrial Control: In a smart manufacturing environment, the system can adjust resource allocation in real time according to production tasks and equipment status to ensure efficient and stable production processes. For example, when production tasks increase or equipment fails, the system can dispatch computing resources in real time to optimize production rhythm.

[0143] B4. Financial transactions and real-time services: In financial and real-time transaction environments, this system can allocate computing resources in real time according to market fluctuations or traffic surges, improve data processing speed, and ensure the smoothness of high-frequency transactions and user services.

[0144] The system's multimodal data fusion and adaptive resource management system based on MMLLM significantly improves the accuracy, response speed and system adaptability of resource allocation through multimodal feature fusion and dynamic reasoning of real-time data. This system is particularly suitable for resource management needs in highly dynamic load environments, providing an intelligent, flexible and efficient solution for modern computing resource management.

[0145] The system extracts multidimensional features from multimodal data sources (text, monitoring indicators, visual data, etc.) and generates a feature vector X(t). The feature vector is defined as follows:

[0146] X(t)=[X text (t),X metrics (t),X visual (t)]

[0147] Where: X text (t) represents the context information extracted from the text log (e.g., task status, abnormal conditions, load peaks). metrics (t) represents the numerical features extracted from system monitoring data (CPU, memory, I / O, network traffic, etc.), reflecting the dynamic situation of system resource usage. visual (t) represents the features extracted from visual data (device status images, etc.) to identify hardware failures and device status changes.

[0148] The feature vector X(t) combines data from multiple modalities to form a multi-dimensional description of the system state. This comprehensive feature vector enables the multimodal large language model (MMLLM) to fully understand resource requirements and system states with the support of information from different modalities.

[0149] MMLLM generates a resource allocation strategy A(t) based on the input multimodal feature vector X(t), which can be described as:

[0150] A(t)=MMLLLM(X(t);θ)

[0151] Where: A(t) = [R1(t), R2(t), …, R N (t)] represents the resource quota allocated to different tasks. i (t) represents the amount of resources allocated to task i at time t. MMLLM(X(t); θ) represents the reasoning process of the multimodal large language model, with X(t) as the input feature and θ as the model parameter.

[0152] The MMLLM model understands the load demand and task characteristics based on data from different modes and generates a resource allocation strategy A(t) to ensure that the system can achieve dynamic resource allocation when the load fluctuates and the task demand changes.

[0153] The system performs resource isolation and dynamic quota management on tasks according to the resource allocation strategy A(t) to ensure the stability and fairness of the system under high load conditions. The resource isolation constraints and dynamic quotas are defined as follows:

[0154]

[0155] Where: C t is the upper limit of the total amount of resources of the system at time t. i (t)≥0 represents the non-negativity constraint of resources for each task i, ensuring that the resources obtained by each task are reasonable values.

[0156] This constraint ensures that the total amount of system resources does not exceed the upper limit C t And through dynamic quotas, resource isolation between tasks is achieved to avoid resource competition that affects system stability.

[0157] The feedback closed-loop mechanism of this system optimizes the prediction accuracy of MMLLM through actual feedback data. The feedback data collection is defined as follows:

[0158] F(t)=[f completion (t),f utilization (t),f response (t)]

[0159] Where: f completion (t) represents the task completion time. utilization (t) represents resource utilization. response (t) represents the system response time.

[0160] The feedback data F(t) is used to adjust the parameter θ of MMLLM. The update process is as follows:

[0161]

[0162] Where: η is the learning rate. L(θ t ; F(t)) is a loss function based on feedback data, which is used to measure the effect of resource allocation strategy; The loss function is expressed as the gradient (partial derivative) of the parameter θ, which is used to guide the direction and amplitude of parameter update and optimize model performance.

[0163] The feedback data F(t) reflects the actual effect of the allocation strategy. The loss function L is used to quantify the deviation of the allocation. The model parameters θ are optimized by gradient descent to achieve adaptive optimization of the model.

[0164] In a multi-task concurrent environment, MMLLM uses the multimodal feature vector X(t) to predict the priority weight u of the task i (t) and resource requirements R i (t) Based on this, the system generates a resource allocation strategy to ensure that the resource requirements of high-priority tasks are met first, while taking into account the basic resource allocation of low-priority tasks. Therefore, MMLLM training is prohibited through the objective function, and the expression of the objective function is as follows:

[0165]

[0166] Where: u i (t) = MMLLM priority (X i (t)) represents the MLLLM based on the feature vector X i (t) The calculated priority weight of task i at time t (i.e., the multimodal large language model MMLLM in the embodiment of the present invention based on the historical feature vector X i (t) The calculated priority weight of computing task i at time t). i (t) = MMLLM resource (X i (t)) represents the resource requirement of task i based on feature vector prediction by MLLLM (that is, the multimodal large language model MLLLM in the embodiment of the present invention is based on the historical feature vector X i (t) The predicted resource requirements of computing task i). N is the total number of computing tasks.

[0167] See also Figure 4 As shown, Figure 4A schematic diagram of the execution results of the resource allocation strategy for the intelligent computing power scheduling-dynamic isolation and quota management system for computing resources. Specifically, it shows the initial allocation strategy, the managed allocation strategy, and the final optimized allocation strategy, showing how the system dynamically adjusts and optimizes resource allocation to meet the needs in a multi-tasking environment.

[0168] Figure 4 In the graph, the horizontal axis (Task Index) indicates the number of tasks, from 0 to 100, showing the resource allocation of the system on multiple tasks. The vertical axis (Allocated Resources) indicates the amount of computing resources allocated to each task. Resource allocation is a simulated continuous variable that reflects the system's estimate of the resource requirements and priority of each task.

[0169] Figure 4 The meaning of the curves of each allocation strategy in :

[0170] Initial Allocation Strategy: The blue dashed line ("--") in the figure represents the resources allocated to each task in the initial stage of the system based on a simple estimate of resource requirements. This stage may be based on a simple analysis of multimodal data (such as text logs, monitoring data, and visual data), but without sophisticated resource management, so the curve fluctuates greatly and lacks uniformity.

[0171] Managed Allocation Strategy: The orange dotted line (":") in the figure represents the allocation strategy after resource isolation and quota management. In this patent, dynamic quota management is used to ensure that the total amount of allocated resources does not exceed the system upper limit. At this stage, the system controls the total resource allocation within a reasonable range through normalization, thereby avoiding the unfairness caused by resource competition between tasks. Compared with the initial allocation, the allocation curve after management is more balanced, but there are still some fluctuations, indicating that the system is still considering the priority of tasks.

[0172] Optimized Allocation Strategy: The green solid line ("-") in the figure represents the final optimized allocation strategy. Based on the management strategy, this strategy uses the feedback and adaptive optimization module to further adjust resource allocation so that the needs of high-priority tasks and dynamic loads can be more accurately met. Through the feedback closed-loop mechanism, the system obtains the actual resource requirements in real time and updates the allocation strategy, gradually improving the accuracy of allocation. The curve at this stage is smooth and fits the actual needs, indicating that the system has achieved the optimal state of resource allocation in a dynamic environment with multiple concurrent tasks.

[0173] The embodiment of the present invention provides a computing power management device for an intelligent computing center, see Figure 5 As shown, Figure 5 This is a principle block diagram of a computing power management device of an intelligent computing center according to an embodiment of the present invention. The computing power management device 50 of the intelligent computing center includes:

[0174] The acquisition module 51 is used to acquire the multimodal operation data of the multimodal computing task running on the intelligent computing center at the current moment, extract the context features of each modal operation data from the multimodal operation data, and combine all the context features to obtain a feature vector; the multimodal operation data includes the modal operation data of at least one of the following modalities: text, monitoring indicators, and visual data;

[0175] An execution module 52 is used to input the feature vector into a multimodal resource allocation large language model deployed in the intelligent computing center to obtain a multimodal resource allocation strategy for allocating computing power resources to the multimodal computing task;

[0176] The execution module 52 is further used to instruct the intelligent computing center to execute the resource allocation strategy.

[0177] In some embodiments of the present invention, optionally, the intelligent computing center is instructed to execute the resource allocation strategy under quota constraints;

[0178] The quota constraint condition is expressed as follows:

[0179]

[0180] Among them, C t R is the upper limit of the total amount of resources of the intelligent computing center at time t; i (t) is the computing power resource of computing task i at time t, and R i (t)≥0, which is the resource non-negativity constraint of each computing task i; N is the total number of computing tasks.

[0181] The computing power management device of the intelligent computing center provided in the embodiment of the present application can achieve Figure 1 The various processes implemented by the method embodiment and achieving the same technical effect are not described here to avoid repetition.

[0182] The embodiment of the present invention provides a training device for a large language model for multimodal resource allocation, see Figure 6 As shown, Figure 6 This is a principle block diagram of a training device for a large language model for multimodal resource allocation according to an embodiment of the present invention. The training device 60 for a large language model for multimodal resource allocation includes:

[0183] The training data acquisition module 61 is used to acquire historical multimodal operation data of the multimodal computing task on the intelligent computing center, extract historical context features of each modal operation data from the historical multimodal operation data, and combine all the historical context features to obtain a historical feature vector; the historical multimodal operation data includes the modal operation data of at least one of the following modalities: text, monitoring indicators, and visual data;

[0184] The labeling module 62 is used to obtain the historical resource allocation strategy of the intelligent computing center for the multimodal computing task corresponding to the historical multimodal operation data; and label the historical feature vector corresponding to the historical resource allocation strategy to obtain a training feature vector;

[0185] The training module 63 is used to train the multimodal large language model using the training feature vector to obtain the multimodal resource allocation large language model.

[0186] In some embodiments of the present invention, optionally, the objective function of training the multimodal large language model is determined according to the priority weight of the multimodal computing task and the resource requirement of the multimodal computing task.

[0187] In some embodiments of the present invention, optionally, the objective function is expressed as follows:

[0188]

[0189] Among them, u i (t) = MMLLM priority (X i (t)) is a multimodal large language model MMLLM based on the historical feature vector X i (t) The calculated priority weight of computing task i at time t; R i (t) = MMLLM resource (X i (t)) is a multimodal large language model MMLLM based on the historical feature vector X i (t) The predicted resource requirements of computing task i; N is the total number of computing tasks.

[0190] The computing power management device of the intelligent computing center provided in the embodiment of the present application can achieve Figure 2 The various processes implemented by the method embodiment and achieving the same technical effect are not described here to avoid repetition.

[0191] An embodiment of the present invention provides an electronic device 70, see Figure 7 As shown, Figure 7This is a principle block diagram of an electronic device 70 according to an embodiment of the present invention, including a processor 71, a memory 72, and a program or instruction stored in the memory 72 and executable on the processor 71. When the program or instruction is executed by the processor, it implements the steps of any method for managing computing power of an intelligent computing center according to the present invention, or implements the steps of any method for training a large language model for multimodal resource allocation according to the present invention.

[0192] An embodiment of the present invention provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of an embodiment of a computing power management method for an intelligent computing center such as any of the above-mentioned items are implemented, or the various processes of an embodiment of a training method for a large language model for multimodal resource allocation such as any of the above-mentioned items are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0193] The readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc. In some examples, the readable storage medium may be a non-transitory readable storage medium.

[0194] An embodiment of the present invention also provides a computer program product, including computer instructions, which, when executed by a processor, implement the various processes of any of the above-mentioned embodiments of the computing power management method for an intelligent computing center, or implement the various processes of any of the above-mentioned embodiments of the training method for a large language model for multimodal resource allocation, and can achieve the same technical effect. To avoid repetition, they are not repeated here.

[0195] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. A computing power management method for an intelligent computing center, characterized in that: include: Step S1: obtaining multimodal operation data of a multimodal computing task running on an intelligent computing center at a current moment, extracting context features of each modal operation data from the multimodal operation data, and combining all the context features to obtain a feature vector; The multimodal operation data includes the modal operation data of at least one of the following modes: text, monitoring indicators, and visual data; Step S2: inputting the feature vector into a multimodal resource allocation large language model deployed in the intelligent computing center to obtain a multimodal resource allocation strategy for allocating computing resources to the multimodal computing task; Step S3: instructing the intelligent computing center to execute the resource allocation strategy.

2. The computing power management method of the intelligent computing center according to claim 1, characterized in that: The step S3 comprises: Instructing the intelligent computing center to execute the resource allocation strategy under quota constraints; The quota constraint condition is expressed as follows: Among them, C t R is the upper limit of the total amount of resources of the intelligent computing center at time t; i (t) is the computing power resource of computing task i at time t, and R i (t)≥0, which is the resource non-negativity constraint of each computing task i; N is the total number of computing tasks.

3. A method for training a large language model for multimodal resource allocation, characterized in that: include: Step S01: acquiring historical multimodal operation data of a multimodal computing task on an intelligent computing center, extracting historical context features of each modal operation data from the historical multimodal operation data, and combining all the historical context features to obtain a historical feature vector; the historical multimodal operation data includes the modal operation data of at least one of the following modalities: text, monitoring indicators, and visual data; Step S02: obtaining a historical resource allocation strategy for the multimodal computing task executed by the intelligent computing center corresponding to the historical multimodal operation data; and labeling the historical feature vector with the historical resource allocation strategy to obtain a training feature vector; Step S03: using the training feature vector to train a multimodal large language model to obtain a multimodal resource allocation large language model.

4. The method for training a large language model for multimodal resource allocation according to claim 3, characterized in that: The objective function of training the multimodal large language model is determined according to the priority weight of the multimodal computing task and the resource requirement of the multimodal computing task.

5. The method for training a large language model for multimodal resource allocation according to claim 4, characterized in that: The expression of the objective function is as follows: Among them, u i (t) = MMLLM priority (X i (t)) is a multimodal large language model MMLLM based on the historical feature vector X i (t) The calculated priority weight of computing task i at time t; R i (t) = MMLLM resource (X i (t)) is a multimodal large language model MMLLM based on the historical feature vector X i (t) The predicted resource requirements of computing task i; N is the total number of computing tasks.

6. A computing power management device for an intelligent computing center, characterized in that: include: An acquisition module, used to acquire multimodal operation data of a multimodal computing task running on an intelligent computing center at a current moment, extract context features of each modal operation data from the multimodal operation data, and combine all the context features to obtain a feature vector; The multimodal operation data includes the modal operation data of at least one of the following modes: text, monitoring indicators, and visual data; An execution module, configured to input the feature vector into a multimodal resource allocation large language model deployed in the intelligent computing center, and obtain a multimodal resource allocation strategy for allocating computing power resources to the multimodal computing task; The execution module is also used to instruct the intelligent computing center to execute the resource allocation strategy.

7. A training device for a multimodal resource allocation large language model, characterized in that: include: A training data acquisition module, used to acquire historical multimodal operation data of a multimodal computing task on an intelligent computing center, extract historical context features of each modal operation data from the historical multimodal operation data, and combine all the historical context features to obtain a historical feature vector; the historical multimodal operation data includes the modal operation data of at least one of the following modalities: text, monitoring indicators, and visual data; A labeling module, used for obtaining a historical resource allocation strategy for the multimodal computing task executed by the intelligent computing center corresponding to the historical multimodal operation data; and labeling the historical feature vector corresponding to the historical resource allocation strategy to obtain a training feature vector; The training module is used to train the multimodal large language model using the training feature vector to obtain the multimodal resource allocation large language model.

8. An electronic device, characterized in that: It includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein when the program or instruction is executed by the processor, it implements the steps in the method for managing computing power of an intelligent computing center as described in any one of claims 1 to 2, or implements the steps in the method for training a large language model for multimodal resource allocation as described in any one of claims 3 to 5.

9. A readable storage medium, characterized in that: The readable storage medium stores programs or instructions, and when the programs or instructions are executed by the processor, they implement the steps in the computing power management method of the intelligent computing center as described in any one of claims 1 to 2, or implement the steps in the training method of the multimodal resource allocation large language model as described in any one of claims 3 to 5.

10. A computer program product, characterized in that It includes computer instructions, which, when executed by a processor, implement the steps in the computing power management method of the intelligent computing center as described in any one of claims 1 to 2, or implement the steps in the training method of the multimodal resource allocation large language model as described in any one of claims 3 to 5.

Images