Resource management method, device, apparatus, storage medium, and computer program product

Abstract

The application discloses a resource management method, device, equipment, storage medium and computer program product, to solve the problem that the existing GPU operation scheme relies on the traditional cloud management platform and artificial experience, and cannot schedule GPU resources according to real-time business load in actual production, leading to the disconnection between resource planning and actual demand, and easily causing resource waste. The method comprises the following steps: constructing a digital twin corresponding to a GPU server according to collected GPU server operation data; generating a resource configuration model according to the GPU server operation data recorded in the digital twin; predicting a received service request according to the resource configuration model, and determining GPU resource demand corresponding to the service request; generating an initial resource allocation strategy corresponding to the service request according to the GPU resource demand; simulating and checking the initial resource allocation strategy according to the digital twin, obtaining a simulation result, and generating a resource allocation strategy according to the simulation result.

Description

Technical Field

[0001] This application relates to the field of wireless communication technology, and in particular to a GPU resource management method, apparatus, device, storage medium, and computer program product based on digital twins. Background Technology

[0002] With the rapid development of technologies such as artificial intelligence, big data analytics, and deep learning, graphics processing units (GPUs) have become core hardware resources supporting various computationally intensive applications due to their powerful parallel computing capabilities. Especially in scenarios such as large-scale model training, scientific computing, and real-time graphics processing, the effective operation and management of GPU resources directly affects computing efficiency, cost control, and service quality.

[0003] Currently, the operation and management of GPU resources mainly rely on traditional cloud management platforms and human experience. This makes it impossible to schedule GPU resources based on the real-time business load in actual production, resulting in a disconnect between resource planning and actual needs, which can easily lead to resource waste and affect business processing efficiency.

[0004] In recent years, digital twin technology has been widely used in industries and urban management due to its advantages in real-time mapping between physical entities and digital models, simulation, and intelligent decision-making. However, in the field of GPU resource operation, a GPU resource management solution based on digital twins has not yet been developed.

[0005] Therefore, how to achieve intelligent operation of GPU resources based on digital twins has become an urgent technical problem to be solved. Summary of the Invention

[0006] This application provides a GPU resource management method based on digital twins to address the problem that existing GPU operation solutions rely on traditional cloud management platforms and human experience, which cannot schedule GPU resources according to the real-time business load in actual production. This leads to a disconnect between resource planning and actual needs, which can easily cause resource waste and affect business processing efficiency.

[0007] This application also provides a GPU resource management device based on digital twins to solve the problem that existing GPU operation solutions rely on traditional cloud management platforms and human experience, which cannot schedule GPU resources according to the real-time business load in actual production, resulting in a disconnect between resource planning and actual needs, which can easily lead to resource waste and affect business processing efficiency.

[0008] This application also provides a GPU resource management device based on digital twins to solve the problem that existing GPU operation solutions rely on traditional cloud management platforms and human experience, which cannot schedule GPU resources according to the real-time business load in actual production, resulting in a disconnect between resource planning and actual needs, which can easily lead to resource waste and affect business processing efficiency.

[0009] This application also provides a computer-readable storage medium to address the problem that existing GPU operation solutions rely on traditional cloud management platforms and human experience, which cannot schedule GPU resources according to the real-time business load in actual production. This leads to a disconnect between resource planning and actual needs, which can easily cause resource waste and affect business processing efficiency.

[0010] A computer program product is designed to address the problem that existing GPU operation solutions rely on traditional cloud management platforms and human experience, which cannot schedule GPU resources based on real-time business loads in actual production. This leads to a disconnect between resource planning and actual needs, easily resulting in resource waste and affecting business processing efficiency.

[0011] The embodiments of this application adopt the following technical solutions: A GPU resource management method based on digital twins includes: constructing a digital twin corresponding to the GPU server based on collected GPU server operation data, wherein the digital twin is used to map and display the physical GPU hardware entity corresponding to the GPU server and the operating status of the services it carries; generating a resource configuration model based on the GPU server operation data recorded in the digital twin; predicting received service requests based on the resource configuration model to determine the GPU resource requirements corresponding to the service requests; generating an initial resource allocation strategy corresponding to the service requests based on the GPU resource requirements; simulating and verifying the initial resource allocation strategy based on the digital twin to obtain simulation results; generating a resource allocation strategy based on the simulation results; and sending the resource allocation strategy to the GPU server for execution.

[0012] A GPU resource management device based on digital twins includes: an economic data acquisition unit, used to spatially align collected economic data according to a preset geographic grid to determine the economic data corresponding to each geographic grid; a noise acquisition unit, used to acquire noise data corresponding to each geographic grid; an association unit, used to determine the association topology between the geographic grids based on the noise data; an association matrix generation unit, used to generate an association matrix for determining the correlation between noise data and economic data based on the temporal changes of the economic data corresponding to each geographic grid and the association topology; and a policy generation unit, used to generate a management policy for each geographic grid based on the association matrix.

[0013] A GPU resource management device based on digital twins, comprising: The processor; and a memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the following operations: constructing a digital twin corresponding to the GPU server based on collected GPU server operation data, wherein the digital twin is used to map and display the physical GPU hardware entity corresponding to the GPU server and the operating status of the services it carries; generating a resource configuration model based on the GPU server operation data recorded in the digital twin; predicting received service requests based on the resource configuration model to determine the GPU resource requirements corresponding to the service requests; generating an initial resource allocation strategy corresponding to the service requests based on the GPU resource requirements; simulating and verifying the initial resource allocation strategy based on the digital twin to obtain simulation results; generating a resource allocation strategy based on the simulation results; and sending the resource allocation strategy to the GPU server for execution.

[0014] A computer-readable storage medium stores one or more programs that, when executed by an electronic device including multiple applications, cause the electronic device to perform the following operations: constructing a digital twin corresponding to a GPU server based on collected GPU server operation data, wherein the digital twin is used to map and display the physical GPU hardware entity corresponding to the GPU server and the operating status of the services it carries; generating a resource configuration model based on the GPU server operation data recorded in the digital twin; predicting received service requests based on the resource configuration model to determine the GPU resource requirements corresponding to the service requests; generating an initial resource allocation strategy corresponding to the service requests based on the GPU resource requirements; simulating and verifying the initial resource allocation strategy based on the digital twin to obtain simulation results; generating a resource allocation strategy based on the simulation results; and sending the resource allocation strategy to the GPU server for execution.

[0015] A computer program product includes a computer program that, when executed by a processor, implements the following: constructing a digital twin corresponding to a GPU server based on collected GPU server operation data, wherein the digital twin is used to map and display the physical GPU hardware entity corresponding to the GPU server and the operating status of the services it carries; generating a resource configuration model based on the GPU server operation data recorded in the digital twin; predicting received service requests based on the resource configuration model to determine the GPU resource requirements corresponding to the service requests; generating an initial resource allocation strategy corresponding to the service requests based on the GPU resource requirements; simulating and verifying the initial resource allocation strategy based on the digital twin to obtain simulation results; generating a resource allocation strategy based on the simulation results; and distributing the resource allocation strategy to the GPU server for execution.

[0016] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects: The GPU resource management method based on digital twins provided in this application involves the following steps: First, a digital twin corresponding to the GPU server is constructed based on the collected GPU server operation data. Then, the physical GPU hardware entity and the running status of the services carried by the GPU server are mapped and displayed through the digital twin. Based on the GPU server operation data recorded in the digital twin, a resource configuration model is generated. Based on the resource configuration model, the received service requests are predicted to determine the GPU resource requirements corresponding to the service requests. Then, an initial resource allocation strategy corresponding to the service requests is generated based on the GPU resource requirements. The initial resource allocation strategy is simulated and verified through the digital twin to obtain simulation results. Then, the initial resource allocation strategy is adjusted based on the simulation results to generate a new resource allocation strategy, which is then sent to the GPU server for execution. The GPU resource management method based on digital twins provided in this application has two advantages. First, it can generate a resource configuration model based on historical and real-time operational data accumulated by the digital twin. This resource configuration model can associate specific business scenario characteristics with multi-dimensional hardware indicators of the GPU. By using this resource configuration model to predict resource requirements for new business requests, the prediction results can be more closely aligned with the actual load of the production environment, thus providing accurate data for GPU resource allocation and significantly improving the scientific nature and accuracy of resource planning. Second, by constructing a digital twin corresponding to the GPU server, the complex GPU hardware status, business operation logic, and the relationship between the two can be visualized in an intuitive graphical way. This allows operators to use the digital twin to determine the overall health of each resource pool in the GPU server, the resource consumption trajectory of a single business, and potential performance bottlenecks in real time, greatly improving operational efficiency and problem location speed. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A schematic diagram illustrating the specific structure of a GPU resource management system based on digital twins, provided in this application embodiment. Figure 2 A schematic diagram illustrating the specific process of a GPU resource management method based on digital twins provided in this application embodiment; Figure 3 A business classification diagram provided for an embodiment of this application; Figure 4 A schematic diagram of a digital twin instance type provided in an embodiment of this application; Figure 5 This application provides a schematic diagram of the specific structure of a GPU resource management device based on digital twins; Figure 6 This is a schematic diagram illustrating the specific structure of a GPU resource management device based on digital twins, provided as an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] This application provides a GPU resource management method based on digital twins to address the problem that existing GPU operation solutions rely on traditional cloud management platforms and human experience, which cannot schedule GPU resources according to the real-time business load in actual production. This leads to a disconnect between resource planning and actual needs, which can easily cause resource waste and affect business processing efficiency.

[0020] The execution subject of the GPU resource management method based on digital twin provided in this application embodiment may be, but is not limited to, at least one of a resource management server, an operation and maintenance server, and a GPU resource scheduling server; in addition, the execution subject of the method may also be the system or application (APP) itself running on these servers.

[0021] For ease of description, the following description uses a resource management system as the execution subject to illustrate the implementation of this method. It should be understood that using a resource management system as the execution subject is merely an illustrative example and should not be construed as a limitation of the method.

[0022] In one implementation, the specific structure of the resource management system provided in this application embodiment can be as follows: Figure 1 As shown, it mainly includes: application layer, twin layer and physical layer.

[0023] The application layer is mainly used to provide a visualization module, which presents the status of the digital twin, model analysis results, and simulation process to users in a visual way. At the same time, it provides a management interface for operators to interact and make decisions.

[0024] The twin layer mainly includes the following parts: 1. Data Acquisition and Processing Layer: This layer is mainly used to collect raw data from the physical layer and then clean, integrate, and store it.

[0025] 2. Digital Twin Modeling Layer: This layer is mainly used to create and maintain various digital twin instances that correspond to physical entities.

[0026] 3. Intelligent analysis and optimization decision layer, mainly used to train and update resource allocation models using digital twin data, and to perform simulation verification and intelligent decision-making on resource allocation schemes based on resource allocation models and digital twins.

[0027] The physical layer includes the actual GPU servers, GPU cards, network devices, and the various applications running on them.

[0028] Based on the above Figure 1 The resource management system shown is illustrated in the schematic diagram of the specific implementation process of the GPU resource management method based on digital twin provided in this application. Figure 2 As shown, the main steps include the following: Step 11: Based on the collected GPU server operation data, construct a digital twin corresponding to the GPU server; In this embodiment of the application, the GPU server operation data collected by the resource management system may include, but is not limited to, the following two categories: Type 1, Hardware Operation Data: Specifically, the resource management system can obtain GPU hardware operation data through the monitoring interface provided by the GPU driver, the server baseboard management controller (BMC), or operating system monitoring tools.

[0029] In one implementation method, the collected hardware operation data includes, but is not limited to, the following: 1. GPU Model Parameters: This includes information such as the GPU manufacturer, model, release performance FLOPS value, configuration, and serial number, which are used to present its product specifications, performance, uniqueness, and other parameters and characteristics.

[0030] 2. GPU utilization: A GPU chip typically contains multiple core units (ALUs). FLOPS (Floating-point Operations Per Second) is a metric used to measure GPU computing performance, representing the number of floating-point operations that can be performed per second. GPU utilization indicates the overall usage of the GPU.

[0031] 3. GPU memory utilization: GPU memory is mainly used to store graphics data and AI model data, including images, videos, 3D models, AI model parameters, etc. Memory utilization indicates the usage of memory.

[0032] 4. GPU memory bandwidth utilization: GPU memory bandwidth reflects the data transfer rate between the GPU chip and the memory. GPU memory bandwidth utilization represents the ratio of the data transfer rate to the upper limit of memory bandwidth.

[0033] 5. Inter-GPU Interconnect Bandwidth Utilization: Inter-GPU bandwidth refers to the bandwidth used for inter-GPU communication in a multi-GPU system (multiple GPU cards in the same GPU server, or a cluster of multiple GPU cards in multiple GPU servers). Inter-GPU interconnect bandwidth utilization indicates the bandwidth utilization status.

[0034] Type 2, Business Execution Data: Specifically, the resource management system can obtain business execution data by collecting business logs from the business scheduling platform, job management system, or corresponding applications. This includes at least: In one implementation method, the collected business execution data may include, but is not limited to, the following: 1. Business type scenarios: In the embodiments of this application, it is possible to construct as follows: Figure 3 The business classification system shown distinguishes between business types and scenarios. For example, Level 1 includes "Artificial Intelligence Training," "Big Data Analysis," and "Graphics Rendering"; Level 2 further subdivides "Artificial Intelligence Training" into "Natural Language Processing (NLP)," "Computer Vision (CV)," and "Scientific Computing," etc.; Level 3 can be further refined, such as "BERT Model Training" and "GPT Model Inference" under NLP. The more detailed the classification, the more accurate the subsequent model will be, and business types and scenarios can be added as needed to improve the accuracy of the calculations.

[0035] 2. Business scale: This can refer to the size of the dataset of business data processed by the GPU, in units of GB, TB or PB.

[0036] 3. Business processing time: refers to the total time spent by the GPU to complete a certain business type scenario and business scale, in hours.

[0037] 4. GPU computing resources corresponding to the business: A list of GPU resources actually allocated and used by the task, including GPU card ID, allocated video memory size, etc.

[0038] In this embodiment, the resource management system can preprocess the collected raw GPU server operation data, including noise removal, missing value filling, data format standardization, and data normalization, to improve data quality. For the preprocessed data, hardware and business data related to the same task at the same time can be associated and integrated, and then stored in a time-series database to form a "business-hardware" associated data sequence that can be queried and analyzed.

[0039] Next, the resource management system can use the collected and processed GPU server operation data to construct a multi-granularity, multi-perspective digital twin by employing digital twin technology, and ensure that the state of the digital twin remains consistent with that of the physical entity through continuous data flow.

[0040] In this embodiment of the application, the resource management system can construct a digital twin according to the following sub-steps, including: Sub-step 1101: Construct a digital twin instance; Specifically, the resource management system can flexibly combine various GPU hardware and business digital twin instances according to actual operational and granular needs, creating them at different dimensions such as single cluster, single GPU server, single GPU card, single GPU slice, and specific business scenario. The types of digital twin instances constructed include... Figure 4 As shown, it includes: Physical resource dimension examples: Specifically, these can include single GPU slice twins, single GPU card twins, single server twins, and cluster twins.

[0041] Example from a logical business dimension: For a specific running AI training task, create a task twin that aggregates the states of all GPU hardware used in that task.

[0042] Sub-step 1102: Extract GPU server operation data.

[0043] Specifically, the extracted GPU server operation data can be real-time data or historical data.

[0044] Sub-step 1103: Construct a digital twin of GPU hardware and business applications.

[0045] By extracting real-time GPU server operation data and historical GPU server operation data through sub-step 1102, a digital twin of the physical entity's GPU hardware and business is constructed based on digital twin technology.

[0046] In this embodiment, the constructed digital twin can be divided into two main categories: real-time state digital twin and historical state digital twin. The real-time state digital twin is used to map the current running state of the GPU physical entity; the historical state digital twin is used to map the past running states of the GPU physical entity.

[0047] Sub-step 1104: Display the generated digital twin on the display module.

[0048] Visualization technology transforms complex data and digital twins into intuitive graphics and images, which are then presented visually through display modules to facilitate understanding and interaction by operations personnel.

[0049] Step 12: Generate a resource configuration model based on the GPU server operation data recorded in the digital twin obtained by performing Step 11; In this embodiment of the application, the resource management system can generate a resource configuration model according to the following sub-steps, including: Sub-step 1201: Determine the hardware operation data corresponding to each service type recorded in the digital twin; In this application embodiment, a digital twin corresponding to a specific business can be used as an example. The resource management system can determine the hardware configuration and number of nodes of the digital twin.

[0050] Specifically, the hardware configuration parameters recorded by the resource management system may include, but are not limited to, the following: GPU model, GPU product release FLOPS value (or FLOPS value allocated to a slice), allocated video memory capacity, allocated video memory bandwidth, and allocated inter-card interconnect bandwidth. In this embodiment, the number of nodes for a digital twin corresponding to a single GPU is 1. For multiple GPU slices, a GPU server containing multiple GPU cards, or a GPU cluster composed of multiple GPU servers, the number of nodes is N, where N is the specific number of GPU cards.

[0051] Sub-step 1202: Based on the hardware operation data obtained by executing sub-step 1201, determine the comprehensive utilization rate corresponding to each business type recorded in the digital twin; The computing efficiency is used to represent the scale of business that can be processed per unit of GPU computing performance for the given business type.

[0052] It's important to note that in current technology, each device manufacturer releases the performance of its GPU products, measured in FLOPS (Floating-Point Operations Per Second). FLOPS = Number of Cores × Clock Speed ​​× Floating-Point Operations Per Core per Clock Cycle. Therefore, GPU computing performance is affected by the number of ALUs (Analog Units) and the clock speed. To achieve the advertised performance of a GPU product, its video memory, memory bandwidth, and inter-card interconnect bandwidth must not reach bottlenecks.

[0053] However, in actual business deployment and operation, depending on different business scenarios, some may be memory-intensive, some may be card-to-card interconnection intensive, and some may be computationally and memory-intensive. Often, one or more of the following—memory, memory bandwidth, and card-to-card interconnect bandwidth—will be fully utilized, while computational performance remains surplus, or vice versa. If hardware resource configuration does not match the business scenario, or if a certain GPU hardware indicator shows a significant discrepancy, the overall GPU utilization rate is highly likely to be low, resulting in resource waste. Therefore, in this embodiment, the overall utilization rate of the GPU can be determined based on the aforementioned multi-dimensional parameters such as GPU memory, memory bandwidth, and card-to-card interconnect bandwidth, and subsequent GPU resource configuration can be based on this overall utilization rate.

[0054] In this embodiment, the overall utilization rate, compared with the traditional GPU utilization rate, incorporates factors such as video memory, video memory bandwidth, and inter-card interconnect bandwidth, thus providing a more comprehensive reflection of GPU usage.

[0055] In this embodiment of the application, the overall utilization rate of a single GPU can be determined as follows: (a) Determine the overall utilization rate of a single GPU: In this embodiment, the overall utilization rate of a single GPU can be calculated according to the following formula [1]: [1] Among them, U ALU This indicates the utilization rate of the GPU core unit ALU, reflecting the usage of computing units. For a GPU slice, it represents the utilization rate of the GPU core unit ALU allocated to that slice.

[0056] U Mem This indicates GPU memory utilization, reflecting the amount of GPU memory used. For GPU slices, it indicates the utilization rate of GPU memory allocated to the slice.

[0057] U MemBW This represents the GPU memory bandwidth utilization rate, reflecting the ratio of data transfer rate to the maximum memory bandwidth. For a GPU slice, it represents the utilization rate of the GPU memory bandwidth allocated to that slice.

[0058] U Interconnect This indicates the utilization rate of GPU interconnect bandwidth, reflecting the bandwidth used for communication in a multi-GPU system. For GPU slices, it represents the utilization rate of the GPU interconnect bandwidth allocated to the slice.

[0059] ω ALU ω Mem ω MemBW ωInterconnect These are weighting factors determined based on the nature of the task and hardware characteristics, and their sum should be 1 or close to 1. These weights can be rationally allocated through learning, training, and optimization using historical data from digital twins of similar business types. For computationally intensive tasks, ω... ALU The ω value will be relatively large, and for memory-intensive tasks, ω MemBW and ω Interconnect It will account for a large proportion.

[0060] (ii) Determine the overall utilization rate of multiple GPU slices: For a logical cluster consisting of multiple GPU slices, such as a logical cluster consisting of multiple GPU slices from a single GPU card, or a logical cluster consisting of multiple GPU slices from multiple GPU cards, assuming there are N GPU slices, the overall utilization rate of the multiple GPU slices can be calculated according to the following formula [2] in this embodiment: [2] in, This represents the overall utilization rate of the i-th GPU slice. It is calculated using U... Cluster The overall utilization rate of the entire logical cluster can be determined.

[0061] (iii) Determine the overall utilization rate of a single GPU server or a single GPU cluster: For a GPU server consisting of multiple GPU cards, or a cluster consisting of multiple GPU servers, such as a cluster of multiple GPU servers or a cluster consisting of several GPU cards from multiple GPU servers, assuming there are N GPU nodes, in this embodiment, the overall utilization rate U of the entire server or cluster can be obtained by averaging the computing power efficiency of each GPU node according to the following formula [3]. Cluster : [3] in, This represents the overall utilization rate of the i-th GPU. It is calculated using U... Cluster This allows us to obtain the average utilization rate of the entire cluster.

[0062] Sub-step 1203: Calculate the actual computing performance of the GPU; It should be noted that in actual business system operation, the resource management system can use GPU utilization (U) to... ALU By combining the GPU product release FLOPS value and the number of nodes in the node configuration, the actual computing performance of the GPU of a certain business digital twin is calculated.

[0063] For a single GPU, its actual computing performance P can be determined according to the following formula [4].Card : [4] Among them, P Card The unit is FLOPS, which represents the number of floating-point operations per second (P). Publish Indicates product release performance and U ALU This indicates GPU utilization.

[0064] For a single GPU server or a single GPU cluster, assuming there are N GPU nodes, the actual computing performance of the GPU is P. Cluster It can be determined according to the following formula [5]: [5] in, and These represent the product release performance and GPU utilization of the i-th GPU, respectively.

[0065] In GPU slicing scenarios, the GPU utilization rate U for slicing is... Slice ALU Related to the resource slicing allocation method. The actual GPU computing performance (P0) in a GPU slicing scenario. Slice P Slices Performance P needs to be determined based on product release. Publish Slice utilization rate U Slice ALU The resource slice allocation method shall be determined according to the following formulas [6] and [7].

[0066] [6] Where ε represents the proportion of GPU resources allocated to this slice.

[0067] [7] in, Used to represent the product release performance of the GPU associated with the i-th slice. Used to represent the proportion of GPU resources allocated to the i-th slice. Used to represent the GPU utilization of the i-th slice.

[0068] Sub-step 1204: Based on the comprehensive utilization rate and actual computing performance calculated in sub-steps 1202 to 1203 above, determine the computing power efficiency corresponding to each business type recorded in the digital twin. It should be noted that when a GPU runs a specific business scenario, it will generate a large amount of hardware and business operation record data. Based on the collected data, a digital twin of the GPU physical entity is constructed using digital twin technology. This twin can display the GPU entity status according to hardware or business dimensions, including GPU hardware data and business data. The digital twin can easily associate the business scale processed at a certain time now or in the past. To avoid the influence of busy or idle time at a certain moment, the total business scale G completed within a certain period (T) can be obtained from the digital twin, and the business processing rate B can be calculated according to the following formula [8]. (Volume / S) The unit can be expressed as MB / s, which represents the volume of business processed per second.

[0069] [8] Business processing rate B (Volume / S) Compared with the actual computing performance of GPU P (P Card P Cluster P Slice P Slices Correspondingly, the business volume that can be completed in each floating-point operation can be calculated according to the following formula [9], that is, the computing power efficiency, in MB / operation: [9] Sub-step 1205: Based on the computing efficiency determined by executing the above sub-steps, construct a predictive model to represent the correspondence between business scale, business processing time and required GPU computing performance, as the resource configuration model.

[0070] Using computing efficiency η, given any two of the following three conditions—total business volume, business processing time, and the actual GPU computing performance corresponding to the business—the demand for the third factor can be predicted using the following methods: Solution (1): Predict the GPU computing performance requirements of the business: After clarifying the total business scale G and the business processing time T requirements, the GPU computing performance P can be predicted according to the following formula

[10] . future Requirements:

[10] Option (2), predicting business processing time requirements: With the total business scale G and the planned GPU computing performance P clearly defined... future Then, the required business processing time T can be predicted according to the following formula

[11] :

[11] It should be noted that the GPU computing performance requirements mentioned above are rigid requirements, meaning no additional redundancy is considered. In actual engineering calculations, considering the high availability of the system, the resource allocation model will take into account the system's own situation and consider a certain redundancy coefficient K to ensure that GPU utilization operates within a reasonable range and that there is a certain switching capability in the event of partial device failure.

[0071] Step 13: Based on the resource configuration model obtained by executing Step 12, predict the received business requests and determine the GPU resource requirements corresponding to the business requests; When the resource management system receives a new business request, it parses the request parameters and obtains at least two of them: the target business type, the target business scale G', and the target processing time T'.

[0072] If the new business request provides the target business size G' and the target processing time T', the required GPU computing performance can be calculated according to the formula above

[10] as the GPU resource requirement.

[0073] If the target business scale G' and fixed computing power P' are given in the request, the target processing time T' can be calculated according to the above formula

[11] .

[0074] Step 14: Generate the initial resource allocation strategy corresponding to the service request based on the GPU resource requirements determined by executing Step 13; Specifically, the resource management system obtains the real-time status of all GPU resources by querying the digital twin. Then, based on the calculated GPU computing performance requirements or business processing time requirements, and combined with the resource pool status, it generates one or more specific, executable allocation schemes. For example, "assigning tasks to the five A100 GPU cards numbered 1-5 in cluster A" is an initial resource allocation strategy.

[0075] Step 15: Simulate and verify the initial resource allocation strategy based on the digital twin to obtain the simulation results, generate a resource allocation strategy based on the simulation results, and send the resource allocation strategy to the GPU server for execution.

[0076] In this embodiment of the application, the resource management system can simulate and verify the initial resource allocation strategy for the following business scenarios, including: Scenario 1, for business scenarios where delivery is completed based on total business volume and a specific timeframe: The resource management system can import the defined total business scale G and business processing time T into the resource configuration model of the corresponding business scenario to calculate the GPU computing performance P. futureRequirements: Based on the current GPU pool resource allocation across the entire network, utilize digital twin technology to improve GPU computing performance (P). future Demands are mapped to digital twins of corresponding GPU hardware, and simulation experiments are conducted. By predicting and comparing the utilization rate and processing time of various GPU hardware indicators, the GPU computing performance is verified. future To ensure the accuracy of the requirements, if any anomalies are found, the resource configuration model parameters will be adjusted and the GPU computing performance requirements will be recalculated. If no anomalies are found, at least three of the most reasonable allocation and deployment schemes will be selected and presented to the operations personnel, along with a visual representation of their digital twins to facilitate their decision-making.

[0077] Scenario 2, for business scenarios where delivery is completed based on the total business volume and a clearly defined GPU computing performance usage: The resource management system can determine the total business scale (G) and GPU computing performance (P). future The requirements are imported into the resource configuration model of the corresponding business scenario to calculate the business processing time T. Through digital twin technology, operations personnel can predict in advance the specific time for business completion and the time when GPU hardware resources can be released, maximizing resource operation efficiency.

[0078] Scenario 3, for business scenarios where delivery is based solely on the total business volume: The resource management system can import the defined total business volume G demand into the resource configuration model of the corresponding business scenario. The other two unknown parameters are matched with the current busy / idle allocation of the entire GPU pool resources, as well as the predicted future resource occupancy and release time. Combined with the digital twin model, the most reasonable GPU computing performance P_future and business processing time T demand are calculated to maximize resource operation efficiency.

[0079] Specifically, in this embodiment of the application, the resource management system can determine the simulation verification result of the initial resource allocation policy according to the following sub-steps, including: Sub-step 1501: Based on the initial resource allocation strategy, simulate the execution of business requests in the virtual environment constructed based on the digital twin; Specifically, the resource management system can locate the GPU hardware twin instances that will be occupied in the digital twin layer according to the initial resource allocation strategy, and inject a simulated virtual task with the target business type and scale into the virtual environment composed of the twins, assuming that it runs according to the typical behavior pattern of the business type, and that its computing, memory access and other operations will consume the virtual hardware resources of the corresponding twin.

[0080] Sub-step 1502: Obtain simulation index data generated during the simulation execution process; In this embodiment of the application, the simulated indicator data obtained by the resource management system may include, but is not limited to, the following: Simulate hardware utilization; Simulated business processing time, which is the total simulation time of a virtual task from start to finish; Simulated business processing progress, representing the percentage of virtual task completion; The simulated hardware bottleneck indicator is used to determine whether a bottleneck exists by calculating the overall GPU utilization and the utilization of various resources during the simulation process.

[0081] Sub-step 1503: Based on the simulation index data obtained by executing sub-step 1502, determine the simulation results corresponding to the initial resource allocation strategy; In this embodiment of the application, the resource management system can determine the simulation results corresponding to the initial resource allocation strategy according to the following method, including: Step 1: Determine if the GPU's overall utilization has reached its maximum or is overloaded; It should be noted here that, ideally, if the weighting factors have been adjusted to their optimal level using historical data, the overall utilization rate U of the GPU will be [high / low]. Card U Slice U Slices or U Cluster It can reach or approach 1.

[0082] In this scenario, if the GPU overall utilization is significantly lower than 1, or if there are instances where the overall utilization of a single GPU differs greatly from that of other GPUs in the cluster at the same time, it indicates a bottleneck in a single GPU or the GPU cluster. If the GPU overall utilization far exceeds a set threshold, it indicates overload in a single GPU or the GPU cluster. The GPU overall utilization threshold can be dynamically adjusted using historical data and redundancy analysis. If the overall utilization result is significantly lower than the threshold, further bottleneck analysis is performed.

[0083] Step 2: Determine if the GPU is a bottleneck or over-saturated; It should be noted that the overall GPU utilization rate provided in this embodiment reflects the overall usage of the ALU, video memory, video memory bandwidth, and inter-card interconnect bandwidth within the GPU component. Therefore, in one implementation, the GPU bottleneck or over-sizing can be determined by comparing the overall GPU utilization rate in a specific business digital twin with the GPU utilization rate.

[0084] Specifically, this can include the following situations: Scenario 1: When the overall GPU utilization is much higher than the GPU utilization rate: This indicates that one or more of the following components—memory, video memory bandwidth, and inter-card interconnect bandwidth—are under excessive load, which may lead to bottlenecks in the future. This will prevent the GPU utilization rate from being further improved, thus failing to reach the FLOPS value released by the GPU product. Further analysis of memory, video memory bandwidth, and inter-card interconnect bandwidth is required through process three.

[0085] Scenario 2, when the overall GPU utilization is much lower than the GPU utilization rate: This indicates that one or more of the following resources—memory, video memory bandwidth, and inter-card interconnect bandwidth—are underloaded, meaning that these resources still have a lot of spare capacity. Even if the GPU utilization reaches 100% in the future, these resources may still be surplus, indicating an over-saturation situation.

[0086] Step 3: Identify the bottleneck hardware indicators and determine the weakest link effect; In this embodiment of the application, the resource management system can analyze the utilization margin of each hardware resource and the bottleneck of the cluster, specifically including: A. For single GPU card or single GPU slice scenarios, the utilization margin can be determined according to the following formula

[12] :

[12] Among them, U Resource This indicates the utilization rate of a certain hardware metric of the GPU, such as GPU utilization U. ALU GPU memory utilization U Mem GPU memory bandwidth utilization U MemBW and GPU card interconnect bandwidth utilization U Interconnect wait.

[0087] B. For cluster scenarios such as GPU clusters, single GPU servers, and multiple GPU slices, in order to determine the bottleneck in the cluster, the following formula

[13] can be used to average each indicator of GPU hardware, such as GPU utilization, GPU memory utilization, and GPU memory bandwidth utilization, respectively:

[13] in, This represents the utilization rate of a hardware metric for GPU index i, such as GPU utilization U. ALU GPU memory utilization U Mem GPU memory bandwidth utilization U MemBW and GPU card interconnect bandwidth utilization U Interconnect These average values ​​represent the overall situation of these hardware metrics in the cluster.

[0088] The cluster utilization margin is defined according to the following formula

[14] :

[14] It's important to note that, when considering redundancy, the "1" in the above formula can be set to a specific threshold. Calculate the reserve R for each type of resource. Resource R Resource.Avg The smaller the resource's margin, the closer it is to the bottleneck. If a resource's margin is close to 0, while other resources have larger margins, then that resource is the current performance bottleneck. This helps operations personnel locate the bottleneck and follow up on its resolution, thereby optimizing system configuration to improve overall performance.

[0089] C. For a specific business scenario, a digital twin can be used to match the backend hardware deployment method, applicable to the above-mentioned single GPU card or single GPU slice scenarios, as well as cluster scenarios such as GPU clusters, single GPU servers, and multiple GPU slices.

[0090] Next, the resource management system can determine the ranking of GPU hardware performance and utilization. In this embodiment, the ranking can be performed in the following two ways: Method 1: For different GPU hardware performances supporting similar business scenarios, the following sorting method can be used: By statistically analyzing historical digital twins or current real-time digital twins of the same business scenario, and through data analysis and machine learning training, the computing power efficiency η value of each GPU hardware or cluster supporting that business scenario is obtained for performance comparison and ranking. For GPUs of different models from the same manufacturer or GPUs from different manufacturers, ranking them by computing power efficiency η value for the same business scenario can assess the true performance differences and rankings of numerous GPU chips; the true performance ranking is then compared with the product release performance P. Publish By ranking and comparing them, we can assess the difference between their actual performance and the performance released in the product, and guide subsequent GPU chip procurement or problem analysis and optimization.

[0091] Method 2: Ranking the overall utilization of the same GPU hardware across different business scenarios: For the same GPU hardware, based on its historical state digital twins across all past business scenarios, or its current real-time state digital twins, and after multiple data analyses and machine learning training iterations, the overall GPU utilization U is compared. Card U Slice U Slices or U Cluster By analyzing bottlenecks, we can assess the most suitable business scenarios and utilization rankings for the GPU hardware, providing a reference for optimal matching of subsequent business deployments and GPU hardware resources.

[0092] Furthermore, the resource management system can optimize the resource allocation model in reverse based on the resolution of hardware performance bottlenecks, the analysis of hardware resource overload, resource surplus, and the ranking of hardware performance, and correct the relevant parameters accordingly.

[0093] Finally, the resource management system can use the simulation results determined above, combined with the busy / idle allocation of the entire GPU pool resources in the current network, the GPU hardware computing power efficiency η value, and the permissions assigned to it by the operators, to independently judge and decide on several optimal allocation and deployment schemes; then, it issues instructions to the external GPU resource operation platform to execute tasks, and synchronizes the running status data to the digital twin in real time to realize the visualization and visualization of the running data.

[0094] The GPU resource management method based on digital twins provided in this application involves the following steps: First, a digital twin corresponding to the GPU server is constructed based on the collected GPU server operation data. Then, the physical GPU hardware entity and the running status of the services carried by the GPU server are mapped and displayed through the digital twin. Based on the GPU server operation data recorded in the digital twin, a resource configuration model is generated. Based on the resource configuration model, the received service requests are predicted to determine the GPU resource requirements corresponding to the service requests. Then, an initial resource allocation strategy corresponding to the service requests is generated based on the GPU resource requirements. The initial resource allocation strategy is simulated and verified through the digital twin to obtain simulation results. Then, the initial resource allocation strategy is adjusted based on the simulation results to generate a new resource allocation strategy, which is then sent to the GPU server for execution. The GPU resource management method based on digital twins provided in this application has two advantages. First, it can generate a resource configuration model based on historical and real-time operational data accumulated by the digital twin. This resource configuration model can associate specific business scenario characteristics with multi-dimensional hardware indicators of the GPU. By using this resource configuration model to predict resource requirements for new business requests, the prediction results can be more closely aligned with the actual load of the production environment, thus providing accurate data for GPU resource allocation and significantly improving the scientific nature and accuracy of resource planning. Second, by constructing a digital twin corresponding to the GPU server, the complex GPU hardware status, business operation logic, and the relationship between the two can be visualized in an intuitive graphical way. This allows operators to use the digital twin to determine the overall health of each resource pool in the GPU server, the resource consumption trajectory of a single business, and potential performance bottlenecks in real time, greatly improving operational efficiency and problem location speed.

[0095] In one embodiment, this application also provides a GPU resource management device based on digital twins to address the problem that existing GPU operation solutions rely on traditional cloud management platforms and human experience, making it impossible to schedule GPU resources according to real-time business loads in actual production. This leads to a disconnect between resource planning and actual needs, easily resulting in resource waste and affecting business processing efficiency. A schematic diagram of the specific structure of this GPU resource management device is shown below. Figure 5 As shown, it includes: a digital twin construction unit 51, a digital twin construction unit 52, a prediction unit 53, an allocation strategy generation unit 54, and a simulation verification unit 55.

[0096] The digital twin construction unit 51 is used to construct a digital twin corresponding to the GPU server based on the collected GPU server operation data. The digital twin is used to map and display the physical GPU hardware entity and the running status of the services carried by the GPU server. The configuration model generation unit 52 is used to generate a resource configuration model based on the GPU server operation data recorded in the digital twin; Prediction unit 53 is used to predict the received service request based on the resource configuration model and determine the GPU resource requirement corresponding to the service request. The allocation strategy generation unit 54 is used to generate an initial resource allocation strategy corresponding to the service request based on the GPU resource requirements. The simulation verification unit 55 is used to perform simulation verification on the initial resource allocation strategy based on the digital twin, obtain simulation results, generate a resource allocation strategy based on the simulation results, and send the resource allocation strategy to the GPU server for execution.

[0097] In one implementation, the collected GPU server operation data includes hardware operation data and business execution data; the collected hardware operation data includes at least one of the following: GPU model parameters, GPU core unit utilization, GPU memory utilization, GPU memory bandwidth utilization, and GPU card interconnect bandwidth utilization; the collected business execution data includes at least one of the following: business type, business scale, business processing time, and GPU computing resources corresponding to the business.

[0098] In one implementation, the configuration model generation unit 52 is specifically configured to: determine the hardware operation data corresponding to each business type recorded in the digital twin; determine the computing power efficiency corresponding to each business type recorded in the digital twin based on the hardware operation data, wherein the computing power efficiency is used to represent the business scale that can be processed by a unit of GPU computing performance for the business type; and construct a prediction model based on the computing power efficiency to represent the correspondence between business scale, business processing time and required GPU computing performance, as the resource configuration model.

[0099] In one implementation, the prediction unit 53 is specifically configured to: determine the target requirements corresponding to the service request, wherein the service requirements include at least two of the following: target service type, target service scale, and target processing time requirement; process the target requirements according to the resource configuration model corresponding to the target service type, and determine the GPU computing performance requirements required by the service request as the GPU resource requirements.

[0100] In one embodiment, the simulation verification unit 55 is specifically configured to: simulate the execution of a business request in a virtual environment constructed based on the digital twin, according to the initial resource allocation strategy; acquire simulation indicator data generated during the simulation execution, wherein the simulation indicator data includes at least one of: simulated hardware utilization, simulated business processing progress, simulated business processing duration, and simulated hardware bottleneck indication; and determine the simulation result corresponding to the initial resource allocation strategy based on the simulation indicator data.

[0101] In one implementation, the risk prediction unit is specifically configured to: determine the utilization rate of each hardware component in the GPU server based on the hardware operation data; determine the overall utilization rate of the GPU server based on the utilization rate of each hardware component; determine whether the overall utilization rate exceeds a preset normal utilization rate range; if the overall utilization rate is lower than the lower limit of the normal utilization rate range, determine that the GPU server has a hardware configuration bottleneck; if the overall utilization rate is higher than the upper limit of the normal utilization rate range, determine that the GPU server has a resource overload risk.

[0102] The GPU resource management device based on digital twin provided in this application, when performing GPU resource management, firstly, constructs a digital twin corresponding to the GPU server based on the collected GPU server operation data. Then, it maps and displays the physical GPU hardware entity and the running status of the services carried by the GPU server through the digital twin. Based on the GPU server operation data recorded in the digital twin, a resource configuration model is generated. Based on the resource configuration model, the received service requests are predicted to determine the GPU resource requirements corresponding to the service requests. Then, an initial resource allocation strategy corresponding to the service requests is generated based on the GPU resource requirements. The initial resource allocation strategy is simulated and verified through the digital twin to obtain simulation results. Then, the initial resource allocation strategy is adjusted based on the simulation results to generate a resource allocation strategy, and the resource allocation strategy is sent to the GPU server for execution. The GPU resource management device based on digital twins provided in this application can, on the one hand, generate a resource configuration model based on historical and real-time operational data accumulated by the digital twin. This resource configuration model can associate specific business scenario characteristics with multi-dimensional hardware indicators of the GPU. By using this resource configuration model to predict resource requirements for new business requests, the prediction results can be more closely aligned with the actual load of the production environment, thereby providing accurate data for GPU resource allocation and significantly improving the scientific nature and accuracy of resource planning. On the other hand, by constructing a digital twin corresponding to the GPU server, the complex GPU hardware status, business operation logic, and the relationship between the two can be visualized in an intuitive graphical way. This allows operators to use the digital twin to determine the overall health of each resource pool in the GPU server, the resource consumption trajectory of a single business, and potential performance bottlenecks in real time, greatly improving operational efficiency and problem location speed.

[0103] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 6 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0104] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0105] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0106] The processor reads the corresponding computer program from non-volatile memory into main memory and then executes it, forming a GPU resource management device based on digital twins at the logical level. The processor executes the program stored in memory and specifically performs the following operations: Based on the collected GPU server operation data, a digital twin corresponding to the GPU server is constructed. This digital twin maps and displays the physical GPU hardware entity corresponding to the GPU server and the operational status of the services it supports. A resource configuration model is generated based on the GPU server operation data recorded in the digital twin. Based on the resource configuration model, received service requests are predicted to determine the GPU resource requirements corresponding to those requests. An initial resource allocation strategy is generated based on the GPU resource requirements. The initial resource allocation strategy is simulated and verified using the digital twin to obtain simulation results. A new resource allocation strategy is generated based on the simulation results and then sent to the GPU server for execution.

[0107] The above is as stated in this application. Figure 6The method for executing a GPU resource management electronic device based on digital twins, as disclosed in the illustrated embodiments, can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0108] Of course, in addition to software implementation, the electronic device of this application does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0109] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by a portable electronic device including multiple applications, enable the portable electronic device to perform... Figure 2 The illustrated embodiment of the GPU resource management method based on digital twins is specifically used to perform the following operations: Based on the collected GPU server operation data, a digital twin corresponding to the GPU server is constructed. This digital twin maps and displays the physical GPU hardware entity corresponding to the GPU server and the operational status of the services it supports. A resource configuration model is generated based on the GPU server operation data recorded in the digital twin. Based on the resource configuration model, received service requests are predicted to determine the GPU resource requirements corresponding to those requests. An initial resource allocation strategy is generated based on the GPU resource requirements. The initial resource allocation strategy is simulated and verified using the digital twin to obtain simulation results. A new resource allocation strategy is generated based on the simulation results and then sent to the GPU server for execution.

[0110] It should be understood that the training and prediction processes of the AI ​​models involved in the various embodiments of this specification all adhere to multiple legal and compliant principles, including legal data sources, compliant data content, compliant data governance, compliant training objectives and schemes, compliant training processes, compliant training environments and tools, and compliant ethical verification of training results, and comply with the requirements of Article 5 of the Patent Law. Among them: Data source legitimacy: All datasets used for AI model training were obtained through legal means, covering three categories: publicly authorized data, data authorized by partners, and self-collected compliant data. Publicly authorized data comes from compliant data sources following open-source licenses such as Apache 2.0, with complete copyright attribution and authorization scope clearly marked, and no unauthorized open-source code or data reuse. Data authorized by partners has been subject to formal data usage agreements, clearly defining the scope, duration, and confidentiality obligations, and possessing a complete authorization chain. For self-collected data involving personal information, strict informed consent procedures have been followed, and anonymization processes (including but not limited to field masking, feature anonymization, and differential privacy technology applications) have been implemented to remove personally identifiable information, fully complying with the requirements of relevant laws and regulations such as the "Interim Measures for the Administration of Generative Artificial Intelligence Services" and the "Personal Information Protection Law."

[0111] Data content compliance: The AI ​​model's dataset undergoes multiple screenings and cleaning processes to remove all content that may violate social morality or harm public interests. It contains no obscene, pornographic, violent, discriminatory, or information that endangers national or public safety, nor does it involve the illegal acquisition or use of genetic resources. For data in sensitive fields (such as healthcare and finance), an additional privacy-preserving computation module (including federated learning and secure multi-party computation technologies) ensures that the data is "usable but not visible," avoiding compliance risks during the original data transmission process and ensuring that the data application scenarios and uses comply with public order and good morals and industry regulatory requirements.

[0112] Data governance norms: A complete data traceability system is established during the AI ​​model training process to automatically record the source, collection time, annotation process, cleaning rules, and permission allocation of training data, generating traceable compliance reports to ensure that the data is verifiable throughout its entire lifecycle. The dataset annotation process for AI models is completed by a professional human R&D team, clearly defining the proportion of human creative contributions and avoiding reliance on AI-generated data that has not undergone substantial human modification, thus meeting the examination requirements for "human main contributions" in AI patent applications.

[0113] Training objectives and plans are compliant: The AI ​​model training objective focuses on simulating the processing of GPU business requests. The training scheme and the final output results do not violate any mandatory provisions of laws and administrative regulations, do not harm the public interest or the legitimate rights and interests of others, and do not pose any potential risks of being used for illegal activities, infringing on privacy, or undermining public safety. It strictly adheres to the ethical principle of "intelligent for good".

[0114] Training process compliance: A closed-loop training framework is adopted to ensure compliance and controllability of the training process. The specific process is as follows: First, training samples are obtained through compliant data sources. After the aforementioned data cleaning and desensitization, they are input into the neural network model to generate preliminary training results. Second, an expert system is introduced to verify the preliminary results. Based on preset rules and human expert experience, the feasibility of the results is evaluated, and outputs that may pose ethical risks or compliance hazards are corrected (such as removing decision-making logic that violates public order and good morals, and adjusting model parameters that do not comply with safety regulations). Finally, the loss function weights are dynamically optimized based on expert system feedback to strengthen the model's learning of compliant results, avoid overfitting errors or non-compliant labels, and form a closed-loop control of "data input - model training - expert verification - parameter optimization - result feedback" to ensure that the entire training process complies with A5 ethical review requirements.

[0115] Training environment and tool compliance: AI model training is implemented using nationally licensed chips and a compliant training platform. All open-source frameworks and components used in the training process have obtained their corresponding licenses, and copyright statements and patent citation information are fully retained, with no instances of infringement or reuse. The training environment is built using virtual devices (containers / virtual machines) with fixed random seeds and initial parameter configurations to ensure the reproducibility of the training process. Furthermore, through access control and operation log recording, risks such as data leakage and parameter tampering during training are prevented, ensuring the security and compliance of the training process.

[0116] Training results ethical verification compliance: After the model is trained, it undergoes additional third-party ethical compliance assessment and algorithm filing review to verify that the model output does not violate social morality or harm public interests. For potentially sensitive scenarios (such as public services and intelligent decision-making), a special result verification mechanism is established to ensure that the model always complies with Article 5 of the Patent Law and relevant laws and regulations in practical applications.

[0117] In summary, the data and training process used in the AI ​​model of this specification strictly comply with the relevant provisions of Article 5 of the Patent Law and the Patent Examination Guidelines (2023 Edition), and there are no violations of laws, social ethics, public interests, or illegal use of genetic resources. It fully meets the compliance requirements for patent authorization.

[0118] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0119] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0120] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0121] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0122] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0123] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0124] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0125] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0126] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0127] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A GPU resource management method based on digital twins, characterized in that, include: Based on the collected GPU server operation data, a digital twin corresponding to the GPU server is constructed. The digital twin is used to map and display the physical GPU hardware entity corresponding to the GPU server and the operating status of the services it carries. A resource configuration model is generated based on the GPU server operation data recorded in the digital twin; Based on the resource configuration model, the received service requests are predicted to determine the GPU resource requirements corresponding to the service requests. Generate an initial resource allocation strategy corresponding to the service request based on the GPU resource requirements; The initial resource allocation strategy is simulated and verified using the digital twin to obtain simulation results. A resource allocation strategy is then generated based on the simulation results and sent to the GPU server for execution.

2. The method according to claim 1, characterized in that, The collected GPU server operation data includes hardware operation data and business execution data; The hardware operation data includes at least one of the following: GPU model parameters, GPU core unit utilization, GPU memory utilization, GPU memory bandwidth utilization, and GPU card interconnect bandwidth utilization. The business execution data includes at least one of the following: business type, business scale, business processing time, and GPU computing resources corresponding to the business.

3. The method according to claim 2, characterized in that, The step of generating a resource configuration model based on the GPU server operation data recorded in the digital twin specifically includes: Determine the hardware operation data corresponding to each service type recorded in the digital twin; Based on the hardware operation data, the computing power efficiency corresponding to each business type recorded in the digital twin is determined, wherein the computing power efficiency is used to represent the business scale that can be processed by a unit of GPU computing performance for the business type. Based on the computing efficiency, a predictive model is constructed to represent the correspondence between business scale, business processing time and required GPU computing performance, which serves as the resource allocation model.

4. The method according to claim 1, characterized in that, The step of predicting the received service requests based on the resource configuration model and determining the GPU resource requirements corresponding to the service requests specifically includes: Determine the target requirements corresponding to the business request, wherein the business requirements include at least two of the following: target business type, target business scale, and target processing time requirement; Based on the resource configuration model corresponding to the target business type, the target requirements are processed to determine the GPU computing performance requirements required by the business request, which are then used as the GPU resource requirements.

5. The method according to claim 1, characterized in that, The step of simulating and verifying the initial resource allocation strategy based on the digital twin to obtain simulation results specifically includes: According to the initial resource allocation strategy, business requests are simulated and executed in a virtual environment constructed based on the digital twin. The simulation indicator data generated during the simulation execution process is obtained, wherein the simulation indicator data includes at least one of the following: simulation hardware utilization, simulation business processing progress, simulation business processing duration, and simulation hardware bottleneck indication. Based on the simulation index data, the simulation results corresponding to the initial resource allocation strategy are determined.

6. The method according to claim 2 or 3, characterized in that, The method further includes: Based on the hardware operation data, determine the utilization rate of each hardware component in the GPU server. The overall utilization rate of the GPU server is determined based on the utilization rate of each hardware component. Determine whether the overall utilization rate exceeds the preset normal utilization rate range; If the overall utilization rate is lower than the lower limit of the normal utilization rate range, it is determined that the GPU server has a hardware configuration bottleneck. If the overall utilization rate is higher than the upper limit of the normal utilization rate range, then the GPU server is determined to be at risk of resource overload.

7. A GPU resource management device based on digital twins, characterized in that, include: A digital twin construction unit is used to construct a digital twin corresponding to the GPU server based on the collected GPU server operation data. The digital twin is used to map and display the physical GPU hardware entity corresponding to the GPU server and the operating status of the services it carries. A configuration model generation unit is used to generate a resource configuration model based on the GPU server operation data recorded in the digital twin; The prediction unit is used to predict the received service request based on the resource configuration model and determine the GPU resource requirement corresponding to the service request. The allocation strategy generation unit is used to generate an initial resource allocation strategy corresponding to the service request based on the GPU resource requirements. The simulation verification unit is used to perform simulation verification on the initial resource allocation strategy based on the digital twin, obtain simulation results, generate a resource allocation strategy based on the simulation results, and send the resource allocation strategy to the GPU server for execution.

8. A GPU resource management device based on digital twin, comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the following operations: Based on the collected GPU server operation data, a digital twin corresponding to the GPU server is constructed. The digital twin is used to map and display the physical GPU hardware entity corresponding to the GPU server and the operating status of the services it carries. A resource configuration model is generated based on the GPU server operation data recorded in the digital twin; Based on the resource configuration model, the received service requests are predicted to determine the GPU resource requirements corresponding to the service requests. Generate an initial resource allocation strategy corresponding to the service request based on the GPU resource requirements; The initial resource allocation strategy is simulated and verified using the digital twin to obtain simulation results. A resource allocation strategy is then generated based on the simulation results and sent to the GPU server for execution.

9. A computer-readable storage medium storing one or more programs that, when executed by an electronic device including a plurality of applications, cause the electronic device to perform the GPU resource management method based on digital twins as described in any one of claims 1-6.

10. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the GPU resource management method based on digital twins as described in any one of claims 1-6.

Images