Large neural network model video memory and memory fusion distribution method

Abstract

The invention provides a video memory and memory fusion distribution method for a large neural network model, and belongs to the technical field of large neural network models, and the method comprises the steps: collecting system state data in real time, including a video memory, a memory utilization rate and related performance indexes; static and dynamic characteristic components are extracted by innovatively applying singular value decomposition, a multi-dimensional mapping relation model of molecular dynamics simulation is constructed, and a coupling state equation set of video memory allocation is established. And solving the equation set by adopting an ant colony optimization algorithm, generating a dynamic storage pointer, and establishing a multi-target storage scheduling equation set through joint characteristic decomposition to optimize resource utilization, access delay, fragment elimination, migration overhead and load balance. Algorithm parameters are updated by utilizing an entropy evaluation method, and finally collaborative scheduling of video memory and memory resources is realized in a unified storage management module, so that the calculation efficiency and the resource utilization rate of a neural network model are remarkably improved, and the problem that allocation of large model memories and video memories is not optimized enough in the prior art is solved.

Description

Technical Field

[0001] This invention belongs to the field of large neural network model technology, and more specifically, relates to a method for fusion allocation of video memory in large neural network models. Background Technology

[0002] Current mainstream GPU memory management strategies mainly include static allocation and dynamic allocation. Static allocation determines the storage space allocation scheme during model loading, offering simplicity but lacking flexibility and prone to memory fragmentation and resource waste. Dynamic allocation, while adjusting storage space according to real-time needs, suffers from severe memory fragmentation and high scheduling overhead. Furthermore, existing memory management solutions often separate GPU memory and system memory, lacking a unified collaborative scheduling mechanism. In practical applications, due to GPU memory capacity limitations, large neural network models often require distributed training using model parallelism or pipelined parallelism. While this approach overcomes the limitations of single-card GPU memory capacity, it introduces significant communication and synchronization overhead, reducing training efficiency. Simultaneously, existing storage management solutions lack in-depth analysis and utilization of the dynamic characteristics during model operation, making it difficult to adapt to complex and changing computational loads and access patterns. In particular, in resource-constrained scenarios such as edge computing and mobile devices, achieving better efficient deployment of large neural networks and efficient memory and GPU memory allocation during inference under limited GPU and system memory resources has become a pressing technical challenge. Existing technologies struggle to effectively balance multiple optimization objectives such as computing performance, storage efficiency, and resource utilization, thus hindering the further application and development of deep learning technologies. Summary of the Invention

[0003] In view of this, the present invention provides a method for fusion allocation of video memory and video memory for large neural network models, which can solve the problem in the prior art that it is difficult to achieve efficient deployment of large neural networks and more optimized allocation of memory and video memory during inference under the condition of limited video memory and video memory resources.

[0004] This invention is implemented as follows:

[0005] This invention provides a method for fusion allocation of video memory for large neural network models, comprising the following steps:

[0006] S10. A hierarchical acquisition mechanism is used to acquire system status data in real time during the operation of the neural network model, including: video memory usage data, memory usage data, storage bandwidth utilization data, access latency data, computing load distribution data, data access frequency data, video memory fragmentation distribution data, and memory fragmentation distribution data, and a video memory-memory swapping cost evaluation model is constructed based on the system status data.

[0007] S20. The singular value decomposition method is used to perform multivariate matrix decomposition operation on the video memory usage data and memory usage data to extract the static feature components of video memory, the dynamic feature components of video memory, the static feature components of memory, and the dynamic feature components of memory.

[0008] S30. Based on the molecular dynamics simulation method, construct a multidimensional mapping relationship model between the static feature components of the video memory, the dynamic feature components of the video memory, the static feature components of the memory, the dynamic feature components of the memory, and the computational load distribution data, data access frequency data, and storage bandwidth utilization data, denoted as the fusion allocation mapping model.

[0009] S40. Calculate the static memory allocation feature matrix, dynamic memory allocation feature matrix, static memory allocation feature matrix, and dynamic memory allocation feature matrix using the fusion allocation mapping model, and establish the coupled state equation system of the four allocation feature matrices.

[0010] S50. Solve the coupled state equations using the ant colony optimization algorithm to obtain a video memory fusion allocation strategy, and allocate dynamic storage pointers to the neural network model parameters based on the video memory fusion allocation strategy.

[0011] S60. Establish a video memory scheduling feature matrix and a memory scheduling feature matrix based on the dynamic storage pointer. Obtain video memory scheduling feature vector and memory scheduling feature vector through joint feature decomposition. At the same time, calculate the storage unit access frequency vector, data block size vector, and hierarchical migration cost matrix.

[0012] S70. Construct a multi-objective storage scheduling equation set to optimize resource scheduling. The multi-objective storage scheduling equation set includes resource utilization optimization equation, access latency optimization equation, fragmentation elimination optimization equation, migration overhead optimization equation, and load balancing optimization equation.

[0013] S80. Analyze the video memory fragmentation distribution data and memory fragmentation distribution data using the entropy evaluation method, update the pheromone distribution of the ant colony optimization algorithm, and recalculate the dynamic storage pointer.

[0014] S90. The dynamic storage pointer is applied in the unified storage management module of the neural network model. The unified storage management module realizes the coordinated scheduling of video memory resources and memory resources through a virtual address mapping mechanism.

[0015] The unified storage management module is a core component for coordinating the scheduling of video memory and RAM resources for large neural network models. Its main function is to ensure the efficient allocation and transfer of neural network model parameters between video memory and RAM, and to achieve unified management and optimized scheduling of the two types of storage resources, thereby improving resource utilization, reducing access latency, reducing memory fragmentation, and balancing the computing load.

[0016] Furthermore, the resource utilization optimization equation is used to maximize the overall utilization efficiency of video memory. The inputs include video memory scheduling feature vector, memory scheduling feature vector, and data block size vector, and the output is the resource utilization coefficient of different storage levels.

[0017] The access latency optimization equation is used to reduce the average time overhead of data access. The inputs include the storage unit access frequency vector, the data storage location matrix, and the access latency data. The output is the system average access latency.

[0018] The fragmentation elimination optimization equation is used to merge discrete storage space fragments. The inputs include video memory fragmentation distribution data, memory fragmentation distribution data, and minimum storage block threshold. The output is a distribution sequence of continuous storage space.

[0019] The migration cost optimization equation is used to balance the performance benefits and resource costs of data migration. The inputs include the hierarchical migration cost matrix, the migration data scale matrix, and the storage bandwidth utilization data. The output is the scheduling sequence of cross-level data migration.

[0020] The load balancing optimization equation is used to achieve dynamic balance of computing load across different storage tiers. The inputs include computing load distribution data and load balancing coefficients, and the output is a balanced allocation scheme for storage resources.

[0021] The coupled state equation set includes the video memory state evolution equation, the memory state evolution equation, the resource occupancy constraint equation, the state transition equation, and the dynamic equilibrium equation.

[0022] The memory state evolution equation is used to describe the change law of memory resource allocation state over time. The input includes memory static feature components, memory dynamic feature components, memory static allocation feature matrix, and memory dynamic allocation feature matrix. The output is the temporal evolution feature of memory resource state.

[0023] The memory state evolution equation is used to describe the change law of memory resource allocation state over time. The input includes memory static feature components, memory dynamic feature components, memory static allocation feature matrix, and memory dynamic allocation feature matrix. The output is the temporal evolution feature of memory resource state.

[0024] The resource occupancy constraint equation is used to limit the boundary conditions for the allocation of video memory and main memory resources. The inputs include computational load distribution data, data access frequency data, and storage bandwidth utilization data. The output is the constraint boundary of resource allocation.

[0025] The state transition equation is used to describe the state transition relationship when data migrates between video memory and main memory. The input includes the video memory-to-main memory exchange cost evaluation model, the hierarchical migration cost matrix, and the access latency time data. The output is the probability distribution of state transition.

[0026] The dynamic equilibrium equation is used to describe the equilibrium state characteristics of the system during the resource allocation process. The inputs include the migration data scale matrix, the calculation load distribution data, and the storage bandwidth utilization data. The output is the steady-state equilibrium characteristics of the system.

[0027] Furthermore, the equations or mathematical models involved in this invention are described in detail below:

[0028] In S30, the fusion allocation mapping model adopts the Langevin equation from molecular dynamics, and is specifically expressed as follows:

[0029] ;

[0030] In the formula, Indicates the first The state vector of each storage unit; The damping coefficient ranges from [0.1, 1.0]. Let be the potential energy function, representing the interaction between memory cells; It is a random force term that follows a normal distribution.

[0031] In S40, the memory state evolution equations in the coupled state equation set are specifically represented as follows:

[0032] ;

[0033] In the formula, This is the video memory state function; The memory diffusion coefficient; This refers to the memory growth rate. This is the maximum capacity of video memory; This refers to the memory loss factor. This is the video memory usage matrix; This is a random disturbance term.

[0034] The memory state evolution equation is specifically represented as follows:

[0035] ;

[0036] In the formula, For memory state functions; , where is the memory diffusion coefficient; This represents the memory growth rate. Maximum memory capacity; This is the memory consumption factor; This is a memory usage matrix; This is a random disturbance term.

[0037] The resource occupancy constraint equation is specifically expressed as follows:

[0038] ;

[0039] In the formula, These are the weighting coefficients; The objective function; As a penalty factor; These are constraints.

[0040] The state transition equation adopts a Markov chain model, specifically expressed as follows:

[0041] ;

[0042] In the formula, The state transition probability; Energy for state transition; Boltzmann constant; This refers to the system temperature parameter.

[0043] The dynamic equilibrium equation is specifically expressed as follows:

[0044] ;

[0045] In the formula, This is the system state vector; Here is the system state transition matrix; The vector of external forces; This is the system noise vector.

[0046] The resource utilization optimization equation is specifically expressed as follows:

[0047] ;

[0048] In the formula, For the first The actual usage of each storage unit; For the first The capacity of each storage unit; These are the weighting coefficients; This is the balance coefficient; This represents the degree of correlation between units.

[0049] The access latency optimization equation is specifically expressed as follows:

[0050] ;

[0051] In the formula, Access frequency; Access delay; For access probability; This is the entropy weighting coefficient.

[0052] The fragmentation elimination optimization equation is specifically expressed as follows:

[0053] ;

[0054] In the formula, For the first The size of each fragment; Minimum fragmentation threshold; For fragment weights; This is the smoothing coefficient.

[0055] The migration cost optimization equation is specifically expressed as follows:

[0056] ;

[0057] In the formula, For migration costs; For migration decision variables; Bandwidth usage; Maximum bandwidth; This is the bandwidth weight.

[0058] The load balancing optimization equation is specifically expressed as follows:

[0059] ;

[0060] In the formula, For the first The load value of each node; This is the average load value; For nodes The set of neighbors; This is the local equilibrium coefficient.

[0061] It should be explained that the parameters in the above equations all consider dimensionless processing. Dimensionless processing, also known as standardization or dimensionless measurement, aims to eliminate the influence of data units on the analysis results. Common methods include various approaches. For example, Min-Max scaling (normalization) linearly maps data to a specified range, preserving the relative relationships of the original data but being sensitive to outliers; standardization (Z-score normalization) converts data into a distribution with a mean of 0 and a standard deviation of 1, suitable for approximately normally distributed data and insensitive to outliers; decimal scaling restricts the absolute value of data to within [-1, 1] by moving the decimal point. In this invention, Min-Max scaling is preferred for dimensionless processing.

[0062] The principles behind the construction of these equations are as follows:

[0063] 1. The state evolution equation adopts the form of a reaction-diffusion equation, taking into account four aspects: spatial diffusion, natural growth, loss, and random perturbation;

[0064] 2. The constraint equations employ the penalty function method, transforming the constraints into penalty terms for the objective function;

[0065] 3. The state transition equation is based on the Boltzmann distribution and reflects the principle of minimizing system energy;

[0066] 4. The family of optimization equations adopts a multi-objective optimization framework, which considers both local optimization and global equilibrium.

[0067] Compared with existing technologies, the beneficial effects of the large neural network model memory fusion allocation method provided by this invention are:

[0068] First, a fusion allocation mapping model is constructed by introducing molecular dynamics simulations. This method can accurately characterize the complex relationship between GPU memory and system memory features and computational load. Experiments show that this mapping model greatly improves the accuracy of resource allocation and significantly reduces computational interruptions and performance fluctuations caused by improper resource allocation during the training of large-scale neural network models. Especially when dealing with ultra-large-scale models, the mapping model of this invention exhibits superior robustness and adaptability.

[0069] Secondly, an ant colony optimization algorithm is used to solve the coupled state equations, enabling dynamic optimization of GPU memory and system memory allocation. This method effectively solves the problem of severe GPU memory fragmentation in traditional allocation strategies. In practical applications, even after running for hundreds of training cycles, the system maintains efficient storage utilization, significantly reducing performance degradation caused by storage fragmentation. Simultaneously, this method also significantly reduces memory access latency, making the model training process smoother and more stable.

[0070] Third, by constructing a multi-objective storage scheduling equation set, the method achieves coordinated optimization of multiple objectives, including resource utilization, access latency, and fragmentation elimination. In actual tests, this method demonstrates excellent overall system performance, especially in maintaining a high level of storage bandwidth utilization when handling dynamically changing computational loads. This optimization effect is particularly prominent in large-scale distributed training scenarios.

[0071] Fourth, the dynamic adjustment mechanism based on the entropy evaluation method enables this invention to adapt to changes in computational load in real time, maintaining stable resource utilization efficiency. Even in scenarios with drastic load fluctuations, the system can maintain a stable operating state, avoiding the drastic performance fluctuations common in traditional methods. This adaptive capability makes the system particularly suitable for handling long-term model training tasks.

[0072] Fifth, a unified management mechanism for GPU memory and system memory is achieved through virtual address mapping, reducing the complexity of storage resource scheduling and significantly improving system scalability. This unified management mechanism has been validated on multiple large-scale machine learning platforms, successfully supporting efficient training and inference of models with hundreds of billions of parameters. Experiments demonstrate that this mechanism can effectively handle complex storage hierarchy structures and maintain stable performance across deep learning tasks of various scales.

[0073] By combining the above technical effects, this invention successfully solves the problem of efficient management of large neural network models under limited video memory and memory resources. It also solves the problem in existing technologies that it is difficult to achieve efficient deployment of large neural networks and more optimized allocation of memory and video memory during inference under limited video memory and memory resources. Attached Figure Description

[0074] Figure 1 A flowchart of the method provided by the present invention. Detailed Implementation

[0075] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0076] like Figure 1 The diagram shows a flowchart of a method for fusion allocation of video memory in a large neural network model provided by this invention. The specific implementation of each step is given below. Step S10 involves using a hierarchical acquisition mechanism to acquire system state data in real time during the operation of the neural network model. First, the method collects video memory usage data, memory usage data, storage bandwidth utilization data, access latency data, computational load distribution data, data access frequency data, video memory fragmentation distribution data, and memory fragmentation distribution data in real time. Then, a video memory exchange cost evaluation model is constructed based on this system state data. This model can quantify the exchange cost between different storage resources, providing a basis for subsequent fusion allocation of storage resources.

[0077] The specific implementation of step S20 involves using singular value decomposition (SVD) to perform multivariate matrix decomposition on the video memory usage data and the system memory usage data. First, these two types of data are constructed as rows based on time-series sampling points and as columns based on monitoring indicators, forming matrices to be decomposed. Then, singular value decomposition is performed on these two matrices respectively, yielding a left singular matrix representing the time feature vector, a singular value diagonal matrix representing the feature importance, and a right singular matrix representing the monitoring indicator feature vector. Finally, based on the singular value diagonal matrix, static video memory feature components, dynamic video memory feature components, static system memory feature components, and dynamic system memory feature components are extracted from the video memory usage data matrix and the system memory usage data matrix, respectively. These feature components will serve as the basic input for the subsequent fusion and allocation process.

[0078] The specific implementation of step S30 is based on the Langevin equation from molecular dynamics, which constructs a fusion allocation mapping model. First, the Langevin equation includes inertial, damping, potential energy, and stochastic force terms. Then, the extracted static, dynamic, static, and dynamic feature components of the video memory are input as state vectors into the Langevin equation. Simultaneously, load distribution data, data access frequency data, and storage bandwidth utilization data are used to construct the potential energy function. Finally, solving the Langevin equation yields a multidimensional mapping model of the fusion allocation of video and memory. This model can describe the complex relationship between various system state characteristics and actual resource allocation.

[0079] The specific implementation of step S40 is to construct a coupled state equation set based on the fused allocation mapping model. First, the output of the fused allocation mapping model is projected onto the video memory space and the main memory space, respectively, to obtain the video memory static allocation feature matrix, the video memory dynamic allocation feature matrix, the main memory static allocation feature matrix, and the main memory dynamic allocation feature matrix. Then, the video memory state evolution equation, the main memory state evolution equation, the resource occupancy constraint equation, the state transition equation, and the dynamic equilibrium equation are combined into a coupled state equation set. Finally, these four allocation feature matrices are used as input to form a complete state description system.

[0080] The specific implementation of step S50 involves using an ant colony optimization algorithm to solve the coupled state equations. First, the ant colony size, maximum number of iterations, and pheromone distribution are initialized. Then, the coupled state equations are used as constraints for the ant colony optimization algorithm. Next, the ant colony optimization algorithm is run to solve the coupled state equations, obtaining a memory fusion allocation strategy. Finally, dynamic storage pointers are allocated to the neural network model parameters according to this allocation strategy. The ant colony optimization algorithm can find an approximately optimal resource allocation scheme under complex multi-constraint conditions.

[0081] The specific implementation of step S60 involves constructing a memory scheduling feature matrix and a memory scheduling feature vector based on dynamic memory pointers. First, a video memory scheduling feature matrix and a memory scheduling feature matrix are established based on the dynamic memory pointers. Then, joint eigenvalue decomposition is performed on these two matrices to obtain the video memory scheduling feature vector and the memory scheduling feature vector. Furthermore, an access frequency vector is constructed by statistically analyzing memory unit access frequencies, a data block size vector is constructed by calculating data block size distribution, and a hierarchical migration cost matrix is ​​constructed based on a video memory-memory swapping cost evaluation model. These feature vectors and matrices provide foundational data for subsequent multi-objective memory scheduling optimization.

[0082] The specific implementation of step S70 involves constructing a multi-objective storage scheduling equation set and optimizing resource scheduling. First, a resource utilization optimization equation is constructed based on the video memory scheduling feature vector, the memory scheduling feature vector, and the data block size vector. This equation maximizes the overall utilization efficiency of video memory and memory. Second, an access latency optimization equation is constructed based on the storage unit access frequency vector, the data storage location matrix, and access latency data. This equation reduces the average time overhead of data access. Third, a fragmentation elimination optimization equation is constructed based on video memory fragmentation distribution data, memory fragmentation distribution data, and the minimum storage block threshold. This equation merges discrete storage space fragments. Furthermore, a migration overhead optimization equation is constructed based on the hierarchical migration cost matrix, the migration data size matrix, and storage bandwidth utilization data. This equation balances the performance gains and resource overhead of data migration. Finally, a load balancing optimization equation is constructed based on the computational load distribution data and the load balancing coefficient. This equation achieves dynamic balancing of computational load across different storage tiers. These five optimization equations constitute the multi-objective storage scheduling equation set.

[0083] The specific implementation of step S80 involves analyzing the video memory fragmentation distribution data and the memory fragmentation distribution data using an entropy evaluation method, and updating the pheromone distribution in the ant colony optimization algorithm. First, entropy values ​​are calculated for the video memory fragmentation distribution data and the memory fragmentation distribution data. Entropy values ​​reflect the distribution characteristics of storage space fragmentation. Then, the pheromone distribution in the ant colony optimization algorithm is updated based on the entropy calculation results. Finally, the dynamic memory pointers are recalculated using the updated pheromone distribution. This adaptive adjustment method allows the ant colony optimization algorithm to dynamically respond to changes in storage fragmentation.

[0084] The specific implementation of step S90 involves applying dynamic memory pointers in the unified storage management module of the neural network model. First, the dynamic memory pointers are loaded into the unified storage management module. Then, this module establishes a correspondence between video memory resources and system memory resources through a virtual address mapping mechanism. Finally, the coordinated scheduling of video memory resources and system memory resources is achieved based on the virtual address mapping mechanism. This unified storage management mechanism can fully utilize the characteristics of different levels of storage resources and leverage the overall advantages of the storage system.

[0085] In summary, the memory fusion allocation method for large neural network models proposed in this invention involves several innovative aspects:

[0086] 1) A hierarchical acquisition mechanism is adopted to acquire various system status data in real time, and a memory swapping cost evaluation model is constructed;

[0087] 2) Singular value decomposition is used to extract static and dynamic feature components of video memory and main memory, providing basic input for fusion allocation;

[0088] 3) Construct a complex multidimensional mapping model based on the Langevin equation to characterize the coupling relationship between system state and resource allocation;

[0089] 4) The ant colony optimization algorithm is used to solve the coupled state equations to obtain the video memory fusion allocation strategy;

[0090] 5) Construct a multi-objective storage scheduling equation set to achieve coordinated optimization of objectives such as resource utilization, access latency, fragmentation elimination, and load balancing;

[0091] 6) Apply entropy evaluation and adaptive pheromone update mechanisms to dynamically adjust resource allocation strategies to cope with changes in system state;

[0092] 7) Implement virtual address mapping and coordinated scheduling of video memory and main memory in a unified storage management module.

[0093] The following is a description of the process of establishing or deriving the equations or models involved in this invention:

[0094] 1. Derivation of the memory state evolution equation:

[0095] ;

[0096] Derivation steps:

[0097] (a) Constructing a model of the evolution of the existing state based on the reaction-diffusion equation;

[0098] (b) Introducing the diffusion term Describe the spatial distribution changes of storage resources;

[0099] (c) Introducing a Logistic growth term Describe the video memory capacity constraints;

[0100] (d) Introducing loss terms Describe the impact of resource consumption;

[0101] (e) Introducing random perturbation terms Describe the system noise.

[0102] Parameter source:

[0103] : Calculated by observing the video memory access mode; : Obtained based on statistics of video memory allocation rate; Graphics card physical parameters; : Obtained by fitting historical data; : The memory usage matrix obtained from real-time monitoring; Random noise that follows a Gaussian distribution.

[0104] 2. The memory state evolution equations all adopt the same form:

[0105] ;

[0106] 3. Derivation of the resource occupancy constraint equation:

[0107] ;

[0108] Derivation steps:

[0109] (a) Establishing the basic form of a multi-objective optimization problem ;

[0110] (b) The quadratic penalty function method is used to handle the constraints;

[0111] (c) Introducing a penalty factor Control the degree of violation of constraints.

[0112] Parameter source:

[0113] The entropy weight method is used to calculate the weights of each objective. : Construct the objective function in real time based on the system status; Determine the optimal value through cross-validation; Establish constraints based on hardware limitations.

[0114] 4. Derivation of the state transition equation:

[0115] ;

[0116] Derivation steps:

[0117] (a) Construct the Markov transition probability matrix;

[0118] (b) Introduce an energy term based on the Metropolis criterion;

[0119] (c) The state transition probability is described using the Boltzmann distribution.

[0120] Parameter source:

[0121] : Calculated through migration costs; System constant, set to 1; The initial value is set to 1, and it is gradually reduced during the optimization process.

[0122] 5. Derivation of the dynamic equilibrium equations:

[0123] ;

[0124] Derivation steps:

[0125] (a) Establish a linear state-space model;

[0126] (b) Introduce an external force term;

[0127] (c) Consider the impact of system noise.

[0128] Parameter source:

[0129] Estimation is achieved through system state transition data; Calculated based on changes in external load; Noise that follows a multidimensional normal distribution.

[0130] 6. Derivation of the resource utilization optimization equation:

[0131] ;

[0132] Derivation steps:

[0133] (a) Construction unit utilization rate item ;

[0134] (b) Introducing an equilibrium term .

[0135] Parameter source:

[0136] : Set according to the importance of the storage unit; The optimal value was determined through experiments; Calculate the correlation degree based on the topology.

[0137] 7. Derivation of the access delay optimization equation:

[0138] ;

[0139] Derivation steps:

[0140] (a) Establish a weighted access delay item;

[0141] (b) Introduce an entropy term to maintain scheduling diversity.

[0142] Parameter source:

[0143] : Statistics based on access count; : Obtained by hardware parameter measurement; Calculated based on access pattern; The experiment determined the optimal value.

[0144] 8. Derivation of the fragmentation elimination optimization equation:

[0145] ;

[0146] Derivation steps:

[0147] (a) Construct fragmentation metrics;

[0148] (b) Introduce a smoothing term to control fragment distribution.

[0149] Parameter source:

[0150] Weights are determined based on the location of the fragments; : Threshold parameters set by the system; Determined through cross-validation.

[0151] The following describes the experimental steps for obtaining parameters:

[0152] 1. Diffusion coefficient and Acquisition experiment:

[0153] Experimental step 1101: Construct a standard test dataset containing neural network models of different sizes;

[0154] Experimental step 1102: Run the neural network model with different data batch sizes and record the access patterns of video memory and RAM;

[0155] Experimental step 1103: Statistically analyze the data migration frequency between adjacent storage units;

[0156] Experimental step 1104: Fit the diffusion equation using the least squares method to obtain the diffusion coefficient.

[0157] 2. Growth rate and Acquisition experiment:

[0158] Experimental step 1201: Set up different model training stages;

[0159] Experimental step 1202: Record the storage resource allocation rate at each stage;

[0160] Experimental step 1203: Constructing a time series dataset;

[0161] Experimental step 1204: Use the Logistic regression method to fit the growth curve and obtain the growth rate parameter.

[0162] 3. Loss coefficient and Acquisition experiment:

[0163] Experimental step 1301: Run a long-term training task;

[0164] Experimental step 1302: Record the process of occupying and releasing storage resources;

[0165] Experimental step 1303: Statistical analysis of the temporal characteristics of resource recovery;

[0166] Experimental step 1304: Obtain the loss coefficient by fitting the exponential decay model.

[0167] 4. Weighting coefficients Acquisition experiment:

[0168] Experimental step 1401: Collect multiple sets of performance index data;

[0169] Experimental step 1402: Calculate the information entropy of each index;

[0170] Experimental step 1403: Calculate weight coefficients based on information entropy;

[0171] Experimental step 1404: Verify the effectiveness of the weights through cross-validation.

[0172] 5. Energy parameters Acquisition experiment:

[0173] Experimental step 1501: Construct a storage tier test environment;

[0174] Experimental step 1502: Test the migration time of data of different sizes;

[0175] Experimental step 1503: Record the energy consumption data during the migration process;

[0176] Experimental step 1504: Establish an energy calculation model.

[0177] 6. State transition matrix Acquisition experiment:

[0178] Experimental step 1601: Collect system state sequence data;

[0179] Experimental step 1602: Calculate the transition probability between adjacent states;

[0180] Experimental step 1603: Construct the state transition probability matrix;

[0181] Experimental step 1604: Optimize the transition matrix using maximum likelihood estimation.

[0182] 7. Relevance Acquisition experiment:

[0183] Experimental step 1701: Analyze the access associations of memory units;

[0184] Experimental step 1702: Constructing a correlation calculation model;

[0185] Experimental step 1703: Count the frequency of common accesses;

[0186] Experimental step 1704: Normalization process to obtain the correlation matrix.

[0187] 8. Smoothing coefficient Acquisition experiment:

[0188] Experimental step 1801: Set different smoothing coefficient values;

[0189] Experimental step 1802: Perform the defragmentation process;

[0190] Experimental step 1803: Record the results and costs of the data collection;

[0191] Experimental step 1804: Determine the optimal smoothing coefficient through binary search.

[0192] Experimental environment configuration:

[0193] 1. Hardware environment: CPU: Intel Xeon processor; GPU: NVIDIA A100 graphics card; Memory: 128GB DDR4; Video memory: 40GB HBM2.

[0194] 2. Software environment: Operating system: Ubuntu 20.04; Deep learning framework: PyTorch 1.8.0; CUDA version: 11.0; Python version: 3.8.

[0195] 3. Test dataset: Model type: ResNet, VGG, Transformer; Data size: from 100MB to 10GB; Batch size: 32 to 512.

[0196] Specifically, the principle of this invention is as follows: a multi-dimensional feature analysis-based memory fusion allocation framework is constructed, which achieves efficient storage resource management through the following key mechanisms:

[0197] First, a layered data acquisition mechanism. This invention designs a multi-layered system status data acquisition scheme, including basic indicators such as hardware layer memory utilization and memory usage, system layer performance indicators such as bandwidth utilization and access latency, and application layer behavioral characteristics such as computational load distribution and data access frequency. This layered acquisition mechanism provides comprehensive data support for subsequent optimization decisions.

[0198] Second, the feature decomposition mechanism. By processing video memory and main memory usage data using singular value decomposition (SVD), complex storage behavior is decomposed into two dimensions: static features and dynamic features. Static features reflect long-term usage patterns of storage resources, while dynamic features capture short-term fluctuations. This decomposition enables the system to more accurately predict and adapt to changes in storage demand.

[0199] Third, the fusion mapping mechanism. A fusion allocation mapping model constructed using molecular dynamics simulations can accurately describe the nonlinear relationship between storage characteristics and computational load. This model transforms the storage resource management problem into an evolutionary process of a multi-body system, obtaining the optimal allocation strategy by solving the state equations.

[0200] Fourth, the dynamic optimization mechanism. The ant colony optimization method guides storage resource allocation decisions by constructing a pheromone distribution. This method possesses strong local search capabilities and global convergence, enabling it to find optimal solutions in complex decision spaces. Simultaneously, the pheromone distribution is dynamically updated using an entropy evaluation method, ensuring the adaptability of the optimization process.

[0201] Fifth, a unified scheduling mechanism. Virtual address mapping enables unified management of video memory and system memory, reducing system complexity. This mechanism can automatically determine data storage location and migration strategies based on access patterns and load characteristics, achieving transparent management of storage resources.

[0202] In summary, this invention decomposes the complex storage management problem into multiple sub-problems, such as feature extraction, mapping construction, and optimization, and employs appropriate mathematical tools for solving them, thus ensuring the feasibility of the solution. Furthermore, the logical relationships between the various technical modules are clear, forming a complete technical system.

[0203] The following is a specific embodiment 1 of the present invention. The detailed implementation of each step in this embodiment 1 is described below: Step S10 is implemented by using a hierarchical acquisition mechanism to acquire system state data in real time during the operation of the neural network model. This includes the following indicators:

[0204] Video memory usage data : indicates at a specific time Below, the ratio of the video memory resources occupied by the neural network model to the total video memory capacity.

[0205] Memory usage data : indicates at a specific time Below, the ratio of memory resources occupied by the neural network model to the total memory capacity.

[0206] Storage bandwidth utilization data : indicates at a specific time Below, the storage bandwidth utilization of the neural network model.

[0207] Access latency data : indicates at a specific time Below, the data access latency of the neural network model.

[0208] Calculate load distribution data : indicates at a specific time The distribution of computational tasks for neural network models across different hardware resources is shown below.

[0209] Data access frequency data : indicates at a specific time Below is the distribution of access frequency of different data blocks by the neural network model.

[0210] Video memory fragmentation distribution data : indicates at a specific time Below is the distribution of fragments in video memory resources.

[0211] Memory fragmentation distribution data : indicates at a specific time Below is the distribution of fragments in memory resources.

[0212] Based on this system state data, a model for evaluating the cost of video memory swapping is constructed. :

[0213] ;

[0214] This model can quantify the exchange costs between different storage resources, providing a basis for subsequent storage resource fusion and allocation.

[0215] The specific implementation of step S20 involves using singular value decomposition (SVD) to perform multivariate matrix decomposition on the video memory usage data and system memory usage data. First, these two types of data are constructed into matrices. and :

[0216] ;

[0217] ;

[0218] Then, singular value decomposition is performed on both matrices respectively:

[0219] ;

[0220] ;

[0221] in, and It is a left singular matrix, representing the time eigenvector; and It is a singular value diagonal matrix, representing the importance of features; and It is a right singular matrix, representing the feature vector of the monitoring indicator.

[0222] Finally, based on singular value diagonal matrices and ,from and Extract the static feature components of the memory respectively Dynamic feature components of video memory Memory static feature components and memory dynamic feature components :

[0223] ;

[0224] ;

[0225] ;

[0226] ;

[0227] in, and This refers to the number of feature dimensions selected, which can be adjusted to control the proportion of static and dynamic features extracted.

[0228] The specific implementation of step S30 is based on constructing a fusion assignment mapping model according to the Langevin equation. The Langevin equation describes the equation of motion of a fully docked hysteresis system:

[0229] ;

[0230] in, It is the system's state vector. It is the damping coefficient. It is the potential energy function. It is a random force term that follows a normal distribution.

[0231] In this problem, the static characteristic components of the video memory will be... Dynamic feature components of video memory Memory static feature components and memory dynamic feature components Composition of state vector This is input into the Langevin equation. Potential energy function. Then calculate the load distribution data. Data access frequency data and storage bandwidth utilization data Build:

[0232] ;

[0233] in, , and These are the weighting coefficients for the corresponding features.

[0234] Finally, solving the Langevin equation yields the fusion assignment mapping model. :

[0235] ;

[0236] This model can describe the multidimensional and complex relationship of video memory fusion and allocation.

[0237] The specific implementation of step S40 is based on constructing a coupled state equation set based on the fusion allocation mapping model. First, the model... The output results are projected onto the video memory space and the main memory space respectively, resulting in:

[0238] Static memory allocation feature matrix ;

[0239] Dynamic memory allocation feature matrix ;

[0240] Memory static allocation feature matrix ;

[0241] Memory dynamic allocation feature matrix ;

[0242] Then, a system of coupled state equations is constructed, including:

[0243] Memory state evolution equation:

[0244] ;

[0245] Memory state evolution equation:

[0246] ;

[0247] Resource occupancy constraint equation:

[0248] ;

[0249] State transition equation:

[0250] ;

[0251] Dynamic equilibrium equations:

[0252] ;

[0253] The meanings of the variables and parameters in each equation are as follows:

[0254] and These represent the states of video memory and main memory, respectively. and It is the diffusion coefficient. and It is the growth rate. and It is the capacity limit. and It is the loss coefficient. and It is an occupancy matrix. and It is a random disturbance term.

[0255] It is the objective function. These are constraints. These are weighting coefficients. It is a punishment factor.

[0256] It is the state transition probability. It is the transfer of energy. It is Boltzmann's constant. It is a temperature parameter.

[0257] It is the system state vector. It is the state transition matrix. It is an external force. It is system noise.

[0258] These equations are combined into a system of coupled state equations, and with , , and With these as inputs, a complete state description system can be obtained.

[0259] The specific implementation of step S50 involves using an ant colony optimization algorithm to solve the coupled state equations. First, the ant colony size needs to be initialized. Maximum number of iterations and pheromone distribution :

[0260] ;

[0261] in, This is the initial cost function value.

[0262] Then, the coupled state equations are used as constraints for the ant colony optimization algorithm:

[0263] ;

[0264] ;

[0265] in, These are the various objective functions. These are constraints.

[0266] Next, the ant colony optimization algorithm is used to solve this optimization problem, and a memory fusion allocation strategy is obtained. :

[0267] ;

[0268] Finally, according to Allocate dynamic storage pointers for the parameters of the neural network model.

[0269] The specific implementation of step S60 is to construct a memory scheduling feature matrix and a memory scheduling feature vector based on dynamic memory pointers. First, a memory scheduling feature matrix is ​​established based on dynamic memory pointers. and memory scheduling feature matrix :

[0270] ;

[0271] ;

[0272] Then, joint eigenvalue decomposition is performed on these two matrices to obtain the memory scheduling eigenvector. and memory scheduling feature vector .

[0273] In addition, the access frequency of statistical storage units is used to construct an access frequency vector. Calculate the data block size distribution and construct the data block size vector. ;Based on the evaluation model of video memory swapping cost Construct a hierarchical migration cost matrix .

[0274] These eigenvectors and matrices provide the foundational data for subsequent multi-objective storage scheduling optimization.

[0275] The specific implementation of step S70 is to construct a multi-objective storage scheduling equation set and perform resource scheduling optimization.

[0276] Resource utilization optimization equation:

[0277] ;

[0278] in, It is the first The actual usage of each storage unit It is the first The capacity of each storage unit These are weighting coefficients. It is the balance coefficient. This refers to the inter-unit correlation. This equation is used to maximize the overall utilization efficiency of video memory.

[0279] Access latency optimization equation:

[0280] ;

[0281] in, It refers to the frequency of access. It's an access delay. It is the access probability. This is the entropy weighting coefficient. This equation is used to reduce the average time cost of data access.

[0282] Fragmentation elimination optimization equation:

[0283] ;

[0284] in, It is the first The size of each fragment, It is the minimum fragmentation threshold. It is fragmented weight. It is a smoothing coefficient. This equation is used to merge discrete memory fragments.

[0285] Migration cost optimization equation:

[0286] ;

[0287] in, It is the cost of migration. It is a migration decision variable. It's bandwidth usage. It is the maximum bandwidth. This is the bandwidth weight. This equation is used to balance the performance gains and resource overhead of data migration.

[0288] Load balancing optimization equation:

[0289] ;

[0290] in, It is the first The load value of each node, This is the average load value. It is a node The set of neighbors, This is the local balance coefficient. This equation is used to achieve dynamic balance of computing load across different storage tiers.

[0291] By solving these multi-objective optimization equations, a fully optimized storage resource scheduling scheme can be obtained.

[0292] The specific implementation of step S80 is to analyze the memory fragmentation distribution data using the entropy evaluation method. and memory fragmentation distribution data And update the pheromone distribution of the ant colony optimization algorithm.

[0293] First, calculate the entropy values ​​of the video memory fragmentation distribution data and the memory fragmentation distribution data:

[0294] ;

[0295] ;

[0296] in, and They are and The probability distribution of each data block. The larger the entropy value, the more uniform the fragment distribution.

[0297] Then, the pheromone distribution in the ant colony optimization algorithm is updated based on the entropy results:

[0298] ; ;

[0299] Finally, the dynamic memory pointers are recalculated using the updated pheromone distribution. This adaptive adjustment method allows the ant colony optimization algorithm to dynamically respond to changes in memory fragmentation.

[0300] The specific implementation of step S90 involves applying dynamic memory pointers in the unified memory management module of the neural network model. First, the dynamic memory pointers are loaded into the unified memory management module. Then, this module establishes the correspondence between video memory resources and system memory resources through a virtual address mapping mechanism.

[0301] ;

[0302] ;

[0303] in, It is a virtual address space. and These are the corresponding video memory address and memory address, respectively.

[0304] Finally, this virtual address mapping mechanism enables coordinated scheduling of video memory and system memory resources. This unified storage management mechanism can fully utilize the characteristics of storage resources at different levels and leverage the overall advantages of the storage system.

[0305] To better understand and implement this invention, a specific application scenario is provided below as Example 2: An internet company developed an intelligent customer service system based on a large neural network model. This system can automatically respond to various user inquiries. To ensure the system's response speed and user experience, the company deployed the neural network model on a cloud computing platform and adopted the large neural network model memory fusion allocation method proposed in this invention.

[0306] First, the company's engineering team monitored various system status data in real time during the operation of the neural network model on the cloud platform. Specifically, this included:

[0307] 1. Video memory usage data At a certain time t, the neural network model occupies 60GB of video memory, and the total video memory capacity is 100GB. What is the video memory utilization rate? .

[0308] 2. Memory usage data At a certain time t, the neural network model occupies 80GB of memory resources, and the total memory capacity is 128GB. What is the memory utilization rate? .

[0309] 3. Storage bandwidth utilization data At a certain time t, the storage bandwidth utilization rate of the neural network model is 80%.

[0310] 4. Access latency data At a certain time t, the average data access latency of the neural network model is 0.2ms.

[0311] 5. Calculate load distribution data At a certain time t, the computational tasks of the neural network model are distributed across the four GPU servers at a ratio of 30%, 25%, 20%, and 15%, respectively.

[0312] 6. Data access frequency At a certain time t, the frequency distribution of access to different data blocks by the neural network model is as follows: 30% of the data blocks are accessed more frequently, 50% of the data blocks are accessed less frequently, and the remaining 20% ​​of the data blocks are accessed at a moderate frequency.

[0313] 7. Memory fragmentation distribution data At a certain time t, there are a large number of discrete small fragments in the video memory resources, and the fragments are relatively scattered.

[0314] 8. Memory fragmentation distribution data At a certain moment t, there are relatively few fragments in memory resources, mainly concentrated in a few specific areas.

[0315] Based on the real-time monitoring data above, the engineers constructed a model for evaluating the cost of video memory swapping. This is used to quantify the exchange cost between different storage resources. (Model) The specific form is as follows:

[0316] ;

[0317] Among them, each coefficient reflects the relative contribution to the exchange cost.

[0318] Next, the engineering team used the singular value decomposition method to analyze the memory usage data matrix. and memory usage data matrix Multivariate matrix decomposition was performed to extract the static feature components of the video memory. Dynamic feature components of video memory Memory static feature components and memory dynamic feature components The specific process is as follows:

[0319] First, the video memory usage data and the main memory usage data are constructed into matrices respectively. and :

[0320] ;

[0321] ;

[0322] Then, respectively and Performing singular value decomposition, we obtain:

[0323] ;

[0324] ;

[0325] in, and It is a left singular matrix, representing the time eigenvector; and It is a singular value diagonal matrix, representing the importance of features; and It is a right singular matrix, representing the feature vector of the monitoring indicator.

[0326] Finally, based on and ,from and Extract the static feature components of the memory respectively Dynamic feature components of video memory Memory static feature components and memory dynamic feature components .

[0327] With these feature components, the engineers then constructed a fusion assignment mapping model. This model employs the Langevin equation from molecular dynamics, treating multidimensional characteristics such as GPU memory, system memory resources, and computational load as the dynamic process of a fully docked hysteresis system. The specific form of the Langevin equation is as follows:

[0328] ;

[0329] in, It is the system's state vector. It is the damping coefficient. It is a random force term that follows a normal distribution. Potential energy function Then by , and Constructed from:

[0330] ;

[0331] By solving this Langevin equation, engineers obtained the fusion allocation mapping model. It describes the complex relationship between video memory, memory resources, and computing load.

[0332] Next, the engineering team based their work on the fusion allocation mapping model. The output of this equation constructs a system of coupled state equations. Specifically, it includes:

[0333] 1. Evolution equation of memory state:

[0334] ;

[0335] 2. Memory state evolution equation:

[0336] ;

[0337] 3. Resource occupancy constraint equation:

[0338] ;

[0339] 4. State transition equation:

[0340] ;

[0341] 5. Dynamic equilibrium equations:

[0342] ;

[0343] The variables and parameters in each equation have the following meanings:

[0344] and These represent the states of video memory and main memory, respectively. and It is the diffusion coefficient; and It is the growth rate; and It is the maximum capacity; and It is the loss coefficient; and It is an occupancy matrix; and It is a random disturbance term.

[0345] and It is the objective function. These are constraints.

[0346] It is the state transition probability. It involves transferring energy.

[0347] It is the system state vector. It is the state transition matrix. It is an external force. It is system noise.

[0348] With this set of coupled state equations, the engineering team used an ant colony optimization algorithm to solve it. The initial parameters of the algorithm were set as follows: ant colony size... Maximum number of iterations Initial pheromone distribution Through iterative calculations, a final strategy for merging and allocating video memory was obtained. .

[0349] according to Engineers established a memory scheduling feature matrix in the unified memory management module of the neural network model. and memory scheduling feature matrix Subsequently, joint eigenvalue decomposition was performed on the two matrices to obtain the memory scheduling eigenvector. and memory scheduling feature vector In addition, they also analyzed the access frequency of storage units. The data block size distribution was calculated. And based on the video memory swapping cost evaluation model A hierarchical migration cost matrix was constructed. .

[0350] With these eigenvectors and matrices, the engineers then constructed a set of multi-objective memory scheduling equations, including:

[0351] 1. Resource utilization optimization equation:

[0352] ;

[0353] 2. Access latency optimization equation:

[0354] ;

[0355] 3. Fragmentation elimination optimization equation:

[0356] ;

[0357] 4. Migration cost optimization equation:

[0358] ;

[0359] 5. Load balancing optimization equation:

[0360] ;

[0361] The variable and parameter values ​​in each equation are shown below:

[0362] Resource utilization optimization equation:

[0363] , , , , , , , ;

[0364] Access latency optimization equation:

[0365] , , , , , , , , , , , , , , , , , , , , , , , ;

[0366] Fragmentation elimination optimization equation:

[0367] , , ;

[0368] Migration cost optimization equation:

[0369] , , , , , , , , ;

[0370] Load balancing optimization equation:

[0371] , , ;

[0372] By solving these multi-objective optimization equations, the engineering team obtained a fully optimized storage resource scheduling scheme.

[0373] Finally, this scheduling scheme was applied to the unified memory management module of the neural network model. This module established a mapping between video memory resources and system memory resources through a virtual address mapping mechanism.

[0374] ;

[0375] ;

[0376] in, It is a virtual address space. and These are the corresponding video memory address and memory address, respectively.

[0377] Through this virtual address mapping, the unified storage management module achieves coordinated scheduling of video memory and system memory resources, making full use of the characteristics of different levels of storage resources and maximizing the overall performance of neural network models on the cloud platform.

[0378] After a period of operation, the intelligent customer service system performed very well on the cloud platform, greatly satisfying the enterprise. The system's response speed improved by over 30%, and user feedback also showed significant improvement. Simultaneously, the system's storage resource utilization was greatly enhanced, and the computational load was allocated more rationally. Through the large-scale neural network model memory fusion allocation method proposed in this invention, the enterprise successfully overcame the storage system performance bottleneck, laying a solid foundation for the vigorous development of its intelligent customer service business.

[0379] It should be noted that the variables involved in the description of this invention are explained in detail in Table 1 below:

[0380] Table 1. Variable Explanation Table

[0381]

[0382] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for fusion allocation of video memory for large neural network models, characterized in that, include: S10. Acquire system state data in real time during the operation of the neural network model, and construct a video memory exchange cost evaluation model based on the system state data; S20. Perform multivariate matrix decomposition on the video memory usage data and memory usage data to extract the static feature components of video memory, the dynamic feature components of video memory, the static feature components of memory, and the dynamic feature components of memory. S30. Construct a multi-dimensional mapping model between the static feature components of video memory, the dynamic feature components of video memory, the static feature components of memory, the dynamic feature components of memory, and the computational load distribution data, data access frequency data, and storage bandwidth utilization data, denoted as the fusion allocation mapping model. S40. Calculate the static allocation feature matrix, dynamic allocation feature matrix, static memory allocation feature matrix, and dynamic memory allocation feature matrix using the fusion allocation mapping model, and establish a set of coupled state equations for the four allocation feature matrices. S50. Solve the coupled state equations to obtain the video memory fusion allocation strategy, and allocate dynamic storage pointers to the neural network model parameters based on the video memory fusion allocation strategy. S60. Establish the video memory scheduling feature matrix and the memory scheduling feature matrix based on the dynamic storage pointer. Obtain the video memory scheduling feature vector and the memory scheduling feature vector through joint feature decomposition. At the same time, calculate the storage unit access frequency vector, the data block size vector, and the hierarchical migration cost matrix. S70. Construct a set of multi-objective storage scheduling equations to optimize resource scheduling; S80. Analyze the video memory fragmentation distribution data and memory fragmentation distribution data, update the pheromone distribution of the ant colony optimization algorithm, and recalculate the dynamic storage pointers. S90. Apply dynamic storage pointers in the unified storage management module of the neural network model. The unified storage management module realizes the coordinated scheduling of video memory resources and memory resources through the virtual address mapping mechanism.

2. The method according to claim 1, characterized in that, The specific implementation of step S20 includes: Step 201: Construct the video memory usage data matrix and the memory usage data matrix into rows according to time series sampling points and into columns according to monitoring indicators to form a matrix to be decomposed; Step 202: Perform singular value decomposition on the matrix to be decomposed to obtain the left singular matrix representing the time feature vector, the singular value diagonal matrix representing the feature importance, and the right singular matrix representing the monitoring indicator feature vector. Step 203: Perform feature decomposition on the video memory usage data matrix based on the singular value diagonal matrix to extract the static and dynamic feature components of the video memory. Step 204: Perform feature decomposition on the memory usage data matrix based on the singular value diagonal matrix to extract the static and dynamic feature components of memory.

3. The method according to claim 1, characterized in that, The specific implementation of step S30 includes: Step 301: Construct a fusion allocation mapping model based on the Langevin equation, which includes inertial terms, damping terms, potential energy terms, and stochastic force terms. Step 302: Input the static feature components of video memory, the dynamic feature components of video memory, the static feature components of memory, and the dynamic feature components of memory as state vectors into the Langevin equation; Step 303: Use the calculated load distribution data, data access frequency data, and storage bandwidth utilization data to construct the potential energy function; Step 304: Solve the Langevin equation to obtain the multidimensional mapping relationship model.

4. The method according to claim 1, characterized in that, The specific implementation of step S40 includes: Step 401: Project the output of the fusion allocation mapping model onto the video memory space and the memory space respectively to obtain the video memory static allocation feature matrix, the video memory dynamic allocation feature matrix, the memory static allocation feature matrix, and the memory dynamic allocation feature matrix. Step 402: Combine the video memory state evolution equation, memory state evolution equation, resource occupancy constraint equation, state transition equation, and dynamic equilibrium equation into a coupled state equation set. Step 403: Input the four assigned feature matrices into the coupled state equation set to form a complete state description system.

5. The method according to claim 1, characterized in that, The specific implementation of step S50 includes: Step 501: Initialize ant colony size, maximum number of iterations, and pheromone distribution; Step 502: Use the coupled state equations as constraints for the ant colony optimization algorithm. Step 503: Solve the coupled state equations using the ant colony optimization algorithm to obtain the video memory fusion allocation strategy; Step 504: Allocate dynamic storage pointers to the neural network model parameters according to the video memory fusion allocation strategy.

6. The method according to claim 1, characterized in that, The specific implementation of step S60 includes: Step 601: Construct the video memory scheduling feature matrix and the memory scheduling feature matrix based on dynamic memory pointers; Step 602: Perform joint eigenvalue decomposition on the video memory scheduling feature matrix and the memory scheduling feature matrix to obtain the video memory scheduling feature vector and the memory scheduling feature vector; Step 603: Calculate the access frequency of storage units and construct a storage unit access frequency vector; Step 604: Calculate the data block size distribution and construct the data block size vector; Step 605: Construct a hierarchical migration cost matrix based on the video memory swapping cost evaluation model.

7. The method according to claim 1, characterized in that, The specific implementation of step S70 includes: Step 701: Construct resource utilization optimization equations based on video memory scheduling feature vectors, memory scheduling feature vectors, and data block size vectors; Step 702: Construct an access latency optimization equation based on the storage unit access frequency vector, data storage location matrix, and access latency time data; Step 703: Construct a fragmentation elimination optimization equation based on video memory fragmentation distribution data, memory fragmentation distribution data, and minimum storage block threshold; Step 704: Construct a migration cost optimization equation based on the hierarchical migration cost matrix, migration data scale matrix, and storage bandwidth utilization data; Step 705: Construct a load balancing optimization equation based on the calculated load distribution data and load balancing coefficient.

8. The method according to claim 1, characterized in that, The specific implementation of step S80 includes: Step 801: Calculate the entropy value of the video memory fragmentation distribution data and the memory fragmentation distribution data; Step 802: Update the pheromone distribution in the ant colony optimization algorithm based on the entropy calculation results; Step 803: Recalculate the dynamic storage pointer using the updated pheromone distribution.

9. The method according to claim 1, characterized in that, The specific implementation of step S90 includes: Step 901: Load the dynamic storage pointer into the unified storage management module of the neural network model; Step 902: The unified storage management module establishes the correspondence between video memory resources and memory resources through a virtual address mapping mechanism; Step 903: Implement coordinated scheduling of video memory resources and main memory resources based on the virtual address mapping mechanism.

10. The method according to claim 1, characterized in that, The coupled state equation set includes the video memory state evolution equation, the memory state evolution equation, the resource occupancy constraint equation, the state transition equation, and the dynamic equilibrium equation.

Images