Large model operation environment adaptive deployment method based on strategy optimization
By constructing a runtime environment graph structure and a topology gating mechanism driven by persistent coherence barcodes, the problem of instability in deploying large models in heterogeneous environments is solved, achieving matching between the computing structure and the runtime environment and improving stability.
Patent Information
- Application Number
- CN202610177039.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-02-06
- Publication Date
- 2026-05-19
AI Technical Summary
Existing large-scale model deployment methods struggle to cope with changes in resource status in heterogeneous operating environments, leading to uneven distribution of computing resources and mismatch between model structure and operating conditions. Furthermore, there is a lack of effective correlation between system modeling of the operating environment topology and model structure adjustment.
By constructing a runtime environment graph structure and introducing a topology gating mechanism driven by persistent homology barcodes, structural path filtering and heterogeneous rank tensor decomposition reparameterization processing are performed on the large model computation structure to form an adaptive deployment scheme, ensuring that the computation structure matches the topology constraints of the runtime environment.
It enables stable deployment of large-scale model computing structures in heterogeneous environments, improves node load balancing, communication efficiency and operational stability, and enhances the controllability and consistency of the deployment process.
Smart Images
Figure CN122069189A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence model deployment technology, and in particular to an adaptive deployment method for a large model runtime environment based on strategy optimization. Background Technology
[0002] With the development of artificial intelligence technology, large models are widely used in fields such as natural language processing, computer vision, and multimodal analysis. These models typically feature large parameter scales, complex computational structures, and high resource consumption, placing high demands on computing power, storage resources, and communication conditions during their operation. In practical applications, large models often need to be deployed in heterogeneous environments composed of multiple computing nodes. These nodes exhibit significant differences in computing capacity, memory capacity, network bandwidth, and communication latency, which complicates the model deployment process considerably.
[0003] Existing large-scale model deployment methods mostly employ static configuration strategies, meaning the model structure and parameter forms are pre-determined before deployment and remain unchanged during runtime. This approach typically assumes a stable operating environment and struggles to cope with changes in resource status during operation. When the operating environment changes, static deployment can lead to uneven distribution of computing resources and mismatches between model structure and operating conditions. Furthermore, some existing technologies reduce model size through model pruning, parameter compression, or low-rank decomposition; however, these methods primarily address the overall model level and do not fully consider the differences in computational complexity and resource requirements among different structural regions within the model.
[0004] Meanwhile, in complex operating environments, the connections and communication characteristics between computing nodes constitute a topological network. Existing deployment methods often fail to systematically model this type of topology and lack the technical means to combine the topological characteristics of the operating environment with the model structure adjustment process, resulting in a lack of effective correlation between model structure adjustments and the operating environment. Therefore, how to rationally construct and adjust the computational structure of large models under the constraints of the operating environment topology has become a problem to be solved in existing technologies. Summary of the Invention
[0005] One objective of this invention is to propose an adaptive deployment method for a large model runtime environment based on strategy optimization. This invention introduces a topology gating mechanism driven by runtime environment graph structure modeling and persistent coherence barcodes. It performs structural path filtering and heterogeneous rank tensor decomposition reparameterization on the large model computation structure to construct an adaptive deployment process that matches the runtime environment state, forming a large model runtime scheme with constrained structure, reconfigurable parameters, and executable deployment.
[0006] An adaptive deployment method for a large model runtime environment based on strategy optimization according to an embodiment of the present invention includes the following steps:
[0007] S1. Collect the computing power capacity, video memory capacity, inter-node communication bandwidth, and communication latency values of the computing nodes in the operating environment, and construct the operating environment graph structure.
[0008] S2. Construct a set of nodes and a set of edges in the runtime environment graph structure to form a topological computation graph, and then configure state feature vectors for the nodes and edges;
[0009] S3. Perform multi-scale filtering on the topology computation graph to generate a topology evolution sequence. Perform persistent cohomology computation on the topology evolution sequence to obtain a persistent cohomology barcode set. Map the bar persistence intervals and quantity distribution in the persistent cohomology barcode set to a topology gating vector. The topology gating vector limits the structural path activation state and operator type selection state in the large model.
[0010] S4. Generate a set of candidate computational structures under the constraint of topological gating vectors;
[0011] S5. For the set of weight tensors in the candidate computational structure set, according to the topological complexity distribution represented by the topological gate vector, the same weight tensor is divided into several index intervals, different rank parameters are configured for different index intervals, and topology-induced heterogeneous rank tensor decomposition is performed to generate a set of decomposed kernel tensors.
[0012] S6. Perform tensor reparameterization on the candidate computational structure set based on the decomposed kernel tensor set to form a large model computational structure;
[0013] S7. Map the large model computation structure to the corresponding computation nodes of the runtime environment graph structure and then complete the deployment and execution.
[0014] Optionally, S1 specifically includes:
[0015] Assign a unique computing node identifier to each computing node in the operating environment, read the computing power capacity value and video memory capacity value corresponding to each computing node identifier, and construct a node status record table using the computing node identifier, computing power capacity value, and video memory capacity value.
[0016] For any two computing nodes, a communication link record is established. The communication link record includes the identifier of the sending computing node, the identifier of the receiving computing node, the communication bandwidth value, and the communication delay value. Based on this, an edge state record table is constructed.
[0017] A set of node identifiers for the runtime environment graph structure is generated using the compute node identifiers in the node state record table, and a set of edge identifiers for the runtime environment graph structure is generated using the sender compute node identifiers and receiver compute node identifiers in the edge state record table.
[0018] Establish a one-to-one index mapping relationship between the node identifier set and the node state record table, and establish a one-to-one index mapping relationship between the edge identifier set and the edge state record table to form the runtime environment graph structure.
[0019] Optionally, S2 specifically includes:
[0020] Read the node identifier set and edge identifier set in the runtime environment graph structure, and build a data structure container for the topology computation graph;
[0021] Write a node entry for each node identifier in the data structure container. The node entry contains the node identifier and a state feature vector consisting of computing power capacity value and video memory capacity value.
[0022] Write an edge entry for each edge identifier in the data structure container. The edge entry contains the identifier of the sending end computing node, the identifier of the receiving end computing node, and a state feature vector composed of communication bandwidth value and communication delay value, thus completing the construction of the topology computing graph.
[0023] Optionally, S3 specifically includes:
[0024] Read the data structure container of the topological computation graph, extract node entries and edge entries to form an initial graph snapshot;
[0025] Define a filter threshold sequence, which contains at least three threshold positions;
[0026] At each threshold position, threshold filtering is performed on the communication bandwidth and communication delay values in the edge entries. Edge entries that meet the threshold relationship are retained, and edge entries that do not meet the threshold relationship are deleted, forming a filter graph snapshot at the corresponding threshold position.
[0027] Arrange the snapshots of each filter graph in order of the filter threshold sequence to form a topological evolution sequence;
[0028] Persistent homology calculation is performed on the topological evolution sequence. Homology generation events and homology extinction events are calculated at each filter map snapshot. The threshold position corresponding to the homology generation event is recorded as the bar start point, and the threshold position corresponding to the homology extinction event is recorded as the bar end point, forming a persistent homology barcode set.
[0029] Read the bar start and end points of the persistent homology barcode set one by one to generate the bar duration interval, and count the number of bars within the same homology dimension to generate the quantity distribution;
[0030] Perform concatenation encoding on the bar-shaped continuous interval and quantity distribution to generate a topology-gated vector. The topology-gated vector contains a structural path-gated component group and an operator type-gated component group.
[0031] Write the structure path gated component group into the structure path activation state table, and write the operator type gated component group into the operator type selection state table.
[0032] Optionally, the topology-gated vector constrains the structural path activation state and operator type selection state in the large model, specifically including:
[0033] Read the topology gate vector, parse the topology gate vector according to the preset vector position, and obtain the structural path gate component group and the operator type gate component group;
[0034] Read the structure path identifier table that records structure path identifiers in the large model. The structure path identifier table contains structure path identifiers and structure path position indexes. Read the component values of the structure path gated component group one by one. Locate the structure path identifier in the structure path identifier table according to the structure path position index. Write the component values and structure path identifiers into the structure path activation state table. The structure path activation state table contains structure path identifiers and activation mark values.
[0035] Read the operator type identifier table that records operator type identifiers in the large model. The operator type identifier table contains operator type identifiers and operator type position indexes. Read the component values of the operator type gated component group component by component. Locate the operator type identifier in the operator type identifier table according to the operator type position index. Write the component values and operator type identifiers into the operator type selection status table. The operator type selection status table contains operator type identifiers and selection flag values.
[0036] Read the structure path activation status table, filter out structure path identifiers whose activation flag values do not meet the activation conditions, and retain structure path identifiers that meet the activation conditions to form the structure path allowed set; read the operator type selection status table, filter out operator type identifiers whose selection flag values do not meet the selection conditions, and retain operator type identifiers that meet the selection conditions to form the operator type allowed set.
[0037] Within the scope of the allowed set of structural paths, read the structural paths in the large model computational structure, construct a candidate computational structure set, and within the scope of the allowed set of operator types, read the operator types, construct operator candidate set constraints.
[0038] Optionally, S4 specifically includes:
[0039] Read the structure path activation status table and operator type selection status table; read the structure path identifier table and operator type identifier table in the large model calculation structure.
[0040] In the structure path activation status table, filter structure path identifiers whose activation flag values meet the activation conditions to form a candidate structure path identifier set;
[0041] In the operator type selection status table, select operator type identifiers whose flag values meet the selection criteria to form a candidate operator type identifier set;
[0042] Read the structural path identifiers one by one from the candidate structural path identifier set, extract the corresponding structural path substructures from the large model calculation structure according to the structural path position index in the structural path identifier table, and write them into the candidate calculation structure set entries.
[0043] Write the candidate operator type identifier set into the candidate computation structure set entries to form the candidate computation structure set.
[0044] Optionally, S5 specifically includes:
[0045] Read the set of weight tensors in the candidate computation structure set, assign a weight tensor identifier to each weight tensor, and then establish an index relationship between the weight tensor identifier and the weight tensor storage location.
[0046] Read the topology gate vector, parse the topology gate vector according to the preset vector position, extract the topology complexity components associated with the weight tensor identifier, and form a topology complexity distribution record at the weight tensor level;
[0047] For each weight tensor, based on the dimensional order and storage layout of the weight tensor, the index space of the weight tensor is divided into several non-overlapping index intervals. The index intervals are marked according to the starting and ending positions of the indexes, forming a set of index intervals.
[0048] Read the index interval markers for each interval of the index interval set, read the topology complexity components of the corresponding index intervals from the topology complexity distribution record, map the topology complexity components to rank parameter values, and form an index interval rank parameter table. The index interval rank parameter table contains index interval markers and rank parameter values.
[0049] According to the index interval rank parameter table, tensor decomposition operation is performed on the weight tensor in each index interval. The tensor decomposition operation independently generates a corresponding decomposition kernel tensor group for each index interval. The decomposition kernel tensor group contains kernel tensors with the same value as the rank parameter.
[0050] The decomposition kernel tensor groups generated for the same weight tensor in different index intervals are aggregated according to the index interval label to form the decomposition kernel tensor set corresponding to the weight tensor. The decomposition kernel tensor set is then written into the candidate computation structure set.
[0051] Optionally, S6 specifically includes:
[0052] Read the candidate computation structure set, read the structure path substructure in the candidate computation structure set one by one, and then locate the weight tensor placeholder corresponding to the weight tensor identifier in each structure path substructure.
[0053] Read the set of decomposition kernel tensors corresponding to the weight tensor identifiers. The set of decomposition kernel tensors contains several groups of decomposition kernel tensors organized by index interval labels.
[0054] For each weight tensor occupancy position, the decomposition kernel tensor group of the corresponding index interval is read sequentially according to the index interval marking order. Based on the original dimension order of the weight tensor and the index interval boundary, the decomposition kernel tensor group is mapped to the corresponding index interval position to form an index interval-level reparameterized weight sub-tensor.
[0055] Several index interval-level reparameterized weight sub-tensors formed under the same weight tensor identifier are concatenated according to the index interval boundary order to generate a reparameterized weight tensor.
[0056] Write the reparameterized weight tensor into the placeholder position of the weight tensor in the structure path substructure, and replace the corresponding weight tensor in the candidate computation structure set.
[0057] Repeatedly perform the weight tensor placeholder replacement operation on all structural path substructures in the candidate computational structure set to form a large model computational structure containing reparameterized weight tensors.
[0058] The beneficial effects of this invention are:
[0059] (1) This invention constructs a runtime environment graph structure and introduces a topology gating vector based on persistent coherence barcodes to transform the computing power capacity, memory capacity and communication relationship in the runtime environment into explicit topology constraints. These topology constraints are then introduced into the generation process of the large model computing structure, ensuring that the construction process of the computing structure is always within the topology constraints of the runtime environment, thus avoiding deployment instability caused by inconsistencies between the computing structure and the runtime environment.
[0060] (2) This invention divides the weight tensors in the candidate computational structure set into index intervals by topology-induced heterogeneous rank tensor decomposition and configures different rank parameters in different index intervals, so that the weight tensors form distinct structural expressions inside, effectively characterizing the differences in parameter organization at different structural regions of the model, and providing a more refined parameter configuration basis for subsequent tensor reparameterization and computational structure generation.
[0061] (3) This invention connects the model structure adjustment process with the deployment execution process in an orderly manner through steps such as structural path screening, decomposition kernel tensor generation and tensor reparameterization, so that the large model computing structure can be mapped to the computing nodes in the runtime environment in a clear data structure form, thereby enhancing the controllability and consistency of the deployment process in terms of structure organization, parameter loading and execution mapping. Attached Figure Description
[0062] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention, but do not constitute a limitation thereof. In the drawings:
[0063] Figure 1 This is a flowchart of an adaptive deployment method for a large model runtime environment based on strategy optimization proposed in this invention.
[0064] Figure 2 This is a schematic diagram of the topology gating generation process based on persistent homology barcodes, which is a large model runtime environment adaptive deployment method based on strategy optimization proposed in this invention.
[0065] Figure 3 This diagram illustrates the topology-induced heterogeneous rank tensor decomposition process of a large model runtime environment adaptive deployment method based on policy optimization proposed in this invention. Detailed Implementation
[0066] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.
[0067] refer to Figures 1-3 An adaptive deployment method for a large model runtime environment based on strategy optimization includes the following steps:
[0068] S1. Collect the computing power capacity, video memory capacity, inter-node communication bandwidth, and communication latency values of the computing nodes in the operating environment, and construct the operating environment graph structure.
[0069] S2. Construct a set of nodes and a set of edges in the runtime environment graph structure to form a topological computation graph, and then configure state feature vectors for the nodes and edges;
[0070] S3. Perform multi-scale filtering on the topology computation graph to generate a topology evolution sequence. Perform persistent cohomology computation on the topology evolution sequence to obtain a persistent cohomology barcode set. Map the bar persistence intervals and quantity distribution in the persistent cohomology barcode set to a topology gating vector. The topology gating vector limits the structural path activation state and operator type selection state in the large model.
[0071] S4. Generate a set of candidate computational structures under the constraint of topological gating vectors;
[0072] S5. For the set of weight tensors in the candidate computational structure set, according to the topological complexity distribution represented by the topological gate vector, the same weight tensor is divided into several index intervals, different rank parameters are configured for different index intervals, and topology-induced heterogeneous rank tensor decomposition is performed to generate a set of decomposed kernel tensors.
[0073] S6. Perform tensor reparameterization on the candidate computational structure set based on the decomposed kernel tensor set to form a large model computational structure;
[0074] S7. Map the large model computation structure to the corresponding computation nodes of the runtime environment graph structure and then complete the deployment and execution.
[0075] In this embodiment, S1 specifically includes:
[0076] Assign a unique computing node identifier to each computing node in the operating environment, read the computing power capacity value and video memory capacity value corresponding to each computing node identifier, and construct a node status record table using the computing node identifier, computing power capacity value, and video memory capacity value.
[0077] For any two computing nodes, a communication link record is established. The communication link record includes the identifier of the sending computing node, the identifier of the receiving computing node, the communication bandwidth value, and the communication delay value. Based on this, an edge state record table is constructed.
[0078] A set of node identifiers for the runtime environment graph structure is generated using the compute node identifiers in the node state record table, and a set of edge identifiers for the runtime environment graph structure is generated using the sender compute node identifiers and receiver compute node identifiers in the edge state record table.
[0079] Establish a one-to-one index mapping relationship between the node identifier set and the node state record table, and establish a one-to-one index mapping relationship between the edge identifier set and the edge state record table to form the runtime environment graph structure.
[0080] In this embodiment, S2 specifically includes:
[0081] Read the node identifier set and edge identifier set in the runtime environment graph structure, and build a data structure container for the topology computation graph;
[0082] Write a node entry for each node identifier in the data structure container. The node entry contains the node identifier and a state feature vector consisting of computing power capacity value and video memory capacity value.
[0083] Write an edge entry for each edge identifier in the data structure container. The edge entry contains the identifier of the sending end computing node, the identifier of the receiving end computing node, and a state feature vector composed of communication bandwidth value and communication delay value, thus completing the construction of the topology computing graph.
[0084] In this embodiment, S3 specifically includes:
[0085] Read the data structure container of the topological computation graph, extract node entries and edge entries to form an initial graph snapshot;
[0086] A filtering threshold sequence is set, which contains at least three threshold positions. Specifically, the filtering threshold sequence includes reading the communication bandwidth value set and communication delay value set of all edge entries in the runtime environment graph structure, sorting the communication bandwidth value set and communication delay value set respectively to obtain an ordered value sequence, selecting several value positions in the ordered value sequence according to a preset segmentation rule to form a threshold value set, arranging the threshold value set according to the value size rule to generate the filtering threshold sequence.
[0087] At each threshold position, threshold filtering is performed on the communication bandwidth and communication delay values in the edge entries. Edge entries that meet the threshold relationship are retained, and edge entries that do not meet the threshold relationship are deleted, forming a filter graph snapshot at the corresponding threshold position.
[0088] Arrange the snapshots of each filter graph in order of the filter threshold sequence to form a topological evolution sequence;
[0089] Persistent homology calculation is performed on the topological evolution sequence. Homology generation events and homology extinction events are calculated at each filter map snapshot. The threshold position corresponding to the homology generation event is recorded as the bar start point, and the threshold position corresponding to the homology extinction event is recorded as the bar end point, forming a persistent homology barcode set.
[0090] Read the bar start and end points of the persistent homology barcode set one by one to generate the bar duration interval, and count the number of bars within the same homology dimension to generate the quantity distribution;
[0091] Perform concatenation encoding on the bar-shaped continuous interval and quantity distribution to generate a topology-gated vector. The topology-gated vector contains a structural path-gated component group and an operator type-gated component group.
[0092] Write the structure path gated component group into the structure path activation state table, and write the operator type gated component group into the operator type selection state table.
[0093] In this embodiment, performing persistent cohomology calculation specifically includes:
[0094] According to the numerical order of the filtering threshold sequence, each filter graph snapshot in the topological evolution sequence is read sequentially, and the filter graph snapshots are organized into an ordered graph structure sequence according to the threshold position order; using the threshold position order as the evolution index, the structural changes between adjacent filter graph snapshots are analyzed step by step.
[0095] During the parsing process, connectivity is parsed for the graph structure formed by node entries and edge entries in each filtered graph snapshot. The topological structural unit that appears for the first time at the current threshold position is extracted, and the occurrence of the structural unit is marked as a homology generation event. At the same time, the current filtered graph snapshot is compared with the filtered graph snapshot corresponding to the previous threshold position to identify the topological structural units that no longer maintain connectivity or no longer satisfy the structural conditions at the current threshold position, and the disappearance of the structural unit is marked as a homology extinction event.
[0096] For each labeled homology generation event, its corresponding filter threshold position is recorded as the starting point of the bar; for the homology extinction event corresponding to that homology generation event, its corresponding filter threshold position is recorded as the ending point of the bar. For homology generation events that have not yet extinguished at the end of the topological evolution sequence, the threshold position corresponding to the end of the sequence is taken as the ending point of the bar.
[0097] In this embodiment, the topology-gated vector constrains the structural path activation state and operator type selection state in the large model, specifically including:
[0098] The topology-gated vector is read and parsed according to the preset vector positions to obtain the structural path-gated component group and the operator type-gated component group. The position parsing involves reading the topology-gated vector and the pre-configured vector position partitioning rules, performing a continuous index interval truncation operation on the topology-gated vector according to the start and end position indices corresponding to the structural path-gated component group and the operator type-gated component group, generating the structural path-gated component group and the operator type-gated component group, and writing the structural path-gated component group and the operator type-gated component group into their respective data buffers to complete the position parsing of the topology-gated vector.
[0099] Read the structure path identifier table that records structure path identifiers in the large model. The structure path identifier table contains structure path identifiers and structure path position indexes. Read the component values of the structure path gated component group one by one. Locate the structure path identifier in the structure path identifier table according to the structure path position index. Write the component values and structure path identifiers into the structure path activation state table. The structure path activation state table contains structure path identifiers and activation mark values.
[0100] Read the operator type identifier table that records operator type identifiers in the large model. The operator type identifier table contains operator type identifiers and operator type position indexes. Read the component values of the operator type gated component group component by component. Locate the operator type identifier in the operator type identifier table according to the operator type position index. Write the component values and operator type identifiers into the operator type selection status table. The operator type selection status table contains operator type identifiers and selection flag values.
[0101] Read the structure path activation status table, filter out structure path identifiers whose activation flag values do not meet the activation conditions, and retain structure path identifiers that meet the activation conditions to form the structure path allowed set; read the operator type selection status table, filter out operator type identifiers whose selection flag values do not meet the selection conditions, and retain operator type identifiers that meet the selection conditions to form the operator type allowed set.
[0102] Within the scope of the allowed set of structural paths, read the structural paths in the large model computational structure, construct a candidate computational structure set, and within the scope of the allowed set of operator types, read the operator types, construct operator candidate set constraints.
[0103] In this embodiment, S4 specifically includes:
[0104] Read the structure path activation status table and operator type selection status table; read the structure path identifier table and operator type identifier table in the large model calculation structure.
[0105] In the structure path activation status table, filter structure path identifiers whose activation flag values meet the activation conditions to form a candidate structure path identifier set;
[0106] In the operator type selection status table, select operator type identifiers whose flag values meet the selection criteria to form a candidate operator type identifier set;
[0107] Read the structural path identifiers one by one from the candidate structural path identifier set, extract the corresponding structural path substructures from the large model calculation structure according to the structural path position index in the structural path identifier table, and write them into the candidate calculation structure set entries.
[0108] Write the candidate operator type identifier set into the candidate computation structure set entries to form the candidate computation structure set.
[0109] In this embodiment, S5 specifically includes:
[0110] Read the set of weight tensors in the candidate computation structure set, assign a weight tensor identifier to each weight tensor, and then establish an index relationship between the weight tensor identifier and the weight tensor storage location.
[0111] Read the topology gate vector, parse the topology gate vector according to the preset vector position, extract the topology complexity components associated with the weight tensor identifier, and form a topology complexity distribution record at the weight tensor level;
[0112] For each weight tensor, based on the dimensional order and storage layout of the weight tensor, the index space of the weight tensor is divided into several non-overlapping index intervals. The index intervals are marked according to the starting and ending positions of the indexes, forming a set of index intervals.
[0113] Read the index interval markers for each interval of the index interval set, read the topology complexity components of the corresponding index intervals from the topology complexity distribution record, map the topology complexity components to rank parameter values, and form an index interval rank parameter table. The index interval rank parameter table contains index interval markers and rank parameter values.
[0114] According to the index interval rank parameter table, tensor decomposition operation is performed on the weight tensor in each index interval. The tensor decomposition operation independently generates a corresponding decomposition kernel tensor group for each index interval. The decomposition kernel tensor group contains kernel tensors with the same value as the rank parameter.
[0115] The decomposition kernel tensor groups generated for the same weight tensor in different index intervals are aggregated according to the index interval label to form the decomposition kernel tensor set corresponding to the weight tensor. The decomposition kernel tensor set is then written into the candidate computation structure set.
[0116] In this embodiment, performing tensor decomposition specifically includes:
[0117] The index interval rank parameter table is read, and each index interval marker and its corresponding rank parameter value are read one by one. Based on the index interval marker, the contiguous storage area in the weight tensor corresponding to the index interval is determined. Subsequently, the sub-tensor corresponding to the index interval marker is extracted from the weight tensor. The sub-tensor maintains the same dimensional order as the weight tensor, but its index range is limited to the corresponding index interval.
[0118] For each extracted sub-tensor, the rank parameter value corresponding to that index interval is read, and this rank parameter value is used as a constraint on the decomposition scale to perform tensor decomposition operations on the sub-tensor. The tensor decomposition operation decomposes the sub-tensor into several low-dimensional kernel tensors while preserving the original dimensional order of the sub-tensor. The number and dimensional configuration of each low-dimensional kernel tensor correspond one-to-one with the rank parameter value. For the same index interval, tensor decomposition operations are performed only within that index interval, and the decomposition results are not shared with sub-tensors of other index intervals.
[0119] After completing the tensor decomposition of a single index interval, the generated low-dimensional kernel tensors are grouped according to the index interval labels to form the decomposition kernel tensor group corresponding to that index interval. The above sub-tensor extraction, rank parameter reading, and tensor decomposition operations are repeated for all index intervals contained in the weight tensor, generating the decomposition kernel tensor group corresponding to each index interval. Finally, the decomposition kernel tensor groups generated for the same weight tensor in each index interval are collected to form the decomposition kernel tensor set corresponding to that weight tensor, and the decomposition kernel tensor set is written into the candidate computational structure set.
[0120] In this embodiment, S6 specifically includes:
[0121] Read the candidate computation structure set, read the structure path substructure in the candidate computation structure set one by one, and then locate the weight tensor placeholder corresponding to the weight tensor identifier in each structure path substructure.
[0122] Read the set of decomposition kernel tensors corresponding to the weight tensor identifiers. The set of decomposition kernel tensors contains several groups of decomposition kernel tensors organized by index interval labels.
[0123] For each weight tensor occupancy position, the decomposition kernel tensor group of the corresponding index interval is read sequentially according to the index interval marking order. Based on the original dimension order of the weight tensor and the index interval boundary, the decomposition kernel tensor group is mapped to the corresponding index interval position to form an index interval-level reparameterized weight sub-tensor.
[0124] Several index interval-level reparameterized weight sub-tensors formed under the same weight tensor identifier are concatenated according to the index interval boundary order to generate a reparameterized weight tensor.
[0125] Write the reparameterized weight tensor into the placeholder position of the weight tensor in the structure path substructure, and replace the corresponding weight tensor in the candidate computation structure set.
[0126] Repeatedly perform the weight tensor placeholder replacement operation on all structural path substructures in the candidate computational structure set to form a large model computational structure containing reparameterized weight tensors.
[0127] Example 1:
[0128] To verify the feasibility of this invention in implementation, it was applied to an operating environment containing multiple computing nodes and multiple communication links. In this operating environment, different computing nodes differ in computing power, memory capacity, and inter-node communication conditions. Traditional deployment methods typically use a fixed model structure directly mapped to the operating environment, lacking characterization of the operating environment's topology. This can easily lead to problems such as concentrated load on some nodes, communication path congestion, and mismatch between the model structure and the operating environment's state.
[0129] In this scenario, the status of each computing node in the runtime environment is first collected, including its computing power capacity, memory capacity, communication bandwidth, and communication latency between nodes, and a runtime environment graph structure is constructed based on this data. Subsequently, a set of nodes and edges is formed within the runtime environment graph structure, and state feature vectors are configured for each node and edge to obtain a topological computation graph. Multi-scale filtering is performed on the topological computation graph to form a topological evolution sequence, and persistent cohomology computation is further performed to generate a set of persistent cohomology barcodes. By encoding the barcode persistence intervals and quantity distribution, a topological gating vector is formed, which constrains the activation states of structural paths and the operator type selection states within the large model.
[0130] Under topological gating vector constraints, structural paths that meet the runtime environment conditions are selected from the original model computational structures to construct a candidate computational structure set. For the weight tensors in the candidate computational structure set, index intervals are partitioned according to the topological complexity distribution, and different rank parameters are configured for different index intervals, performing heterogeneous rank tensor decomposition. Subsequently, tensor reparameterization processing is performed on the candidate computational structures based on the decomposed kernel tensor set, forming a large model computational structure adapted to the runtime environment, and this computational structure is mapped to the corresponding computational nodes for deployment and execution.
[0131] Through multiple rounds of comparative operation, it can be observed that, compared with the traditional fixed deployment method, the present invention shows significant differences in node load distribution, communication path utilization efficiency, and overall operational stability, verifying the feasibility of the method of the present invention in complex operating environments.
[0132] Table 1: Quantitative Comparison Data of Adaptive Deployment in Operating Environment
[0133] Comparison indicators Traditional deployment method Method of the present invention Numerical variation range Stability state Node load standard deviation 0.38 0.17 Decrease of 0.21 Stablize Maximum load deviation of nodes 0.62 0.29 Decrease of 0.33 Stablize Average communication latency fluctuation 0.44 0.19 Decrease of 0.25 Stablize Communication path congestion rate 0.31 0.08 Decrease of 0.23 Stablize Continuous operation structure adjustment ratio 0.05 0.18 Up 0.13 Stablize
[0134] Table 1 presents a quantitative comparison of the operational status under the same operating environment conditions, comparing the traditional deployment method with the method of this invention. To avoid direct numerical differences caused by different scales of operating environments, all indicators in the table are expressed as normalized relative values to reflect the changing trends and stability of the operational status under different deployment methods.
[0135] The node load standard deviation is used to characterize the dispersion of load distribution among computing nodes during operation. A larger value indicates a more significant load difference between different nodes. The data in the table shows that the node load standard deviation remains at a high level under traditional deployment methods, indicating an uneven distribution of the model's computational structure among nodes. However, after adopting the method of this invention, this indicator is significantly reduced, indicating a more balanced distribution of model computational tasks among different nodes and an effective mitigation of load differences between nodes.
[0136] The maximum load deviation of a node reflects the degree of deviation between the busiest node and the overall average load. A large value for this indicator can easily lead to local nodes being under high load for extended periods. The comparison results in the table show that the method of this invention has a significantly lower value for this indicator than the traditional deployment method. This indicates that by using the runtime environment graph structure and topology gating constraints, the large model computation structure can avoid centralized mapping of at least a few nodes, thereby reducing the risk of local node overload.
[0137] The average communication latency fluctuation amplitude is used to describe the stability of the communication state during operation. The larger the value, the more significantly the communication path is affected by topology changes. The data in the table show that the communication latency fluctuation amplitude is large under traditional deployment methods, while this indicator decreases significantly after adopting the method of this invention. This indicates that by performing multi-scale analysis on the topology computation graph and introducing a topology gating mechanism, the model structure selection can be made more consistent with changes in communication conditions, thereby reducing the fluctuation of the communication state.
[0138] The communication path congestion rate reflects the frequency of high-pressure communication links during operation. Comparative results show that the congestion rate corresponding to the method of this invention is significantly lower than that of traditional deployment methods, indicating that the computational structure generated under topology-gated vector constraints can effectively avoid paths with limited communication conditions and reduce the concentrated occupation of communication resources.
[0139] The continuous operation structure adjustment ratio reflects the frequency with which the model adjusts its structure during operation. This indicator shows a high value under the method of this invention, indicating that the model's computational structure can continuously adjust according to changes in the operating environment. In contrast, traditional deployment methods, due to their fixed structure, maintain a lower level for this indicator. Combining the aforementioned indicators, it can be seen that this invention achieves adaptive response to changes in the operating environment while maintaining operational stability.
[0140] The comparison results of the various quantitative indicators in Table 1 show that the present invention exhibits more coordinated operating characteristics in terms of node load distribution, communication status stability, and structural adjustment flexibility, verifying the effectiveness and rationality of the method of the present invention in complex operating environments.
[0141] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. An adaptive deployment method for a large-scale model runtime environment based on strategy optimization, characterized in that, Includes the following steps: S1. Collect the computing power capacity, video memory capacity, inter-node communication bandwidth, and communication latency values of the computing nodes in the operating environment, and construct the operating environment graph structure. S2. Construct a set of nodes and a set of edges in the runtime environment graph structure to form a topological computation graph, and then configure state feature vectors for the nodes and edges; S3. Perform multi-scale filtering on the topology computation graph to generate a topology evolution sequence. Perform persistent cohomology computation on the topology evolution sequence to obtain a persistent cohomology barcode set. Map the bar persistence intervals and quantity distribution in the persistent cohomology barcode set to a topology gating vector. The topology gating vector limits the structural path activation state and operator type selection state in the large model. S4. Generate a set of candidate computational structures under the constraint of topological gating vectors; S5. For the set of weight tensors in the candidate computational structure set, according to the topological complexity distribution represented by the topological gate vector, the same weight tensor is divided into several index intervals, different rank parameters are configured for different index intervals, and topology-induced heterogeneous rank tensor decomposition is performed to generate a set of decomposed kernel tensors. S6. Perform tensor reparameterization on the candidate computational structure set based on the decomposed kernel tensor set to form a large model computational structure; S7. Map the large model computation structure to the corresponding computation nodes of the runtime environment graph structure and then complete the deployment and execution.
2. The adaptive deployment method for a large model runtime environment based on strategy optimization according to claim 1, characterized in that, S1 specifically includes: Assign a unique computing node identifier to each computing node in the operating environment, read the computing power capacity value and video memory capacity value corresponding to each computing node identifier, and construct a node status record table using the computing node identifier, computing power capacity value, and video memory capacity value. For any two computing nodes, a communication link record is established. The communication link record includes the identifier of the sending computing node, the identifier of the receiving computing node, the communication bandwidth value, and the communication delay value. Based on this, an edge state record table is constructed. A set of node identifiers for the runtime environment graph structure is generated using the compute node identifiers in the node state record table, and a set of edge identifiers for the runtime environment graph structure is generated using the sender compute node identifiers and receiver compute node identifiers in the edge state record table. Establish a one-to-one index mapping relationship between the node identifier set and the node state record table, and establish a one-to-one index mapping relationship between the edge identifier set and the edge state record table to form the runtime environment graph structure.
3. The adaptive deployment method for a large model runtime environment based on strategy optimization according to claim 2, characterized in that, S2 specifically includes: Read the node identifier set and edge identifier set in the runtime environment graph structure, and build a data structure container for the topology computation graph; Write a node entry for each node identifier in the data structure container. The node entry contains the node identifier and a state feature vector consisting of computing power capacity value and video memory capacity value. Write an edge entry for each edge identifier in the data structure container. The edge entry contains the identifier of the sending end computing node, the identifier of the receiving end computing node, and a state feature vector composed of communication bandwidth value and communication delay value, thus completing the construction of the topology computing graph.
4. The adaptive deployment method for a large model runtime environment based on strategy optimization according to claim 3, characterized in that, S3 specifically includes: Read the data structure container of the topological computation graph, extract node entries and edge entries to form an initial graph snapshot; Define a filter threshold sequence, which contains at least three threshold positions; At each threshold position, threshold filtering is performed on the communication bandwidth and communication delay values in the edge entries. Edge entries that meet the threshold relationship are retained, and edge entries that do not meet the threshold relationship are deleted, forming a filter graph snapshot at the corresponding threshold position. Arrange the snapshots of each filter graph in order of the filter threshold sequence to form a topological evolution sequence; Persistent homology calculation is performed on the topological evolution sequence. Homology generation events and homology extinction events are calculated at each filter map snapshot. The threshold position corresponding to the homology generation event is recorded as the bar start point, and the threshold position corresponding to the homology extinction event is recorded as the bar end point, forming a persistent homology barcode set. Read the bar start and end points of the persistent homology barcode set one by one to generate the bar duration interval, and count the number of bars within the same homology dimension to generate the quantity distribution; Perform concatenation encoding on the bar-shaped continuous interval and quantity distribution to generate a topology-gated vector. The topology-gated vector contains a structural path-gated component group and an operator type-gated component group. Write the structure path gated component group into the structure path activation state table, and write the operator type gated component group into the operator type selection state table.
5. The adaptive deployment method for a large model runtime environment based on strategy optimization according to claim 4, characterized in that, The topological gating vector constrains the structural path activation state and operator type selection state in the large model, specifically including: Read the topology gate vector, parse the topology gate vector according to the preset vector position, and obtain the structural path gate component group and the operator type gate component group; Read the structure path identifier table that records structure path identifiers in the large model. The structure path identifier table contains structure path identifiers and structure path position indexes. Read the component values of the structure path gated component group one by one. Locate the structure path identifier in the structure path identifier table according to the structure path position index. Write the component values and structure path identifiers into the structure path activation state table. The structure path activation state table contains structure path identifiers and activation mark values. Read the operator type identifier table that records operator type identifiers in the large model. The operator type identifier table contains operator type identifiers and operator type position indexes. Read the component values of the operator type gated component group component by component. Locate the operator type identifier in the operator type identifier table according to the operator type position index. Write the component values and operator type identifiers into the operator type selection status table. The operator type selection status table contains operator type identifiers and selection flag values. Read the structure path activation status table, filter out structure path identifiers whose activation flag values do not meet the activation conditions, and retain structure path identifiers that meet the activation conditions to form the structure path allowed set; read the operator type selection status table, filter out operator type identifiers whose selection flag values do not meet the selection conditions, and retain operator type identifiers that meet the selection conditions to form the operator type allowed set. Within the scope of the allowed set of structural paths, read the structural paths in the large model computational structure, construct a candidate computational structure set, and within the scope of the allowed set of operator types, read the operator types, construct operator candidate set constraints.
6. The adaptive deployment method for a large model runtime environment based on strategy optimization according to claim 5, characterized in that, S4 specifically includes: Read the structure path activation status table and operator type selection status table; read the structure path identifier table and operator type identifier table in the large model calculation structure. In the structure path activation status table, filter structure path identifiers whose activation flag values meet the activation conditions to form a candidate structure path identifier set; In the operator type selection status table, select operator type identifiers whose flag values meet the selection criteria to form a candidate operator type identifier set; Read the structural path identifiers one by one from the candidate structural path identifier set, extract the corresponding structural path substructures from the large model calculation structure according to the structural path position index in the structural path identifier table, and write them into the candidate calculation structure set entries. Write the candidate operator type identifier set into the candidate computation structure set entries to form the candidate computation structure set.
7. The adaptive deployment method for a large model runtime environment based on strategy optimization according to claim 6, characterized in that, S5 specifically includes: Read the set of weight tensors in the candidate computation structure set, assign a weight tensor identifier to each weight tensor, and then establish an index relationship between the weight tensor identifier and the weight tensor storage location. Read the topology gate vector, parse the topology gate vector according to the preset vector position, extract the topology complexity components associated with the weight tensor identifier, and form a topology complexity distribution record at the weight tensor level; For each weight tensor, based on the dimensional order and storage layout of the weight tensor, the index space of the weight tensor is divided into several non-overlapping index intervals. The index intervals are marked according to the starting and ending positions of the indexes, forming a set of index intervals. Read the index interval markers for each interval of the index interval set, read the topology complexity components of the corresponding index intervals from the topology complexity distribution record, map the topology complexity components to rank parameter values, and form an index interval rank parameter table. The index interval rank parameter table contains index interval markers and rank parameter values. According to the index interval rank parameter table, tensor decomposition operation is performed on the weight tensor in each index interval. The tensor decomposition operation independently generates a corresponding decomposition kernel tensor group for each index interval. The decomposition kernel tensor group contains kernel tensors with the same value as the rank parameter. The decomposition kernel tensor groups generated for the same weight tensor in different index intervals are aggregated according to the index interval label to form the decomposition kernel tensor set corresponding to the weight tensor. The decomposition kernel tensor set is then written into the candidate computation structure set.
8. The adaptive deployment method for a large model runtime environment based on strategy optimization according to claim 7, characterized in that, S6 specifically includes: Read the candidate computation structure set, read the structure path substructure in the candidate computation structure set one by one, and then locate the weight tensor placeholder corresponding to the weight tensor identifier in each structure path substructure. Read the set of decomposition kernel tensors corresponding to the weight tensor identifiers. The set of decomposition kernel tensors contains several groups of decomposition kernel tensors organized by index interval labels. For each weight tensor occupancy position, the decomposition kernel tensor group of the corresponding index interval is read sequentially according to the index interval marking order. Based on the original dimension order of the weight tensor and the index interval boundary, the decomposition kernel tensor group is mapped to the corresponding index interval position to form an index interval-level reparameterized weight sub-tensor. Several index interval-level reparameterized weight sub-tensors formed under the same weight tensor identifier are concatenated according to the index interval boundary order to generate a reparameterized weight tensor. Write the reparameterized weight tensor into the placeholder position of the weight tensor in the structure path substructure, and replace the corresponding weight tensor in the candidate computation structure set. Repeatedly perform the weight tensor placeholder replacement operation on all structural path substructures in the candidate computational structure set to form a large model computational structure containing reparameterized weight tensors.